Remove Watermark Wondershare

PDFelement
Elective I Laboratory T.E.(Sem-V) [2022-23]

Elective I Laboratory
Course Code Course Name Teaching Scheme Credits
(Hrs./ week)
Elective I Laboratory Pattern
317522(C) 2 01
Recognition

Course Objectives:
• To understand the fundamentals of pattern recognition.
• To Study the syntactic approach in pattern recognition.
• To study statistical approaches in pattern recognition.
• To study artificial neural network-based pattern recognition

Course Outcomes:
On completion of the course, the learner will be able to–
• CO1: Apply statistical pattern recognition approaches.
• CO2: Implement different approaches to syntactic pattern recognition.
• CO3: Develop an artificial neural network-based pattern recognition system.

5 Department of Artificial Intelligence and Data Science, DYPCOE, Akurdi, Pune

 Remove Watermark Wondershare
PDFelement
Elective I Laboratory T.E.(Sem-V) [2022-23]

Table of Contents
Sr. Title of Experiment CO PO/PSO
No Mapping mapped
Group A
1 • Use Bayesian Decision theory of statistical pattern recognition to CO 1 PO-1, PO-2,
classify the object PO-4, PO-5
,PO-9, PO-
11, PO-12

PSO-1,PSO-2,
PSO-3
2 • Implement Cocke–Younger–Kasami (CYK) Parsing Algorithm using CO 2 PO-1, PO-2,
Syntactic Pattern Recognition PO-4, PO-5
,PO-9, PO-
11, PO-12

PSO-1,PSO-2,
PSO-3
Group B
3 • Generate a Pattern from String using a syntactical Pattern Approach CO 2 PO-1, PO-2,
PO-4, PO-5
,PO-9, PO-
11, PO-12

PSO-1,PSO-2,
PSO-3
4 • Apply suitable pattern recognition technique to perform Character CO 2 PO-1, PO-2,
Recognition PO-4, PO-5
,PO-9, PO-
11, PO-12

Group C PSO-1,PSO-2,
5 • Develop a system for Handwritten Digit Recognition using Neural CO 3 PSO-3
PO-1, PO-2,
Network PO-3
PO-4, PO-5
,PO-9, PO-
11, PO-12

6 Department of Artificial Intelligence and Data Science, DYPCOE, Akurdi, Pune

 Remove Watermark Wondershare
PDFelement
Elective I Laboratory T.E.(Sem-V) [2022-23]

Group A

7 Department of Artificial Intelligence and Data Science, DYPCOE, Akurdi, Pune

 Remove Watermark Wondershare
PDFelement
Elective I Laboratory T.E.(Sem-V) [2022-23]

ASSIGNMENT NO: 01
Problem Statement: Use Bayesian Decision theory of statistical pattern recognition to classify the
object.

Title: Use Bayesian Decision theory of statistical pattern recognition to classify the object

Objective:
• Understand and implement Bayesian Decision theory
• Analyze statistical pattern recognition

Outcomes:
• Ability to apply Bayesian Decision theory for problem-solving method and knowledge
representation techniques.

Software Required:
• Python 3.0 or extended version.

Theory:
Introduction
Bayesian decision theory refers to the statistical approach based on tradeoff quantification among
various classification decisions based on the concept of Probability (Bayes Theorem) and the costs
associated with the decision. It is basically a classification technique that involves the use of the Bayes
Theorem which is used to find the conditional probabilities.

In Statistical Pattern Recognition, to will focus on the statistical properties of patterns that are
generally expressed in probability densities (pdf’s and pmf’s), and this will command most of the
attention in this article and try to develop the fundamentals of the Bayesian decision theory. A random
variable is a function that maps a possible set of outcomes to some values like while tossing a coin and
getting head H as 1 and Tail T as 0 where 0 and 1 are random variables.

Bayes Theorem
The conditional probability of A given B, represented by P(A | B) is the chance of occurrence of A
given that B has occurred.
P(A | B) = P(A,B)/P(B) or
By Using the Chain rule, this can also be written as:
8 Department of Artificial Intelligence and Data Science, DYPCOE, Akurdi, Pune
 Remove Watermark Wondershare
PDFelement
Elective I Laboratory T.E.(Sem-V) [2022-23]

P(A,B) = P(A|B)P(B)=P(B|A)P(A)
P(A | B) = P(B|A)P(A)/P(B) ——- (1)
Where, P(B) = P(B,A) + P(B,A’) = P(B|A)P(A) + P(B|A’)P(A’)
Here, equation (1) is known as the Bayes Theorem of probability
The aim is to explore each of the components included in this theorem. Let’s explore step by step:

(a) Prior or State of Nature:

• Prior probabilities represent how likely is each Class is going to occur.
• Priors are known before the training process.
• The state of nature is a random variable P(wi).
• If there are only two classes, then the sum of the priors is P(w1) + P(w2)=1, if the classes are exhaustive.

(b) Class Conditional Probabilities:

• It represents the probability of how likely a feature x occurs given that it belongs to the particular class.
It is denoted by, P(X|A) where x is a particular feature
• It is the probability of how likely the feature x occurs given that it belongs to the class wi.
• Sometimes, it is also known as the Likelihood.
• It is the quantity that to have to evaluate while training the data. During the training process, to have
input(features) X labeled to corresponding class w and to figure out the likelihood of occurrence of that
set of features given the class label.

(c) Evidence:

• It is the probability of occurrence of a particular feature i.e. P(X).

• It can be calculated using the chain rule as, P(X) = Σin P(X | wi) P(wi)
• As to need the likelihood of class conditional probability is also figure out evidence values during
training.

(d) Posterior Probabilities:

• It is the probability of occurrence of Class A when certain Features are given

• It is what to aim at computing in the test phase in which to have testing input or features (the given
entity) and have to find how likely trained model can predict features belonging to the particular class
wi. For a better understanding of the above theory, to consider an example

9 Department of Artificial Intelligence and Data Science, DYPCOE, Akurdi, Pune

 Remove Watermark Wondershare
PDFelement
Elective I Laboratory T.E.(Sem-V) [2022-23]

Problem Description

Suppose to have a classification problem statement where to have to classify among the object-1 and
object-2 with the given set of features X = [x1, x2, …, xn]T.

The main objective of designing a such classifier is to suggest actions when presented with
unseen features, i.e., object not yet seen i.e., not in training data.
In this example let w denotes the state of nature with w = w1 for object-1 and w = w2 for
object-2. Here, to need to know that in reality, the state of nature is so unpredictable that to
generally consider that was variable that is described probabilistically.

Priors
• Generally, to assume that there is some prior value P(w1) that the next object is object-1 and
P(w2) that the next object is object-2. If to have no other object as in this problem than the
sum of their prior is 1 i.e. the priors are exhaustive.
• The prior probabilities reflect the prior knowledge of how likely to will get object-1 and
object-2. It is domain-dependent as the prior may change based on the time of year they are
being caught.
It sounds somewhat strange and when judging multiple objects (as in a more realistic
scenario) makes this decision rule stupid as to always make the same decision based on the
largest prior even though to know that any other type of objective also might appear
governed by the leftover prior probabilities (as priors are exhaustive in nature).

Consider the following different scenarios:

• If P(ω1)>>> P(ω2), the decision in favor of ω1 will be correct most of the time to predict.
• But if P(ω1) = P(ω2), half probable of the prediction of being right. In general,the probability
of error is the minimum of P(ω1) and P(ω2), and later in this article, to will see that under
these conditions no other decision rule can yield a larger probability of being correct.

Feature Extraction process (Extract feature from the images)

10 Department of Artificial Intelligence and Data Science, DYPCOE, Akurdi, Pune

 Remove Watermark Wondershare
PDFelement
Elective I Laboratory T.E.(Sem-V) [2022-23]

A suggested set of features- Length, width, shapes of an object, etc.In the example, to use
the width x, which is more discriminatory to improve the decision rule of the classifier.
The different objects will yield different variable-width readings and to usually see this
variability in probabilistic terms and also to consider x to be a continuous random variable
whose distribution depends on the type of object wj, and is expressed as p(x|ωj) (probability
distribution function pdf as a continuous variable) and known as the class-conditional
probability density function. Therefore, The pdf p(x|ω1) is the probability density function
for feature x given that the state of nature is ω1 and the same interpretation for p(x|w2).

Fig. Picture Showing pdf for both classes

Image Source: Google Images

Suppose that to are toll aware of both the prior probabilities P(ω j) and the conditional
densities p(x|ωj). Now, to can arrive at the Bayes formula for finding posterior probabilities:

11 Department of Artificial Intelligence and Data Science, DYPCOE, Akurdi, Pune

 Remove Watermark Wondershare
PDFelement
Elective I Laboratory T.E.(Sem-V) [2022-23]

Fig. Formula of Bayes Theorem

Image Source: Google Images

Bayes’ formula gives us intuition that by observing the measurement of x to can convert the
prior P(ωj) to the posteriors, denoted by P(ωj|x) which is the probability of ωj given that
feature value x has been measured.
p(x|ωj) is known as the likelihood of ωj with respect to x.The evidence factor, p(x), works as
merely a scale factor that guarantees that the posterior probabilities sum up to one for all the
classes.

Bayes’ Decision Rule

The decision rule given the posterior probabilities is as follows
If P(w1|x) > P(w2|x) to would decide that the object belongs to class w1, or else class w2.
Probability of Error

To justify the decision to look at the probability of error, whenever to observe x, to have,

P(error|x) = P(w1|x) if to decide w2, and P(w2|x) if to decide w1

As they are exhaustive and if to choose the correct nature of an object by probability P then the leftover
probability (1-P) will show how probable is the decision that it the not the decided object.
To can minimize the probability of error by deciding the one which has a greater posterior and the rest
as the probability of error will be minimum as possible. So to finally get,
P (error|x) = min [P(ω1|x), P(ω2|x)] And the Bayes decision rule as, Decide ω1 if P(ω1|x) >P(ω2|x);
otherwise decide ω2
This type of decision rule highlights the role of the posterior probabilities. With the help Bayes theorem,
to can express the rule in terms of conditional and prior probabilities.

12 Department of Artificial Intelligence and Data Science, DYPCOE, Akurdi, Pune

 Remove Watermark Wondershare
PDFelement
Elective I Laboratory T.E.(Sem-V) [2022-23]

The evidence is unimportant as far as the decision is concerned. As to discussed earlier it is working as
just a scale factor that states how frequently to will measure the feature with value x; it assures P(ω1|x)
+ P(ω2|x) = 1.

So by eliminating the unrequired scale factor in the decision rule to have, the similar
decision rule by Bayes theorem as,

Decide ω1 if p(x|ω1)P(ω1) >p(x|ω2)P(ω2); otherwise decide ω2

Now, let’s consider 2 cases:

• Case-1: If class conditionals are equal i.e, p(x|ω1)= p(x|ω2), then to arrive at the premature
decision rule governed by just priors.
• Case-2: On the other hand, if priors are equal i.e, P(ω1)= P(ω2) then the decision is entirely
based on class conditionals p(x|ωj).

This completes the example formulation!

Generalization of the preceding ideas for Multiple Features and Classes

Bayes classification: Posterior, likelihood, prior, and evidence

P(wi | X)= P(X | wi) P(wi) / P(X)

Posterior = Likelihood* Prior/Evidence

To now discuss those cases which have multiple features as toll as multiple classes,

Let the Multiple Features be X1, X2, … Xn and Multiple Classes be w1, w2, … wn, then:

P(wi | X1, …. Xn) = P(X1,…. , Xn|wi)*P(wi)/P(X1,… Xn)

Where,

Posterior = P (wi | X1, …. Xn)

13 Department of Artificial Intelligence and Data Science, DYPCOE, Akurdi, Pune

 Remove Watermark Wondershare
PDFelement
Elective I Laboratory T.E.(Sem-V) [2022-23]

Likelihood = P(X1,…. , Xn|wi)

Prior = P(wi)

Evidence = P(X1,… ,Xn)

In cases of the same incoming patterns, to might need to use a drastically different cost
function, which will lead to different actions altogether. Generally, different decision tasks
may require features and yield boundaries quite different from those useful for the original
categorization problem.

To observe that the posterior distribution is more concentrated to the specific values of the
parameter that explain the dataset toll. In particular, to observe the same two peaks
around θ=−0.5 and θ=2 observed in the maximum likelihood exercise.

Inputs:

Data Set
i) Feature Vector
Implementation:

Algorithm:

Step 1. Calculate prior probability

Step 2. Calculate likelihood probability.

Step 3. Find posterior probability (Bayes decision)

Step 4. Build a classifier based on the discriminant function.

Pseudo code:

14 Department of Artificial Intelligence and Data Science, DYPCOE, Akurdi, Pune

 Remove Watermark Wondershare
PDFelement
Elective I Laboratory T.E.(Sem-V) [2022-23]

• Implement the prior and posterior functions below. These functions receive as input a
vector of parameters θ (assumed to be sorted from smallest to largest, linearly spaced, and
covering the range of values where most of the probability mass lies). The posterior
function also receives a dataset D as input. Both functions return a vector containing the
probability scores associated to each value of θ.

def prior(THETA):

return pdf(0,THETA/10)/10
def posterior(D,THETA):

Building a Classifier

To now would like to build a Bayes classifier based on the discriminant function

h(x)=log⁡P(x|D1)−log⁡P(x|D2)+log⁡P(ω1)−log⁡P(ω2)
where the dataset-conditioned densities are obtained from the original data density model and the
parameter posterior as

p(x|Dj)=∫p(x|θ)p(θ|Dj)dθ

Task:

• Implement a function fit that produces the parameter posteriors p(θ|D1) and p(θ|D2).
• Implement a function predict computing the new discriminant function h based on the dataset-
conditioned data densities.
• class BayesClassifier

Output:

Classification of data object.

Conclusion: Thus, to have studied and implemented successfully Bayesian Decision Theory of
statistical pattern recognition.

15 Department of Artificial Intelligence and Data Science, DYPCOE, Akurdi, Pune

 Remove Watermark Wondershare
PDFelement
Elective I Laboratory T.E.(Sem-V) [2022-23]

ASSIGNMENT No: 02

Problem Statement: Understand Cocke–Younger–Kasami (CYK) Parsing Algorithm using

Syntactic Pattern Recognition

Title: Implement Cocke–Younger–Kasami (CYK) Parsing Algorithm using Syntactic Pattern

Recognition

Objective: Implement Cocke–Younger–Kasami (CYK) Parsing Algorithm using Syntactic Pattern

Recognition

• Understand and implement Cocke–Younger–Kasami (CYK ) Parsing Algorithm

• Analyze Syntactic pattern recognition

Outcomes:
• Ability to apply Cocke–Younger–Kasami (CYK ) Parsing Algorithm using Syntactic Pattern
Recognition for problem-solving method and knowledge representation technique.

Software Required:
• Python 3.0 or extended version.

Theory:
In computer science, the Cocke–Younger–Kasami algorithm (alternatively called CYK, or CKY) is
a parsing algorithm for context-free grammars published by Itiroo Sakai in 1961.[1] The algorithm is
named after some of its rediscoverers: John Cocke, Daniel Younger, Tadao Kasami, and Jacob T.
Schwartz. It employs bottom-up parsing and dynamic programming.
The standard version of CYK operates only on context-free grammars given in Chomsky normal
form (CNF). Hotover, any context-free grammar may be transformed (after convention) to a CNF
grammar expressing the same language.
The importance of the CYK algorithm stems from its high efficiency in certain situations.
Using big O notation, the worst case running time of CYK is, where is the length of the parsed string
and is the size of the CNF grammar This makes it one of the most efficient parsing algorithms in terms
of worst-case asymptotic complexity, although other algorithms exist with better average running time
in many practical scenarios.

Standard form

16 Department of Artificial Intelligence and Data Science, DYPCOE, Akurdi, Pune

 Remove Watermark Wondershare
PDFelement
Elective I Laboratory T.E.(Sem-V) [2022-23]

The dynamic programming algorithm requires the context-free grammar to be rendered into Chomsky
normal form (CNF), because it tests for possibilities to split the current sequence into two smaller
sequences. Any context-free grammar that does not generate the empty string can be represented in
CNF using only production rules of the forms , , and where is the start symbol.

As prose
In informal terms, this algorithm considers every possible substring of the input string and sets to be
true if the substring of length starting fromcan be generated from the nonterminal. Once it has
considered substrings of length 1, it goes on to substrings of length 2, and so on. For substrings of
length 2 and greater, it considers every possible partition of the substring into two parts, and checks to
see if there is some production that matches the first part and matches the second part. If so, it records as
matching the whole substring. Once this process is completed, the input string is generated by the
grammar if the substring containing the entire input string is matched by the start symbol.

Example

Sentence parsing using the CYK algorithm

This is an example grammar:

17 Department of Artificial Intelligence and Data Science, DYPCOE, Akurdi, Pune

 Remove Watermark Wondershare
PDFelement
Elective I Laboratory T.E.(Sem-V) [2022-23]

Now the sentence she eats a fish with a fork is analyzed using the CYK algorithm. In the following
table, in , i is the number of the row (starting at the bottom at 1), and j is the number of the column
(starting at the left at 1).

CYK table

VP

VP PP

S NP NP

NP V, VP Det. N P Det N

she eats a fish with a fork

For readability, the CYK table for P is represented here as a 2-dimensional matrix M containing a set of
non-terminal symbols, such that Rk is in if, and only if, In the above example, since a start symbol S is
in the sentence can be generated by the grammar.
Generating a parse tree
The above algorithm is a recognizer that will only determine if a sentence is in the language. It is simple
to extend it into a parser that also constructs a parse tree, by storing parse tree nodes as elements of the
array, instead of the Boolean 1. The node is linked to the array elements that tore used to produce it, so
as to build the tree structure. Only one such node in each array element is needed if only one parse tree
is to be produced. Hot over, if all parse trees of an ambiguous sentence are to be kept, it is necessary to

18 Department of Artificial Intelligence and Data Science, DYPCOE, Akurdi, Pune

 Remove Watermark Wondershare
PDFelement
Elective I Laboratory T.E.(Sem-V) [2022-23]

store in the array element a list of all the ways the corresponding node can be obtained in the parsing
process. This is sometimes done with a second table B[n,n,r] of so-called back pointers. The end result
is then a shared-forest of possible parse trees, where common trees parts are factored bettoen the various
parses. This shared forest can conveniently be read as an ambiguous grammar generating only the
sentence parsed, but with the same ambiguity as the original grammar, and the same parse trees up to a
very simple renaming of non-terminals, as shown by Lang.

Parsing non-CNF context-free grammars

As pointed out by Lange & Leiß (2009), the drawback of all known transformations into Chomsky
normal form is that they can lead to an undesirable bloat in grammar size. The size of a grammar is the
sum of the sizes of its production rules, where the size of a rule is one plus the length of its right-hand
side. Using denote the size of the original grammar, the size blow-up in the worst case may range from
to , depending on the transformation algorithm used. For the use in teaching, Lange and Leiß propose a
slight generalization of the CYK algorithm, "without compromising the efficiency of the algorithm,
clarity of its presentation, or simplicity of proofs".

Parsing context-free grammars

It is also possible to extend the CYK algorithm to parse strings using stochastic context-free grammars.
(probabilities) are then stored in the table P instead of Booleans, so P[i,j,A] will contain the minimum
(maximum probability) that the substring from i to j can be derived from A. Further extensions of the
algorithm allow all parses of a string to be enumerated from lowest to highest (highest to lowest
probability).

Input:
i) String
ii) Production rules
iii) Terminals and Non terminals

Algorithm:
pseudocode

The algorithm in pseudocode is as follows:

let the input be a string I consisting of n characters: a1 ... an.

let the grammar contain r nonterminal symbols R1 ... Rr, with start symbol R1.

let P[n,n,r] be an array of booleans. Initialize all elements of P to false.

19 Department of Artificial Intelligence and Data Science, DYPCOE, Akurdi, Pune

 Remove Watermark Wondershare
PDFelement
Elective I Laboratory T.E.(Sem-V) [2022-23]

let back[n,n,r] be an array of lists of back pointing triples. Initialize all elements of back to the empty
list.

for each s = 1 to n

for each unit production Rv → as

set P[1,s,v] = true

for each l = 2 to n -- Length of span

for each s = 1 to n-l+1 -- Start of span

for each p = 1 to l-1 -- Partition of span

for each production Ra → Rb Rc

if P[p,s,b] and P[l-p,s+p,c] then

set P[l,s,a] = true,

append <p,b,c> to back[l,s,a]

if P[n,1,1] is true then

I is member of language

return back -- by retracing the steps through back, one can easily construct all possible parse trees of
the string.

else

return "not a member of language"

Output:
Given string is valid in the language of the grammar or not.

Conclusion:
Cocke–Younger–Kasami (CYK) Parsing Algorithm using Syntactic Pattern Recognition implemented.

20 Department of Artificial Intelligence and Data Science, DYPCOE, Akurdi, Pune

 Remove Watermark Wondershare
PDFelement
Elective I Laboratory T.E.(Sem-V) [2022-23]

Group B

21 Department of Artificial Intelligence and Data Science, DYPCOE, Akurdi, Pune

 Remove Watermark Wondershare
PDFelement
Elective I Laboratory T.E.(Sem-V) [2022-23]

ASSIGNMENT No: 03

Problem Statement: To generate a Pattern from String using a syntactical Pattern Approach

Title: Generate a Pattern from String using syntactical Pattern Approach

Objective:
• To understand and implement patterns from the string.
• To analyze the syntactical pattern approach.

Outcomes:
• Ability to choose and apply an appropriate problem-solving method and knowledge
representation technique.

Software Required:
• Python 3.0 or extended version.

Theory:
The principal objective of this experiment is to provide an introduction to basic concepts of syntactic
pattern recognition. The theory of automata and formal languages is one of the principal elements in the
study of digital machines and their processing capabilities. Syntactic pattern recognition employs this
theory in innovative ways to develop pattern recognition approaches that are based on knowledge of the
underlying structure of pattern classes. This chapter presents also the fundamentals of formal language
and automata theory as they apply to pattern recognition.

3.1 Basics Pattern recognition techniques are among the most important tools used in the field of
machine intelligence. Pattern recognition can be defined as the categorization of input data into
identifiable classes via the extraction of significant features or attributes of the data from a background
of irrelevant detail C1353. A pattern is essentially an arrangement. It may be defined as a quantitative or
structural description of an object or some other entity of interest. A pattern class is a set of patterns that
share some common properties. The subject matter of 29 pattern recognition by machine deals with

22 Department of Artificial Intelligence and Data Science, DYPCOE, Akurdi, Pune

 Remove Watermark Wondershare
PDFelement
Elective I Laboratory T.E.(Sem-V) [2022-23]

techniques for assigning patterns to their respective classes, automatically and with as little human
intervention as possible.

The study of pattern recognition problems may be logically divided into two major categories:

1. The study of the pattern recognition capability of human beings and other living organisms.

2. The development of underlying theory and practical techniques for machine implementation of a
given recognition task. The first area falls in the domain of psychology, physiology and biology while
the second area is in the domain of engineering, computer science and applied mathematics.

Approaches to pattern recognition system design may be divided into two principal categories:

(2) the decision theoretic approach ; and

(3) the syntactic approach.

The decision-theoretic approach is based on the utilization of decision functions for classifying pattern
vectors and is ideally suited for applications where patterns can have meaningful representation in
vector form. There are applications, where the structure of a pattern plays an important role in the
classification process. In these situations, the decision theoretic approach has serious drawbacks because
it lacks a suitable formalism for handling pattern structures and their relationships. For example, the
decision—theoretic approach finds few applications to ECG analysis, since in this case the structure and
relationships of the various components of ECG are of fundamental importance in establishing a
meaningful recognition scheme.

The syntactic approach to pattern recognition has been receiving increased attention during the past
few years because it possesses the hand1ing capability lacked by the decision-theoretic approach.
Syntactic pattern recognition is based on concepts from formal language theory, the origins of which
may be traced to the middle 1950s with the development of mathematical models of grammar by Noam
Chomsky.

Basic to the syntactic pattern recognition approach is the decomposition of patterns into sub patterns or
primitives. By tracking a complex pattern it is possible to detect and encode the primitives in the form
of a string of qualifiers. Suppose that to interpret each primitive as being a symbol permissible in some
grammar, where a grammar is a set of rules of syntax for the generation of sentences from the given
symbols. Once the grammar has been established, the syntactic pattern recognition process is, in
principle, straight forward. Given a sentence representing an input pattern, the problem is to decide

23 Department of Artificial Intelligence and Data Science, DYPCOE, Akurdi, Pune

 Remove Watermark Wondershare
PDFelement
Elective I Laboratory T.E.(Sem-V) [2022-23]

whether the input pattern represents a valid sentence. If a pattern is not a sentence of the language under
consideration, it is assigned to a rejection class. 3.2 String grammars and language

As indicated in section 3.1, the methods of syntactic pattern recognition are based on mathematical
systems in which the patterns of a class are represented as elements of a language. One of the principal
requirements in designing a syntactic pattern recognition system is the development of a grammar
capable of generating a given class of patterns. Most of the material presented here deals with the
definition and interpretation of 31 grammars suitable for pattern representation and recognition.

The following concise definitions establish some of the principal notation and concepts used in formal
language theory.

Two categories of sets of interest in formal language theory are finite sets, which have a finite number
of elements, and countably infinite sets whose elements can be placed in a one-to-one correspondence
with the positive integer.

Given sets A and S, their Cartesian product >4 X S is the collection of all ordered pairs in ^ and & in B
. For sets A^ this may be extended to the y^i-fold Cartesian product .4^ X X ...X A^^, which is the set
of all ri-tuples , ... ) for a.^ in in A . The null set is the set containing no elements. V l ^ A relation from
set A to set S is a subset of ^4 X S, i.e., R ^ A X B . If for each element a of A there is exactly one
element h of B such that (Start by putting any one of the

In syntactic pattern recognition, a parser or error-correcting parser checks an unknown input whether it
is in accordance with the rules of a grammar that describes all members of a pattern class. This chapter
reviews basic concepts and algorithms applied in structural and syntactic pattern recognition.

What is Regular Expression?

In the real world, string parsing in most programming languages is handled by regular expression. The
regular expression in python programming language is a method used for matching text patterns.

The “re” module which comes with every python installation provides regular expression support.

Input:
1) Search String

2) Pattern

24 Department of Artificial Intelligence and Data Science, DYPCOE, Akurdi, Pune

 Remove Watermark Wondershare
PDFelement
Elective I Laboratory T.E.(Sem-V) [2022-23]

Implementation procedure:
In python, a regular expression search is typically written as:
match = re.search(pattern, string)
The re.search() method takes two arguments, a regular expression pattern and a string and searches for
that pattern within the string. If the pattern is found within the string, search() returns a match object or
None otherwise. So in a regular expression, given a string, determine whether that string matches a
given pattern, and, optionally, collect substrings that contain relevant information.

Matching patterns

Regular expressions are complicated mini-language. They rely on special characters to match unknown
strings, but let's start with literal characters, such as letters, numbers, and the space character, which
always match themselves. Let's see a basic.

Matching a string

The “re” module of python has numerous method, and to test whether a particular regular expression
matches a specific string, you can use re.search(). The re.MatchObject provides additional information
like which part of the string the match was found.

Syntax

matchObject = re.search(pattern, input_string, flags=0)

Output:

If the pattern is found within the string, search() returns a match object or None otherwise.

Conclusion: In a regular expression, given a string, determine whether that string matches a given
pattern.

25 Department of Artificial Intelligence and Data Science, DYPCOE, Akurdi, Pune

 Remove Watermark Wondershare
PDFelement
Elective I Laboratory T.E.(Sem-V) [2022-23]

ASSIGNMENT NO: 04

Problem Statement: Implement a suitable pattern recognition technique to perform Character

Recognition.

Title: Apply suitable pattern recognition technique to perform Character Recognition

Objective:

• Understand pattern recognition

• Implement character recognition

Outcome:

• What is OCR Technology?

• How does optical character recognition work?

• Software Required: Python 3.0 or extended version.

Theory:
Pattern recognition is the ability of machines to identify patterns in data, and then use those patterns to
make decisions or predictions. This guide provides an overview of the most important techniques used
to recognize patterns and real-world applications. This article will cover what pattern recognition is,
forms of pattern recognition, and analyze forms of pattern recognition in artificial intelligence.
Optical Character Recognition (OCR) defines the process of mechanically or electronically converting
scanned images of handwritten, typed, or printed text into machine-encoded text. Think of it as the
process of turning analog data, digital.
.What is OCR?
As OCR stands for optical character recognition, OCR technology deals with the problem of
recognizing all kinds of different characters. Both handwritten and printed characters can be recognized
and converted into a machine-readable, digital data format.

Think of any kind of serial number or code consisting of numbers and letters that you need digitized. By
using OCR you can transform these codes into a digital output. The technology makes use of many

26 Department of Artificial Intelligence and Data Science, DYPCOE, Akurdi, Pune

 Remove Watermark Wondershare
PDFelement
Elective I Laboratory T.E.(Sem-V) [2022-23]

different techniques. Put simply, the image taken is processed, the characters extracted, and are then
recognized.

What OCR does not do is consider the actual nature of the object that you want to scan. It simply “takes
a look” at the characters that you aim to transform into a digital format. For example, if you scan a word
it will learn and recognize the letters, but not the meaning of the word.

Image Source: Google Images

Optical Character Recognition Steps:

Three basic steps of optical character recognition: image pre-processing; character recognition; and the
post-processing of the output.

Step 1: Image Pre-Processing in OCR

OCR software often pre-processes images to improve the chances of successful recognition. The aim of
image pre-processing is an improvement of the actual image data. In this way, unwanted distortions are
suppressed and specific image features are enhanced. These two processes are important for the
following steps.

Step 2: Character Recognition in OCR

Character Recognition of License Plates

For actual character recognition, it is important to understand what “feature extraction” is. When the
input data is too large to be processed, only a reduced set of features is selected. The features selected

27 Department of Artificial Intelligence and Data Science, DYPCOE, Akurdi, Pune

 Remove Watermark Wondershare
PDFelement
Elective I Laboratory T.E.(Sem-V) [2022-23]

are expected to be the important ones while those that are suspected to be redundant are ignored. By
using the reduced set of data instead of the initial large one, the performance is increased.

For the process of OCR, this is important as the algorithm has to detect specific portions or shapes of a
digitized image or video stream.

Step 3: Post-Processing in OCR

Post-processing is another error correction technique that ensures the high accuracy of OCR. The
accuracy can be further improved if the output is restricted by a lexicon. That way, the algorithm can
fall back to a list of words that are allotted to occur in the scanned document for example.

OCR is not only used to identify proper words but can also read numbers and codes. This is useful for
identifying long strings of numbers and letters, such as serial numbers used in many industries.

To better deal with different types of input OCR, some providers started to develop specific OCR
systems. These systems are able to deal with the special images, and to improve the recognition
accuracy, even more, they combined various optimization techniques.

For example, they used business rules, standard expressions, or rich information contained in the color
image. This strategy of merging various optimization techniques is called “application-oriented OCR”
or “customized OCR”. It is used in applications such as business card OCR, invoice OCR, and ID card
OCR.

Use Cases for OCR Technology

The possibilities for using optical character recognition software is widespread as OCR can be
combined with a broad range of technologies. Here are a few examples of possible use cases including
OCR software:

1. Identification Processes in OCR

Machine Readable Zone (MRZ) in a Passport

Passports and IDs have a machine-readable zone (MRZ) that can be scanned. OCR can speed up the
process of identifying and registering people. This is useful for security forces at borders or other

28 Department of Artificial Intelligence and Data Science, DYPCOE, Akurdi, Pune

 Remove Watermark Wondershare
PDFelement
Elective I Laboratory T.E.(Sem-V) [2022-23]

checkpoints. It can also be used for commercial purposes to increase customer engagement, such as the
check-in process within hotels, or the registration process with banks and other businesses.

2. Marketing Campaigns with OCR

Leading brands are making use of OCR to run innovative and engaging campaigns to drive engagement
with their customers. Think of all the voucher codes that customers can redeem by typing them in. Or
numbers printed on the inside of a bottle cap that you need to collect.

All of these campaigns can make use of OCR by integrating the software, which easily integrates into
company to websites and apps. That way, they minimize the hurdle of online registration and remove
the need for customers to typing in a series of numbers and letters.

Take a look at how PepsiCo uses OCR in one of their marketing campaigns in Turkey to scan voucher
codes inside packets of their popular chips like Lays, Ruffles, and Doritos:

OCR in Payment Processes

IBAN Scanning with OCR

The International Bank Account Number (IBAN) serves to identify bank accounts across borders. The
IBAN may come in different lengths and can consist of numbers as toll as letters. To ease cross-border
transactions banking apps can easily integrate OCR software. That way their customers can scan their
IBAN instead of tediously typing it in.

Output: Identification of Input character

Conclusion: OCR is a suitable pattern recognition technique to perform Character Recognition.

29 Department of Artificial Intelligence and Data Science, DYPCOE, Akurdi, Pune

 Remove Watermark Wondershare
PDFelement
Elective I Laboratory T.E.(Sem-V) [2022-23]

Group C

30 Department of Artificial Intelligence and Data Science, DYPCOE, Akurdi, Pune

 Remove Watermark Wondershare
PDFelement
Elective I Laboratory T.E.(Sem-V) [2022-23]

ASSIGNMENT No: 05

Problem Statement: To implement a system for Handwritten Digit Recognition using Neural
Network.

Title: Develop a system for Handwritten Digit Recognition using Neural Network
Objective:
• Study neural network
• Understand and implement Handwritten digit recognition.

Outcomes:
• Ability to choose an appropriate system for Handwritten Digit Recognition using Neural
Network

Software Required:
• Python with Keras library and the Tkinter library for building GUI.

Theory:
The handwritten digit recognition is the ability of computers to recognize human handwritten digits. It is
a hard task for the machine because handwritten digits are not perfect and can be made with many
different flavors. Handwritten digit recognition is the solution to this problem which uses the image of a
digit and recognizes the digit present in the image.

Handwritten digit recognition using MNIST dataset is a major project made with the help of Neural
Networks. It basically detects the scanned images of handwritten digits.

To have taken this a step further where the handwritten digit recognition system not only detects
scanned images of handwritten digits but also allows writing digits on the screen with the help of an
integrated GUI for recognition

31 Department of Artificial Intelligence and Data Science, DYPCOE, Akurdi, Pune

 Remove Watermark Wondershare
PDFelement
Elective I Laboratory T.E.(Sem-V) [2022-23]

Handwritten digit recognition using MNIST dataset is a major project made with the help of Neural
Network. It basically detects the scanned images of handwritten digits.

taken this a step further where handwritten digit recognition system not only detects scanned images of
handwritten digits but also allows writing digits on the screen with the help of an integrated GUI for
recognition.

Approach:
will approach this project by using a three-layered Neural Network.

• The input layer: It distributes the features of the examples to the next layer for calculation of
activations of the next layer.
• The hidden layer: They are made of hidden units called activations providing nonlinear ties for
the network. A number of hidden layers can vary according to the requirements.
• The output layer: The nodes here are called output units. It provides us with the final prediction
of the Neural Network on the basis of which final predictions can be made.

A neural network is a model inspired by how the brain works. It consists of multiple layers having many
activations, this activation resembles neurons of the brain. A neural network tries to learn a set of
parameters in a set of data which could help to recognize the underlying relationships. Neural networks
can adapt to changing input; so the network generates the best possible result without needing to
redesign the output criteria.

Methodology:
To implement Neural Network with 1 hidden layer having 100 activation units (excluding bias units).
The data is loaded from a .mat file, and features(X) and labels(y) tore extracted. Then features are
divided by 255 to rescale them into a range of [0,1] to avoid overflow during computation. Data is split
up into 60,000 training and 10,000 testing examples. Feedforward is performed with the training set for
calculating the hypothesis and then backpropagation is done in order to reduce the error bettoen the
layers. The regularization parameter lambda is set to 0.1 to address the problem of overfitting. The
optimizer is run for 70 iterations to find the best fit model.

32 Department of Artificial Intelligence and Data Science, DYPCOE, Akurdi, Pune

 Remove Watermark Wondershare
PDFelement
Elective I Laboratory T.E.(Sem-V) [2022-23]

Figure: Neural Network Layers

This is one of the most popular datasets among machine learning and deep learning enthusiasts.
The MNIST dataset contains 60,000 training images of handwritten digits from zero to nine and 10,000
images for testing. So, the MNIST dataset has 10 different classes. The handwritten digits images are
represented as a 28×28 matrix where each cell contains a grayscale pixel value.

Input:
Data Set of hand written characters.

The steps to implement the handwritten digit recognition:

1. Import the libraries and load the dataset

First, going to import all the modules that need for training the model. The Keras library already
contains some datasets and MNIST is one of them. So to can easily import the dataset and start working
with it. The mnist.load_data() method returns us the training data, its labels and also the testing data and
its labels.

33 Department of Artificial Intelligence and Data Science, DYPCOE, Akurdi, Pune

 Remove Watermark Wondershare
PDFelement
Elective I Laboratory T.E.(Sem-V) [2022-23]

2. Preprocess the data

The image data cannot be fed directly into the model so to need to perform some operations and process
the data to make it ready for the neural network. The dimension of the training data is (60000,28,28).
The CNN model will require one more dimension so to reshape the matrix to shape (60000,28,28,1).

3. Create the model

Now will create the CNN model in Python data science project. A CNN model generally consists of
convolutional and pooling layers. It works better for data that are represented as grid structures, this is
the reason why CNN works toll for image classification problems. The dropout layer is used to
deactivate some of the neurons and while training, it reduces offer fitting of the model. will then
compile the model with the Ad delta optimizer.

4. Train the model

The model.fit() function of Keras will start the training of the model. It takes the training data,
validation data, epochs, and batch size. It takes some time to train the model. After training, to save the
rights and model definition in the ‘mnist.h5’ file.
5. Evaluate the model
Have 10,000 images in the dataset which will be used to evaluate how good the model works. The
testing data was not involved in the training of the data therefore, it is new data for the model. The
MNIST dataset is toll balanced so can get around 99% accuracy.

6. Create GUI to predict digits

Now for the GUI, to have created a new file in which to build an interactive window to draw digits on
canvas and with a button, to can recognize the digit. The Tkinter library comes in the Python standard
library. To have created a function predict_digit() that takes the image as input and then uses the trained
model to predict the digit.
Then to create the App class which is responsible for building the GUI for the app. To create a canvas
where to can draw by capturing the mouse event and with a button, to trigger the predict_digit()
function and display the results.

34 Department of Artificial Intelligence and Data Science, DYPCOE, Akurdi, Pune

 Remove Watermark Wondershare
PDFelement
Elective I Laboratory T.E.(Sem-V) [2022-23]

Output:
Hand written character recognition.

Conclusion:

Developed hand written character recognition using neural network successfully.

35 Department of Artificial Intelligence and Data Science, DYPCOE, Akurdi, Pune