INTRODUCTION AND HANDWRITTEN CHARACTER

RECOGNITION
BIOMETRICS:

Biometrics are body measurements and calculations related to human characteristics.

Biometrics authentication (or realistic authentication) is used in computer science as
a form of identification and access control. It is also used to identify individuals in
groups that are under surveillance. Biometrics has the potential to make
authentication dramatically faster, easier and more secure than traditional
passwords, but companies need to be careful about the biometric data they collect.
Examples of these biometric identifiers are fingerprints, facial patterns, voice or
typing cadence. Each of these identifiers is considered unique to the individual, and
they may be used in combination to ensure greater accuracy of identification.

HISTORY OF BIOMETRICS

• The biometric system is not new in this world. It was there, since the 14th
century. At that period, China had introduced fingerprinting and had started
taking fingerprints of its merchants and their children to distinguish them from
all others. Fingerprinting has evolved since then.
• In the 19th century, a Scientist named Alphonse Bertillion discovered a
system, i.e., measurements of the body of a person to identify them. It took
the subject's photograph and recorded height, the length of one foot, an arm,
and index finger. He had acknowledged that some traits of the human body
such as the length of the fingers remain unchanged concerning other physical
characteristics such as length of hair, weight, etc., gets altered. This method
disapproved quickly as the persons with same body measurements would be
 falsely taken as one. Hence, with the help of this research, a Scotland based
scientist Richard Edward Henry discovered a new method of fingerprinting.
• The notion of retinal identification was first introduced by Dr. Carleton Simon
and Dr. Isadore Goldstein in 1935. First face recognition paper was published
in 1971 (Goldstein et al.).The research and development effort was put in at
Eye Dentify Inc in 1976. The first commercial retina scanning system was
developed in 1981.
• John Daugman introduced the early successful Iris recognition in 1993 at
Cambridge University.
• FBI installed IAFIS in 2000 with a database of about 47 million prints;
average of 50,000 searches per day; 15% of searches are in lights out mode
and 2 hour response time for the criminal search.
• Biometrics Automated Toolset (BAT) was introduced in 2001, which
provided an accurate identification technique.

TYPES OF BIOMETRICS

Biometric technology is more refined, advanced, and super sensitive. It protects

companies and individuals. Most importantly, it is impossible to duplicate
biometrics works on biological characteristics of an individual. An individual has
two types of biological traits: behavioral and physical. So, there are two types of
biometrics: Behavioral Biometrics and Physiological Biometrics.

Behavioral Biometrics:

Behavioral Biometrics is the scientific study of how people's and animals' bodies
function. It is further divided into:
 • Signature Recognition
• Voice Recognition
• Keystroke

biometrics

Physical Behavioral
biometrics biometrics

Bertilion age Speaker

biometrics recognition

fingerprints
Signature
verification

Face
biometrics Keystroke
recognition
Hand geometry

iris

retina

DNA

Speaker recognition:

Voice is a behavioral as well as a physiological trait that depends on the physical

structure of the throat and mouth as well as chronic constituents. Being dependent
on many factors, voice becomes a crucial biometric identifier which can be used to
distinguish the speaker. Voice recognition recognizes the speaker as well as the
speech. Speech recognition is an emerged technology-powered way of verifying
what is being spoken, while speech recognition is about identifying who is talking,
i.e., the identification of the speaker. Many people use speaker recognition and
speech recognition interchangeably, but these two have different objectives, and
approaches to implementation, except that both are related to human voice. A visual
record of speech analyzed concerning frequency, duration, and amplitude is called
the voice print or spectrogram. Voice recognition is very popular and low-cost, but
it is less accurate and sometimes lengthy enrollment.

Signature recognition:

Signatures have been in use from decades for personal identification as well as for
high-value transactions. It is a behavioral characteristic and can produce a lot of
statistically reliable data and also can be captured by electronic means. Earlier,
manual methods of signature verification were used which includes confirmation of
its shape. The biometric signature recognition system can verify a lot more to make
sure that if it is an authorized user or an imposter. Banks and financial service
providers use signatures for authentication and authorization.

Keystroke Dynamics :

The rhythm with which one types at a keyboard is sufficiently distinctive to form the
basis of the biometric technology known as keystroke dynamics. Different people
have different striking rate and style. It is analyzed and kept as a record for various
security purposes.
Physiological/physical Biometrics

Physiological biometrics is based on a behavioral trait of an individual. It involves

all physical characteristics like, ears, eyes, iris, fingerprints, etc. It is further divided
into:

Bertillon age biometrics:

Alphonse Bertillon, a Perisian anthropologist and police desk clerk, developed a

method for identifying criminals that became known as Bertillonage. Bertillonage
was a form of anthropometry, a system by which measurements of the body are taken
for classification and comparison purposes. Bertillons system of anthropometry
required numerous and percise measurements of the bony parts of a humans anatomy
for identification. It also involved recording shapes of the body in relation to
movements and differential markings on the surface of the body such as scars, birth
marks, tattoos, etc. Bertillon estimated that the odds of duplicate records were
286,435,456 to 1 if 14 traits were used. This was the primary system of criminal
identification used during the 19th century. it turned out to be the case that the
features by which Bertillon based his identification system were not unique to any
one individual. This led to the possibility of one person being convicted of another
persons crimes. This possibility became abundantly clear in 1903 when a Will West
was confused with a William West. Though it would later turn out to be the case that
the two were identical twins, the issues posed by the Bertillonage system of
identification were clear.

Fingerprint recognition

The Fingerprint recognition is one of the oldest and, an efficient biometrics method
since fingerprints have long been recognized as an accurate and primary
identification method as the fingerprints of different individuals are not the same.
Like all the other biometrics technology, it identifies and verifies a person's
fingerprint with data saved in it beforehand. It has been a part of the forensic
investigation since the early days of fingerprint identification. Fingerprint
recognition has come a long way since then as the process of fingerprint
identification and matching advanced in forensic investigation agencies with the use
of technology. Nowadays, fingerprint recognition is extensively used from mobile
devices to door locks and even for high-security access control. Identification and
authentication have been made possible by tiny yet efficient fingerprint sensors for
mobile devices.

Face biometrics:

Our facial structure is the primary approach that people leverage to remember each
other. It is also a well exposed biometric technology that is used for human
identification and authentication. Now when we have high-quality cameras with the
ability to zoom, it is also likely to recognize a subject from the distance, making
facial recognition also suitable for security and surveillance applications.

• Facial recognition technology is straightforward to set up. A digital camera

and facial recognition software are all you need to set up this recognition
technology. For inclined security applications, more hardware like the camera
and infrared light emitter, multi-camera setup, etc are deployed.
• Facial recognition is one of the quickly developing versatile biometric
technologies. The reason behind this growth is the continuous rise of
smartphones and personal computing devices. The Modern-day gadgets carry
two cameras' one at the front and the other at the back. Hence, making it easy
to leverage the face recognition for user authentication.
 • Nowadays gadgets makers include additional hardware like Infrared emitter
for face recognition and a camera to capture IR illuminated 3D map of facial
structure. Apple was the first company to introduce this setup with iPhone X
with Face ID moniker.

Hand geometry biometrics

Hand geometry systems are commonly available in two main forms. Full hand
geometry systems take an image of the entire hand for comparison while Two Finger
readers only image two fingers of the hand. Hand recognition technology is currently
one of the most deployed biometrics disciplines world wide
A camera capture an image of the hand, with the help of a mirror to get also the edge
and some geometrical characteristics stored

Iris biometrics

In the human eye, the iris is the colored portion in the shape of a ring. If you look
closely, you will find it is made of many asymmetric thick thread-like structures.
These thread-like structures are the muscles that help adjust shape of the pupil and
only allow appropriate amount of light in the eye. By measuring the unique folds of
these muscles, biometric authentication tools can confirm identity with incredible
accuracy. Liveness detection – like requiring a user to blink for the scan – adds an
additional layer of accuracy and security.

Retina biometrics

Retinal scans capture capillaries deep within the eye by using unique near-infrared
cameras. The raw image is first preprocessed to enhance the image then processed
again as a biometric template to use during both enrollment and verification.
DNA matching

DNA matching has been introduced in our world a few decades before only. In this
biometric technology, some part of an individual's body like saliva, nail, hair or
blood is picked by forensic folks and then is taken through all forensic labs for
various investigations and medicinal purposes. DNA contains information about
race, paternity, and medical conditions for specific disease. In movies and especially
in CID we have seen these kinds of thrillers and CSI stories. DNA matching is a
proven method of personal identification where a physical sample of an individual
can be analyzed to confirm his/her identity. With the invention of this technology,
forensic investigations have got great relief as this biometrics helps in tracing the
criminals. A DNA sample is always collected with the subject's consent. The DNA
sample is usually gathered through a buckle smear. However, semen, blood, saliva,
etc may also be obtained and used for DNA profiling.

• The collected sample is collected through one of many methods available for
DNA profiling like STR Analysis, RFPL Analysis, AmpFLP, PCR Analysis,
etc.
• The outcomes are then compared with other fetched samples to find a match.
DNA profiling is an exact and accurate method of identification.

GENERAL ARCHITECTURE OF BIOMETRIC SYSTEM:

Biometric systems have made a significant impact on personal identification and

authentication practices worldwide. This technology has not only changed the way
people are identified, but also dramatically reduced time taken by identification and
verification processes. Verification of a claimed identity does not take more than a
few seconds with biometrics, while with traditional methods, it could even take
several days. Running an identification query on a biometric database with millions
of records has also become much faster with the development of advance computing
machines. Biometric identification has curbed identity theft and other identity
related crimes to a certain extent, as with biometric identity it becomes nearly
impossible to imitate one’s physiological and behavioral characteristics. On the
other hand, traditional card and document based identity requires only a few
copy/paste and edits to manipulate someone else’s identity.

Biometric system architecture is the representation of a system as a whole, including

a mapping of functionality onto hardware and software components, a mapping of
the software architecture onto the hardware architecture, and human interaction with
these components. Different biometric recognition systems may have different set
of sensors, sub-systems, algorithms, to achieve the objective of specific pattern
recognition and matching.

Enrollment and recognition modules:

There are two main processes in biometric systems, enrollment and recognition.
The generalized architecture of enrolment process.

In the enrollment module, biometric database is prepared including the input

traits of all the individuals to be verified. The database may be divided into
several parts depending on the type of images, complexity and format of the
input images, size of the input, etc.
During the enrollment process, the characteristics of an individual are measured.
First of all, an input is captured through image acquisition system or using a sensor
module. The input is subjected to training by using certain methods and the features,
on which the recognition is to take place, are extracted. As a part of representation,
the features are stored in the form of templates that correspond to the feature set for
the input images or any other traits. The templates arc then stored as template
database. Features are not originally stored for scanned or captured images but
their equivalent and compact forms are stored that are known as templates.
These templates are used in comparison with the templates of images being
tested.

Quality check is also implemented that ensures the quality of the

input biometric trait, or otherwise the recognition fails because of
poor quality of inputs.

The recognition module follows similar steps, first, a digital

representation is made and then it is processed into a template. The
processed template is then compared with all the templates stored in the
system to correctly identify the individual or the template can be
 compared to just one given individual's template to correctly verify the
individual.

Performance

There are three main factors involved in determining the performance

of any biometric system—accuracy, speed and size of templates. The
recognition accuracy depends on the rate of false matches and false non-
matches. A false match is the case where an individual is identified as
someone else, and a genuine individual cannot be correctly identified in
the case of false non-match.

Accuracy can be increased by minimizing these two rates in a biometric

system. The speed depends on processing time of individual steps. The time
required in capturing an image, feature extraction, representation of these
features into templates and other processes affect the speed of the system.
Retrieval time at the time of matching or testing is increased with the
increase in database size of users. Recent biometric systems can perform
these tasks within a minute or less.

This size is the amount of bytes required to store a template. This amount
further affects the storage capacity and the amount of time required to
transfer the template across a network. The size can be reduced to
enhance the performance of a system but keeping in mind that the
inherent content of input should, not be lost.

Biometric functionalities:

Best biometric measure or trait can be selected on the basis of the following
characteristics:
1. Unique: It must be different across users.
2. Repeatable: it must be similar across for each user.
3. Accessible: it should be easily linked to sensors.
4. Universal: It should work for all the people.
5. Acceptable: It must not he objectionable to display by user.
Verification and identification:
Recognition is based on verification and identification. Two types of matching are there,
one-to-one and one-to-many

1. Verification: One-to-one (1:1) matching of a biometric trait for a person.

The given trait is compared with all the traits present in the database. It can
verify the identity of a person by comparing the already stored characteristics
to the characteristics provided at the time of recognition. This is called
verification which is also referred to as a one-to-one match.

2. Identification: One-to-many (1:N) matching of a biometric trait against a

biometric database in an attempt to identify an unknown individual. A
biometric data captured is compared against the data available in the
database and it is found that the biometric data being tested is present in the
database. It means that there is matching of one-to-many.

BASIC WORKING OF BIOMETRIC MATCHING (TEMPLATES)

It is very important to note that even though various technologies differ in many
ways but the basic working concept particularly of matching of templates is
almost similar.

The processes involved in biometrics are briefly discussed as under:

Enrollment: A user, who wants to get him/her self registered, is enrolled by
capturing his/her biometric traits. The biometric modality captured may be face,
fingerprint, iris or any other trait. These traits are stored in the database and
subjected to feature extraction after some sort of preprocessing

Feature extraction: The traits or inputs captured are subjected to feature

extraction. Features are very important for any image because comparison
of these features forms the basis of recognition.
Creation of templates: The features are not stored in their original form.
Rather they are stored in the form of compact and equivalent representations
known as templates.
Biometric matching: During biometric recognition, the input being
tested is compared against all the templates already stored in the database.
Based on the degree of correlation between the templates of image being
matched and templates of all the enrolled images in the database, decision
is made. The decision results in either match or non-match status of the trait
being tested. Templates are representation of features but as per their usage
at different stages, we can classify them conveniently into two types:
 (a) Enrollment templates: These templates are generated at the time of
enrollment. The features extracted are converted into equivalent
representation. These templates are stored in the enrollment database
for future use.
(b) Matching templates: These templates are generated again at the time
of testing the user to be identified. They are then compared with the
templates available in the enrollment templates.

Template Transformations

The templates also need to be transformed for their protection. Templates

generated for a user must not be misused by someone else. Transformation should
not alter the inherent content of biometric traits as this would only change an
equivalent representation identical to the templates to be transformed. A template
generated in a format for a particular application X could not he misappropriated
and used to authenticate a user for application Y because the enrollment database
for application Y would have a different format than those enrolled for application
X.

A suitable transformation function F used for biometric templates is required to

satisfy the following conditions:

1. The function F must not change the length of the template otherwise the content
of the input trait may change.

2. F must not change the value of the any specific bits used for encoding.

3. F must not change the number of data bit pairs for matching or mismatching.

BIOMETRIC SYSTEM ERROR AND PERFORMANCE MEASURES

All the biometric systems, irrespective of the biometric data captured, suffer from some
errors due to many factors such as problem with the equipments, noisy or corrupt data,
large database size, etc. The errors affect the system performance. Lesser the values
of errors, better is the system. Therefore, performance measures are sometimes
evaluated in terms of errors.

Most of the problems are found in uni-modal biometrics in which only one biometric
trait is involved. The implementation of multimodal systems helps avoid or diminish
these problems to certain extent. Reliability is a very important issue in multimodal
systems.

The accuracy of biometric techniques as: false match (imposter or a false person
is identified), match (correct user is rejected), failure to enroll (user cannot
register in the system). False match rate (FMR) or false acceptance rate (FAR)
are used as performance measures in biometrics. False non-match rate (FNMR) or
false rejection rate (FRR) represents the probability that a user's verification
template will be judged not to match that same user's enrolment template. Failure
to enroll (FTE) rate represents the probability that a particular user will be
unable to enroll in a biometric system due to insufficient biometric samples.
The performance measures have been discussed as follows:

1. False match rate (FMR): This is a probability of matching a

template with the template of person in enrollment templates
database. This may be due the imposter trying to pose himself as a
legitimate user. This is the most important evaluation parameter
because the imposters or unauthorized users can be rejected by using
this statistical parameter.
 2. False non-match rate (FNMR): This is the likelihood of non-matching
of user when matched with his templates available in the enrollment
database. High value of this parameter will affect cost unnecessarily
as this is not as important as FMR.
3. Failure to enroll (FTE) rate: This is the possibility of not letting a
user get enrolled which may be because of many factors such as physical
reason, case of imposter, that is, his metrics are already available in the
database. This might be also be due to change in behavioral traits or any
other factors that cause problems in enrollment process.
These three metrics are always interrelated. If any one of the metrics is not
proper all the metrics need to be recorded and that increases time. If FMR
decreases, that is, it makes system less suitable to imposters, then legitimate
users will be rejected (FNMR) and vice-versa. The values of FMR or FAR
and FNMR or FRR can form curves whose intersect is called equal error
rate (EER) or the crossover accuracy of the system.

DESIGN OF BIOMETRIC SYSTEMS

Biometric systems are designed based on many factors such as types of features,
nature of application, choice of traits, etc.

Based on the Type of Application

There are three main areas of implementation for biometric technologies

commercial applications, Government applications and forensic applications.

Commercial applications are commonly used biometrics because of their

availability, accessibility, acceptability, cost, simplicity and ease of use. This type
of applications makes use of uni-modal biometrics.
The basic requirement of biometric applications in government applications is
increased security. This application makes use of multimodal biometrics. Best
example for this is AADHAR card.

Based on the Choice of Biometric Modality

Each type of biometrics has its own advantages and challenges. All the requirements
and performance measures cannot be achieved satisfactorily in any of the
biometrics. Therefore, multimodal biometrics is recommended for important
applications.

Criteria must be designed to select which biometrics is suitable for a particular

application. The requirement type, the choice of traits used, potential cost are
the factors that may be taken care of while selecting a biometric solution for a
particular application. a biometric system should be selected in such a manner
that the changes in biometric traits are minimum or none and then only the
accuracy with which biometric solution was envisaged will be more and the
solutions will be efficient. Accurate identification or verification is very
difficult to achieve if there are possibilities of variations in biometric modal -
ities. Metrics selected should be enough for verification to achieve a unanimous
decision.

For each type of decision, there are two possible outcomes, true or false, that is
match or non-match. This results in possible outcomes such as, a true individual
is accepted, a true individual is rejected, an imposter is accepted or rejected.
FAR and FRR are always recommended for evaluating.

APPLICATIONS OF BIOMETRICS

Commercial Sector
Commercial applications of biometrics are largely used by banking and financial
service sectors. Some of the commercial applications are reported as under:

1. Account access
The use of biometrics for accessing an account in a particular bank helps the
bank authority to check the authority of customers
2. ATM
Use of biometrics enhances the security of transaction made in ATM
booths.
3. Expended service kioks
Special purpose kioks that would offer a wider variety Of financial
transactions may be set up where biometrics can find sample application
for identification.
4. Online banking
Apart from Passwords and virtual keyboards available to the customers of
internet banking, there are many more security enhancement techniques
yet to be achieved.
5. Telephonic transactions
Voice based biometrics may be deployed for verification of customers
and secure transactions could be made.
6. Personal computer/Network security
Biometric based tools can be embedded and implemented to protect the
unauthorized access of computers and networks to protect the databases.

7. E-commerce
Similar to online transactions, biometrics can be used to enhance security
of transaction in e-commerce applications.

Government Sector
 There are numerous applications of biometrics in government sector. In many
government driven programs and activities, biometrics is widely used. Some of
the typical applications of biometrics in this sector are:

1. National identification card

This kind of cards can be used for various purposes by the beneficiaries of
the country such as collection of their wages and benefits, getting voter
identity cards, etc. Most of these applications are based on fingerprint
technologies and automated fingerprint identification system (AFIS).
2. Voter ID card
Voter ID cards can use the simplest tools of biometrics for common people
but that project is yet to be implemented in India. Existing voter ID cards are
not biometric based but very soon this will be achieved.
3. Driving license
Driving license is also considered as one of the authentication tools in many
countries for its citizens. Biometric features may be included in driving
license issued to a person that would help in monitoring the traffic, speed of
vehicles, etc.

Employee Authentication

Various organizations employ biometrics solutions for security purposes,

authentication of their employees, for secure network and data management, etc.
Tata Consultancy Services (TCS), Infosys, offices of Defence Research and
Development Organization (DRDO) and other such reputed organizations use
biometric systems to a great extent.

Military Programs
In all military headquarters and important offices, authentication measures are
stringent. Until the person being tested is identified and verified, he is not
permitted to enter the premises.

Health Care

Biometrics may be used in health care sectors in enrolling or registering the

patients at the time of admission, and rest of the transactions can be made based on
his biometric traits. This may prevent fraud cases of hospitalization and distribution
or selling of medicines. Some typical applications are:

1. Access to personal information: Using biometrics, the medical patient

information may be stored using a biometric based smart card system or
secured networks which enable the access of the patients to their personal
information.
2. Patient identification: In case of emergency or death, biometrics can help in
identification of an individual and accordingly a communication can be made
to his address.
Travel and Immigration

Biometrics is used for identification or verification of an individual interacting

during the course of travel,acting as a traveler or immigration employee with a
travel company or as an immigration entity. Typical applications are:

1. Air travel: Many airports are already using biometrics systems.

2. Border crossing: Checking the law, e-number of immigrants and reducing
biometrics is used.
3. Employee access: Several airports se biometrics to control the physical
access of employees in secure areas
BENEFITS OF BIOMETRICS VERSUS TRADITIONAL
AUTHENTICATION METHODS
The most pertinent and most recent example of multimodal biometrics is
distribution of unique ID cards to citizens of India. The UIDAI was established
under the leadership of Mr.Nandan Nilekani with a mandate by central government
that a unique ID card will be issued to all the citizens. Quality of data captured,
awareness among masses, well qualified vendors and persons involved in the
process of preparing the cards are the challenges of this project. Before the advent
of modern biometric tools, conventional authentication systems were used. The
development of biometrics took place in the latter half of twentieth century.
However, it only became a commonly used technology around the turn of the
millennium. Biological characteristics are most important entities which are used
in biometric recognition technologies.

1. Use of biological features: The main reason why biometric authentication

is so prevalent and popular is because it measures a biological feature, such
as a fingerprint or iris. Biological features are almost unique characteristics
and therefore these are effective. This is better than conventional card
system or PIN number system that could be stolen and misused.

2. Fast processing: Being electronic, the existing biometric systems are very
fast in processing of authentication applications.

Biometric authentication systems have many difficulties in acquiring the traits,

extracting the features and in representing into templates. Protection of
biometric data is essential otherwise fraudulent cases or cases would increase.
Biometrics helps the users by relieving them from memorizing many passwords for
various applications.

CHARACTER RECOGNITION

Character recognition is a particular area of pattern recognition which involves

recognizing letters, nurnerals and special characters. Pattern recognition is the
process of recognizing various types of patterns or objects using suitable
algorithms and with-the help of computers. Pattern recognition includes
identification of different characters, numerals or certain objects. The objects
may be images or some other signals depending on the application. Pattern
recognition is a very important part of machine vision and intelligence and is
based on decision making with the help of soft computing and other tools. In
machine vision approach, images are captured through cameras and trained by
using computers or machines. Some features are then extracted and comparison
and analysis are made easily by the machine. The image or a pattern is
described by machine by applying suitable algorithm or method. Handwritten
character recognition is mainly divided into two types—offline arid online.
When scanning and interpretation of handwriting takes place simultaneously
with writing, it is called online character recognition and if the analysis of data
is made later, it is called as offline character recognition.

Character recognition is used in many academic institutions and industrial

companies involved in the field of handwritten character recognition. The
process of recognizing the unique style of writing by computer software is
known as handwriting recognition. Variation in handwriting is one prominent
problem and achieving a high degree of accuracy is a tedious risk.
 SYSTEM OVERVIEW
This system accepts the characters as images scanned by using scanners or read
by image readers. Different characters are segmented and separated by using
morphological operations. The features are extracted and classified and the
character is recognized. The following procedure is adopted in character
recognition:

1. Handwritten characters are scanned and represented as digital images.

2. The red, green and blue (RGB) or color images converted into binary images.
3. Edges are detected and morphological operations such as dilation, opening
and hole filling are performed.
4. The results are trained with the help of multilayer feed forward back
propagation neural network.
5. Noisy images or handwritten characters are subjected to de-noising and
preprocessing so that they can be trained properly using neural network.
A block diagram of the proposed online recognition system of isolated Roman
characters may be explained in terms of following steps:

1. Acquire the input data (handwritten characters).

2. Preprocess the input image such as converting RGB to gray or binary image.
3. Remove noise in digital images.
4. Detect the edges.
5. Apply morphological operations to thicken the edges and fill the holes.
6. Detect the characters.
7. Train these characters using neural network
8. Extract the features and develop the database of features
9. Characters are recognized.
 Scan or
capture input
image

Pre-process the input

image

Detect the edges

Apply morphological
operation

Extract the characters

Database of training
Train the characters
weights and features
using neural networks

Character is recognized
or classified

Block diagram of the system

Data acquisition:

The samples are first scanned and then stored. Upper case letters are taken without
any constraints to style the writing. The characters required are then isolated.
 Input characters stored (A to D) as images.

Preprocessing:

The optical character recognition system first converts scanned pages into separate
characters. This step is known as preprocessing and converts RGB or color image
into gray scale image that again gets converted into binary image.

(a) (b)

Processing of (a) binary and (b) inverted binary images.

Morphological operations:

The term morphology usually means a study that deals with the form and structure
of animals and plants. The morphological operations here, are used to obtain the
structure of various images. The operations are performed in extracting various
image components such shape, boundaries or skeleton as structure. In character
recognition, two main operations — dilation and erosion have been used. These are
further used in the opening operation for salt noise reduction from the digital images.
Following is the sequence of steps performed by morphological operations:

1. Opening operation is performed to remove salt noise.

2. Dilation is applied to thicken the broken characters.
3. Hole filling is done.
4. Characters are labeled.
5. Characters are extracted.
6. Features of the characters are extracted.
7. Character recognition is performed on the basis of matching with the
features of characters.
The opening works like erosion and r some of the foreground (bright) pixels
from the regions having edge information. The foreground legions that arc similar
in shape with the structuring element are preserved by the operator and all other
regions of foreground pixels are eliminated. In opening operation, two inputs are
required, that is an image to be opened and a structuring element.

Image before and after opening operation.

• Detection of edges:
Edge detection contains high contrast value in the image. image segmentation and
clustering are mainly based on edge detection. When the edges are recovered the data
storage gets reduced to some extent.

Edge detected characters

• Dilation:

Its another morphological operation which is applied on binary images. The dilation
operator takes inputs as image and structuring element. The image which is to be
dilated is subjected to the dilation operator and structuring element is used as
kernel element which is a set of coordinate points. The boundaries of foreground
pixels are gradually enlarged in binary image. Generally, the foreground pixels
are white. So, as the area of foreground pixels incr,:ases in size, the holes within
those regions become small.

• Hole filling:
The marker image is chosen as zero everywhere except on the image boundary. For
boundaries of an image, marker is set to one. On selecting the marker images and
applying hole filling, the resulting image can be seen as

Image after hole filing

Character detection

In character detection process, the input image is converted into binary image
which contains only zero and one pixel values. All the dark or black pixels carry
zero whereas the bright or white pixels carry one. Characters are detected by using
two main following steps:

• Calculate the number of rows:

A suitable algorithm may be used to search for either presence or absence
of white pixels or black pixels in top to bottom approach. The gap between
the two rows is indicated by the absence of white pixels or presence of black
pixels. The search is then applied from left to right. If there is absence of
white pixel continuously for some pixels and again its presence is detected
then that row number is recorded.

• Detect the boundaries of characters

Their boundaries are required so that whole character area can be detected for
efficient character recognition. The boundaries are calculated by searching
from left to right and top to bottom. When searched from left to right, the
first white pixel is the leftmost boundary and the last white pixel is the
rightmost boundary in a given row of the binary image. The search is
carried out from top to bottom.

Character detection with its boundaries.

FEATURE EXTRACTION FOR CHARACTER RECOGNITION
Feature extraction is done using three steps:
1. Extreme coordinates are measured
2. Characters are taken into grids.
3. Characters are finally digitalized.

The handwritten characters are detected by searching extreme pixels from

left to right and top to bottom. The image containing characters is divided
into a rectangular grid of certain number of rows and columns. The grid
size is automatically adjusted according to the dimensions of the characters
because the characters are detected searching the presence of particular
pixels. The character forming pixels are marked as "on" whereas the outer
pixels are marked as "off". Therefore, the characters are digitized. The
 digitized characters are subjected to neural network for training. Each
character is represented as 7 x 5 grid.

Character digitization of A without noise

However, the above image might be achieved with some amount of noise
because no imaging system is perfect and free from noise. The possible
noisy image of the above image is

Digitized character A with some noise

NEURAL NETWORK FOR HANDWRITTEN CHARACTER

RECOGNITION

Neural networks are used for the purpose of classification. . Since the neural
network is analogy of human brain, it is known as artificial neural network.

The neural network consists of three main layers- input, output and hidden layers.
Input layer has 35 nodes assuming 7 x 5 grid size of character so the number of
nodes in input layer is 35. The hidden layer comprises of 20 nodes and an output
layer consists of 26 nodes for 26 different alphabetical characters.

Input Layer: As the name suggests, it accepts inputs in several different formats
provided by the programmer.

Hidden Layer: The hidden layer presents in-between input and output layers. It
performs all the calculations to find hidden features and patterns.

Output Layer: The input goes through a series of transformations using the hidden
layer, which finally results in output that is conveyed using this layer.

Single layer neural network

A sample of artificial neural network

Training of the neural network is done by adjusting the weights of various nodes.
This is done with the back propagation algorithm. In the network, a position with
minimum error function has to be reached. Back propagation is called so because
the weights are adjusted going back and covering the nodes in previous layers until
a desired goal is achieved in output layer. The error is successively computed which
is actually the difference between desired and actual output. Based on the error, the
weights are adjusted again and information of error is passed to a previous layer until
the first layer is reached. The error signal is propagated back through the network.

The term Neural Network is derived from Biological neural networks that develop
the structure of a human brain. Similar to the human brain that has neurons
interconnected to one another, artificial neural networks also have neurons that are
interconnected to one another in various layers of the networks. These neurons are
known as nodes

MULTILAYER NEURAL NETWORK FOR HANDWRITTEN

CHARACTER RECOGNITION

A multi-layer neural network contains more than one layer of artificial neurons or
nodes. They differ widely in design.
 Neural network accepts 35 input Boolean values through 35 nodes as 35
element input vector. Output vector has 26 elements for presenting 26
letters. If a character is recognized, it results as output value of one. Due to
some amount of noise in image, classification and recognition may have some
error still in the output with certain value of mean and standard deviation.
An error with zero mean value and standard deviation less than 0.2 is
recommended.

The neural network receives 35 inputs and 26 neurons are there in its
output layer to identify the letters. Three layered neural network uses log
sigmoid function to produce output. The log sigmoid transfer function is
preferred since its output range from zero to one which is compatible for
learning Boolean values.

Training the neural network:

Once a network has been structured for a particular application, that network is ready
to be trained. To start this process the initial weights are chosen randomly. Then, the
training, or learning, begins.

There are two approaches to training - supervised and unsupervised. Supervised

training involves a mechanism of providing the network with the desired output
either by manually "grading" the network's performance or by providing the desired
outputs with the inputs. Unsupervised training is where the network has to make
sense of the inputs without outside help.

The vast bulk of networks utilize supervised training. Unsupervised training is used
to perform some initial characterization on inputs.
Supervised training: In supervised training, both the inputs and the outputs are
provided. The network then processes the inputs and compares its resulting
outputs against the desired outputs. Errors are then propagated back through the
system, causing the system to adjust the weights which control the network. This
process occurs over and over as the weights are continually tweaked. The set of
data which enables the training is called the "training set." During the training of a
network the same set of data is processed many times as the connection weights
are ever refined.

Unsupervised training: The other type of training is called unsupervised training. In

unsupervised training, the network is provided with inputs but not with desired
outputs. The system itself must then decide what features it will use to group the
input data. This is often referred to as self-organization or adaption.

Performance of the system:

The performance of the neural network used for character recognition system
is evaluated by testing the network with many input vectors with varying
amount of noise. Noise with a mean of,-(and a standard deviation from 0 to
0.5 is added to input vectors.

The solid line in the plot shown in Figure indicates the reliability for the
network trained with noise. The performance of the same network for the
training without noise can be seen as dashed line. Apparently, training the
network for noisy input vector has larger error. If higher accuracy is required
then the network can be trained for a longer time and with increased number
of neurons in hidden layer.
 System performance plot

DEVANAGIRI NUMERAL RECOGNITION

The Devanagari script (commonly used in writing Hindi, Sanskrit, Marathi

and Nepali) is abugida or alphasyllabary as each constant has an inhere nt
vowel that can be changed with different vowel signs. The vowel can be
written both as independent character and by using a variety of diacritical
marks. These diacritical marks are written above, below, before or after the
consonants they belong to.

The handwritten numerals data collected of 10 different persons from 3

different states of India. The data is written on paper and scanned and finally
stored as image.
Processing converts the scanned RGB image into gray scale and further to
binary image. The binary image is then inverted because morphological
operations such as edge detection are easily applied to black background of
the image. Black pixels have zero gray scale values and white pixels carry
one as gray scale value

Image after edge detection.

Result of image dilation

The resulting image after hole filling

Numeral detection and feature extraction

Now, numerals can be detected similar to character detection the algorithm is

used to -search from left to right for white pixels starting from left top corner
of the area of image. Detectio n of the pixels indicates the presence of a
character.

After calculating the number of rows by applying the process of character

detection, the height and width of each numeral is measured. Feature
extraction process involves similar steps as that of character. recognitio n:
finding the coordinates or pixels of characters, taking the character into grid
and digitization of the character. The hand — written numerals are captured and
divided into a rectangular grid of certain number of rows and columns. The
size of character detected, decides the size of grid. Then the presence of
character pixel in every box of the grid is searched. The pixels of character are
marked as "on" and those of character grid are marked as "off". A binary string
of each character is formed locating the "on" and "off" boxes and presented to
the neural network input for training and recognition.

Devanagiri numeral 3 and numeral detection and feature extraction

ISOLATED HANDWRITTEN DEVANAGARI CHARACTER
RECOGNITION USING FOURIER DESCRIPTOR AND HIDDEN
MARKOV MODEL (HMM)

Image Normalization and Feature Extraction:

Once the characters are written using Microsoft(MS) paint, they are stored as
binary image that consists of a large white background with a black foreground
in front. Features of the image invariant to translation, scaling and rotation are
extracted. The features are extracted from all the images in the database and are applied
to HMM training.

Original image for character ka and image after performing scaling and
translation

HMM approach:

The HMM is a doubly stochastic process. Since the process is not observable,
the model is called hidden but second process can be observed through another
stochastic process which produces the sequence of observations. The process
involved in HMM approach consists of a set of states connected to each other
by transitions with probabilities. The observed process includes set of
outputs or observations, each of which is emitted by a state. This is done
in accordance with some output probability density function (PDF).
Different types of HMM models can be developed based on different PDFs.
 The HMM with multivariant Gaussian state conditional distribution
function consists of row vector which further contains probability distribution
for the first unobserved state. The mean vector and covariance matrix are stored
one above the other in two different ways, depending upon whether full or diagonal
covariance matrices are used. For isolated handwritten character recognition, it is
useful to consider the left-to-right models. In the left-to-right model, transition
from state i to state j is only allowed if j>= I, resulting in a smaller number
of translation probabilities to be trained.
The matching process computes a matching score between the
sequence of observation vector and each character model using the Viterbi
algorithm. After post processing, a lexicon sorted by matching score is
the final output of the character recognition system.