Volume 4, Issue 6, June – 2019 International Journal of Innovative Science and Research Technology

ISSN No:-2456-2165

Handwritten Digit Recognition using CNN

1
Vijayalaxmi R Rudraswamimath , Bhavanishankar K2
1Computer Science and Engineering, RNS Institute of Technology, Channasandra, Bangalore, India
2Assistant Professor, Computer Science and Engineering, RNS Institute of Technology, Bangalore

Abstract:- Digit Recognition is a noteworthy and document. Deep Learning models can accomplish state-of-art
important issue. As the manually written digits are not of a accuracy, beyond the human level performance. The digit
similar size, thickness, position and direction, in this recognition model uses large datasets in order to recognize
manner, various difficulties must be considered to digits from distinctive sources.
determine the issue of handwritten digit recognition. The
uniqueness and assortment in the composition styles of Handwriting recognition of characters has been around
various individuals additionally influence the example and since the 1980s. The task of handwritten digit recognition,
presence of the digits. It is the strategy for perceiving and using a classifier, has extraordinary significance and use such
arranging transcribed digits. It has a wide range of as – online digit recognition on PC tablets, recognize zip
applications, for example, programmed bank checks, codes on mail, processing bank check amounts, numeric
postal locations and tax documents and so on. sections in structures filled up by hand (for example ‐ tax
forms) and so on. There are diverse challenges faced while
The aim of this project is to implement a classificatio attempting to solve this problem. The handwritten digits are
n algorithm to recognize the handwritten digits. The after not always of the same size, thickness, or orientation and
effects of probably the most broadly utilized Machine position relative to the margins. The main objective was to
Learning Algorithms like SVM, KNN and RFC and with actualize a pattern characterization method to perceive the
Deep Learning calculation like multilayer CNN utilizing handwritten digits provided in the MINIST data set of images
Keras with Theano and Tensorflow. Utilizing these, the of handwritten digits (0‐9).
accuracy of 98.70% utilizing CNN (Keras + Theano) when
contrasted with 97.91% utilizing SVM, 96.67% utilizing II. LITERATURE SURVEY
KNN, 96.89% utilizing RFC was obtained.
Anuj Dutt in his paper demonstrated that utilizing Deep
Keywords:- KNN, SVM, RFC, CNN. Learning systems, he had the capacity to get an extremely high
measure of accuracy. By utilizing the convolutional Neural
I. INTRODUCTION Network with Keras and Theano as backend, he was getting a
accuracy of 98.72%. In addition, execution of CNN utilizing
Recognition is identifying or distinguishing a thing or an Tensorflow gives a stunningly better consequence of 99.70%.
individual from the past experiences or learning. Similarly, Despite the fact that the complication of the procedure and
Digit Recognition is nothing but recognizing or identifying the codes appears to be more when contrasted with typical
digits in any document. Digit recognition framework is simply Machine Learning algorithms yet the accuracy he got is
the working of a machine to prepare itself or interpret the increasingly obvious.
digits. Handwritten Digit Recognition is the capacity of a
computer to interpret the manually written digits from various In a paper published by Saeed AL-Mansoori, Multilayer
sources like messages, bank cheques, papers, pictures, and so Perceptron (MLP) Neural Network was implemented to
forth and in various situations for web based handwriting recognize and predict handwritten digits from 0 to 9.The
recognition on PC tablets, identifying number plates of proposed neural system was trained and tested on a dataset
vehicles, handling bank cheques, digits entered in any forms achieved from MNIST.
etc.
A. Existing System
Machine Learning provides various methods through These days, an ever-increasing number of individuals
which human efforts can be reduced in recognizing the use pictures to transmit data. It is additionally main stream to
manually written digits. Deep Learning is a machine learning separate critical data from pictures. Image Recognition is an
method that trains computers to do what easily falls into place imperative research area for its generally used applications. In
for people: learning through examples. With the utilization of general, the field of pattern recognition, one of the difficult
deep learning methods, human attempts can be diminished in undertakings is the precise computerized recognition of human
perceiving, learning, recognizing and in a lot more regions. handwriting. Without a doubt, this is a very difficult issue
Using deep learning, the computer learns to carry out because there is an extensive diversity in handwriting from an
classification works from pictures or contents from any individual to another individual. In spite of the fact that, this

IJISRT19JU358 www.ijisrt.com 182

Volume 4, Issue 6, June – 2019 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
difference does not make any issues to people, yet, anyway it A. Pre-Processing:
is increasingly hard to instruct computers to interpret general The role of the pre-processing step is it performs various
handwriting. For the image recognition issue, for example, tasks on the input image. It basically upgrades the image by
handwritten classification, it is essential to make out how making it reasonable for segmentation. The fundamental
information is depicted onto images. motivation behind pre-processing is to take off a fascinating
example from the background. For the most part, noise
Handwritten Recognition from the MNIST dataset is filtering, smoothing and standardization are to be done in this
well known among scientists as by utilizing different stage. The pre-processing additionally characterizes a smaller
classifiers for various parameters, the error rate has been portrayal of the example. Binarization changes over a gray
decreased, for example, from linear classifier (1-layer NN) scale image into a binary image.
with 12% to 0.23% by a board of 35 convolution neural
systems. The scope of this is to implement a Handwritten The initial approach to the training set images that are to
Digit Recognition framework and think about the diverse be processed in order to reduce the data, by thresholding them
classifiers and different techniques by concentrating on how to into a binary image. The Figure 2 shows a sample of images
accomplish close to human performance. For an undertaking taken from the MNIST database.
of composing diverse digits (0-9) for various people the
general issue confronted would be of digit order issue and the
closeness between the digits like 1 and 7, 5 and 6, 3 and 8, 9
and 8 and so forth.

Additionally, individuals compose a similar digit from

various perspectives, the uniqueness and assortment in the
handwriting of various people likewise impact the
development and presence of the digits.

III. ARCHITECTURE

The reason behind this document is to look into the Fig 2:- Sample images taken from MNIST database
design possibilities of the proposed system, such as
architecture design, block diagram, sequence diagram, data B. Segmentation:
flow diagram and user interface design of the system in order Once the pre-processing of the input images is
to define the steps such as pre-processing, feature extraction, completed, sub-images of individual digits are formed from
segmentation, classification and recognition of digits. the sequence of images. Pre-processed digit images are
segmented into a sub-image of individual digits, which are
assigned a number to each digit. Each individual digit is
resized into pixels. In this step an edge detection technique is
being used for segmentation of dataset images.

C. Feature Extraction:
After the completion of pre-processing stage and
segmentation stage, the pre-processed images are represented
in the form of a matrix which contains pixels of the images
that are of very large size. In this way it will be valuable to
represent the digits in the images which contain the necessary
information. This activity is called feature extraction. In the
feature extraction stage redundancy from the data is removed.

D. Classification and Recognition:

In the classification and recognition step the extracted
Fig 1:- Architecture of the Proposed System
feature vectors are taken as an individual input to each of the
following classifiers. In order to showcase the working system
The above Figure 1 illustrates the architecture diagram
model extracted features are combined and defined using
of the proposed system. The proposed model contains the four
following three classifiers:
stages in order to classify and detect the digits:
 K-Nearest Neighbor
A. Pre-processing
B. Segmentation  Random Forest Classifier
C. Feature Extraction  Support Vector Machine
D. Classification and Recognition

IJISRT19JU358 www.ijisrt.com 183

Volume 4, Issue 6, June – 2019 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
IV. METHODOLOGY

Each research work needs some estimation, to measure

the accuracy and performance of handwritten digits, MNIST
dataset is being used for such reasons. MNIST is the most
broadly utilized standard for handwritten digit recognition.
MNIST is a huge and a standard database of handwritten
digits. MNIST dataset has been commonly used as a standard
for testing classification algorithms in handwritten digit
recognition frameworks.

The initial step to be carried out is to place the dataset,

which can be effectively done through the Keras programming
interface. The images in the MNIST dataset are available in
type of a cluster comprising of 28x28 values constituting to an
image along with their labels. This is equivalent if there could
be an occurrence of the testing images. The pixels are given as
a variety of 784-d pixels and the range extends from 0 to 255
for example 0 implies Black and 255 implies White.

A. Pseudocode

Fig 3:- Block Diagram of proposed model

1) K-Nearest Neighbor: KNN is an instance based learning

algorithm. There are two main benefits of using KNN
algorithm, that is, it is robust to noisy training data and it is
very efficient if the data is very large in size. To perform
admirably, this algorithm requires a set of training datasets
which includes perfectly labeled data points. KNN is also a
non-parametric classifier. The algorithm considers new data
point as its input and performs classification by calculating
distance between new and labeled data points using the
Euclidean or Hamming distance formulas. The Euclidean
distance is calculated using the following formula:

2) Random Forest Classifier: RFC is a supervised learning

method. It infers that there is an immediate connection
between the total number of trees and the result it gets: the
bigger the number of trees, the more precise the outcome will
be. This classifier can be used for regression as well as
classification. For RFC algorithm if there are sufficient trees
then the classifier will not over fit the model, instead it avoids
the over fitting issues. This classifier can deal with the missing
quantities. Once the training is done, predictions are taken
from each individual tree and the average is calculated using
the following formula: Fig 4:- Sequence Diagram of the System

3) Support Vector Machine: SVM is also a supervised learning The Figure 4 illustrates the sequence diagram of the
method. It is also used for both classification and regression system model. The figure describes the sequence of steps to be
tasks. In this type of algorithm, there are data items which are taken while performing execution. The CNN model works in
considered as points in an n-dimensional space. This classifier the following sequence. User uploads a particular image of
finds the hyper plane by performing classifications between any digit which he wants to recognize. The image will be
the two classes. One of the main advantages of this algorithm processed by the system. On running the system code the
is that it provides a regularization parameter which avoids the output is generated that shows which is the digit uploaded by
over fitting problems. The block diagram shown below in the the user and also displays the accuracy rate predicted by the
Figure 3 describes all these above steps. model. On uploading image with different resolutions other

IJISRT19JU358 www.ijisrt.com 184

Volume 4, Issue 6, June – 2019 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
than the one mentioned in the code, the output generated  Anaconda3 5.3.1
shows error, and displays an error message to the user. Anaconda is a free and open-source appropriation of the
Python and R programming for logical figuring like
B. Algorithm Used information science, AI applications, large-scale information
The following Figure 5 describes the Data flow diagram preparing, prescient investigation, and so forth. Anaconda
of the proposed system model. There are two ways to provide accompanies in excess of 1,400 packages just as the Conda
input to the system. The user can either upload the image of package and virtual environment director, called Anaconda
the digit he wants to detect or the data from the MNIST Navigator, so it takes out the need to figure out how to
dataset. The input images are pre-processed. Using the introduce every library freely. Anaconda Navigator is a
different classifiers the recognized digits’ accuracy is graphical UI (GUI) incorporated into Anaconda appropriation
compared and the result is obtained. The results obtained are that enables clients to dispatch applications and oversee conda
displayed along with the accuracy. packages, conditions and channels without utilizing command-
line directions.

B. HARDWARE PLATFORM

 3.2.1 NVIDIA GeForce Graphic Card

Nvidia Corporation more regularly specified to as Nvidia
(adapted as NVIDIA), is an American innovation
organization. It structures graphical processing units (GPUs)
for the gaming and expert markets, just as a framework,
system on a chip units (SoCs) for the versatile figuring and car
showcase. It’s essential GPU product is named as "GeForce".
With outstanding GPU fabricating, Nvidia gives parallel
processing capacities to analysts and researchers that enable
them to effectively run superior applications.
Fig 5:- Dataflow Diagram of the system model
C. RESULTS
V. IMPLEMENTATION The following Figure 6 shows the front-end design of the
system output. There are four buttons for the four algorithms
as shown in the figure.
A. SOFTWARE PLATFORM

 Tensorflow
TensorFlow is an amazing information stream in
machine learning library made by the Brain Team of Google
and made open source in 2015.It is intended to ease the use
and broadly relevant to both numeric and neural system issues
just as different spaces. Fundamentally, TensorFlow is a low-
level tool for doing entangled math and it targets specialists
who recognize what they're doing to construct exploratory
learning structures, to play around with them and to transform
them into running programs. For the most, it can be
considered as a programming framework in which one can
entitle to calculations as graphs. Nodes in the graph speak the
math activities, and the edges contain the multi-dimensional
information clusters (tensors) related between them.

 Python 3.7
Python is broadly utilized universally and is a high-level Fig 6:- Front-end design of the system output
programming language. It was primarily introduced for
prominence on code, and its language structure enables The following figures show the sequence of steps to be
software engineers to express ideas in fewer lines of code. carried out to obtain the required output. The Figure 7 shows
Python is a programming language that gives you a chance to the commands to run the code of classifiers.
work rapidly and coordinate frameworks more effectively.

IJISRT19JU358 www.ijisrt.com 185

Volume 4, Issue 6, June – 2019 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165

Fig 7:- Command to run the Classifying code Fig 10:- Output generated by KNN

The Figure 8 shows the commands to run the code that The following Figure 11 describes the digit recognized
trains the images. by SVM algorithm. The confusion matrix is demonstrated
using a matrix where each row shows the examples in a
predicted class, while each column shows them in an actual
class.

Fig 8:- Command to train the code

The Figure 9 shows the commands that are used to run

the Tensorflow virtual environment.
Fig 11:- Digit Recognized by SVM algorithm

The Figure 12 shows the output generated by training

CNN.

Fig 9:- Commands to create virtual environment to run the

code

The Figure 10 shows the output generated by running the

KNN algorithm.
Fig 12:- Output generated by CNN

IJISRT19JU358 www.ijisrt.com 186

Volume 4, Issue 6, June – 2019 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
VI. CONCLUSION

In this paper, the Handwritten Digit Recognition using

Deep learning methods has been implemented. The most
widely used Machine learning algorithms, KNN, SVM, RFC
and CNN have been trained and tested on the same data in
order acquire the comparison between the classifiers. Utilising
these deep learning techniques, a high amount of accuracy can
be obtained. Compared to other research methods, this method
focuses on which classifier works better by improving the
accuracy of classification models by more than 99%. Using
Keras as backend and Tensorflow as the software, a CNN
model is able to give accuracy of about 98.72%. In this initial
experiment, CNN gives an accuracy of 98.72%, while KNN
gives an accuracy of 96.67%, while RFC and SVM are not
that outstanding.

ACKNOWLEDGEMENT

I would like to thank Mr. Bhavanishankar K (Asst Prof,

Dept of CSE, RNSIT, Bangalore) for his valuable suggestions
and comments that helped improving this works, this support
is greatly appreciated.

REFERENCES

[1]. M. Wu and Z. Zhang, Handwritten Digit Classification

using the MNIST Dataset, 2010.
[2]. A. Dutta and A. Dutta, Handwritten digit recognition
using deep learning, International Journal of Advanced
Research in Computer Engineering & Technology
(IJARCET), vol. 6, no. 7, July 2017.
[3]. Al Maadeed, Somaya, and Abdelaali Hassaine,
Automatic prediction of age, gender, and nationality in
offline handwriting. EURASIP Journal on Image and
Video Processing, no. 1 2014.
[4]. Gaurav Jain, Jason Ko, Handwritten DigitsRecognition,
Project Report, University of Toronto, 11/21/2008.
[5]. Hamid, Norhidayu Abdul, and NilamNur Amir
Sjarif,Handwritten recognition using SVM, KNN and
neural network, arXiv preprint
arXiv:1702.00723 (2017).
[6]. R.G.Mihalyi, Handwritten digit classification using
support vector machines, 2011.
[7]. Z. Dan, C. Xu, The Recognition of Handwritten Digits
Based on BP Neural Networks and the Implementation
on Android, In: 3rd International Conference on
Intelligent System Design and Engineering
Applications, pp. 1498-1509, 2013.
[8]. http://cvisioncentral.com/resources-
wall/?resource=135.

IJISRT19JU358 www.ijisrt.com 187