International Journal of Innovative Research in Computer Science & Technology (IJIRCST)

ISSN: 2347-5552, Volume- 8, Issue- 4, July- 2020

https://doi.org/10.21276/ijircst.2020.8.4.16
www.ijircst.org

Handwritten Digit Recognition Using Various

Machine Learning Algorithms and Models
Pranit Patil, Bhupinder Kaur


ABSTRACT- Handwritten digit recognition is a technique  There is no suitable and effective ideal Machine
or technology for automatically recognizing and detecting Learning model for handwritten digit recognition.
handwritten digital data through different Machine Learning Different researchers use different methods but there is
models. In this paper we use various Machine Learning no ideally suited model for recognizing handwritten
algorithms to enhance the productiveness of technique and digits.[9] In this paper topic is mainly focused on
reduce the complexity using various models. Machine performance and consistency of machine learning
Learning is an application of Artificial Intelligence that models.
learns from previous experience and improves automatically  Using defined image recognition patterns, the
through experience. We illustrate various Machine learning handwritten digits cannot be identified, since they are
algorithms such as Support Vector Machine, Convolutional specific for various writers.
Neural Network, Quantum Computing, K-Nearest Neighbor
Algorithm, Deep Learning used in Recognition technique. Machine Learning had mainly three type of learning as
follows:
KEYWORDS- Convolutional Neural Network, Support A. Supervised Learning
Vector Machine, HandWritten Digit Recognition, Artificial B. Unsupervised Learning
Intelligence, Deep Learning. C. Reinforcement Learning
A. Supervised Learning
I. INTRODUCTION
Supervised Learning in which model is trained with a
Currently, Handwritten Digit Recognition is a pivotal labeled dataset. We have two variables one is input and
concern in computer vision. Machine Learning technology other is output and we have to map from input to the output.
makes a machine efficient to perform pattern or text Like input(x) and output(y).
recognition. In fact, the time complexity of current y = f(x)
algorithms or models is very high because of contiguous Example of Supervised Learning Algorithm:
accumulation and incremental development of handwritten  Naive Bayes
digit sample collection of precision of identification [1].  Random Forest
The major concerns of handwritten digit recognition using
 Linear Regression
machine learning models are illustrated as follow [2]:
 Support Vector Machine
 Handwriting patterns differ according to the speaker it is  K-Nearest Neighbor
normally quite difficult to recognize even humankind  Decision Trees
handwritten digits as the writing types vary greatly.
 Similarities between handwritten digits, for instance six B. Unsupervised Learning
and four, that look the same based on writing style. Unsupervised Learning in which you need to allow the
Likewise, one and seven may look the same. model to work on unlabeled data, non-classified or
categorized. In these we have only input data and to input we
don’t have output data, i.e. input variable(x) not have output
variable.
Example of unsupervised Learning Algorithm:
 K-mean clustering
 Neural Network
 Multivariate Analysis
Manuscript received July 23, 2020  Anomaly detection
Pranit Patil, Student, B.Tech, Department of Computer
C. Reinforcement Learning
Science and Engineering, Lovely Professional University,
Phagwara, Punjab, India (email: [email protected]) Reinforcement learning is about sequential decision
taking. To what output is dependent on current input and
Bhupinder Kaur, Assistant Professor, Department of
next input depends on previous output input.
Computer Science and Engineering, Lovely Professional
Example of Reinforcement Learning algorithm:
University, Phagwara, Punjab, India
 Positive

Copyright © 2020. Innovative Research Publication. All Rights Reserve 337

Electronic copy available at: https://ssrn.com/abstract=3692489
 Handwritten Digit Recognition Using Various Machine Learning Algorithms and Models

 Negative
 Markov Decision Process
 Q Learning
II. LITERATURE SURVEY
Year Paper Title Abstract Analysis

IJTP[1] Usage of Quantum Study the efficiency of Before this Algorithm time complexity was
K-Nearest Neighbor quantum computing using 12R after these algorithm complexity reduce
Algorithm to improve Grover Algorithm and K- to me O(kM2) for accuracy
handwritten digit Nearest Algorithm Algorithm - K- Nearest Neighbor
recognition
ICOASE[2] A comparison of three Comparison of three After Evaluation for all algorithms on 46K
classification algorithms classification algorithms In instances with 10 cross validations for these
for the identification of other terms Multilayer K-star get highly accuracy of 82.36% follow by
handwritten digits Perceptron (MLP), Naive NB of 67.04% then MLP by 78.35%.
Bayes(NB), and K-Star Algorithm: K-Star
IJECE[3 Recognition of This paper tested the In these model is trained using decision tree
handwritten digits with standard digital dataset from algorithm with a standard dataset consist of 42K
classification of decision kaggle for recognition of rows and 720 columns and from this model the
tree: A machine learning handwritt-en digits using a accuracy came to be 83.4%. Accuracy for 0-9
method machine learning algorithm digits as follows: 0 = 83.5%, 1
decision tree. Accuracy will = 93.7%, 2 = 83.6%, 3 = 83.1%,
from 0 -9 digits 4 = 83.8%, 5 = 83.6%, 6 = 83.4%,
7 = 83.8%, 8 = 84.1%, 9=
83.7%
Algorithm: Decision Tree
IEEE[4 Development of a high Determine accuracy and After completion of training with dataset the
precision handwritten efficiency using accuracy of neural network was found to be
digit recognition detector Convoltion-al neural 92.6% for training set and for test set it was
based on a network with two layers on 90.1%. Convolutional neural network is much
Convolution-Neural with 32 images and another accurate in-depth learning models and give
Network with 64 images with some excellence performance.
neurons on each layer. Algorithm: CNN

MDPI[5 Improved Handwritten It optimized various SGD In these algorithm the learning rate in training
Digit Recognition Using algorithm to improve the parameter is of 0.01 and epoch count of maximum
Convolution-al Neural efficiency of handwritten 4. The accuracy was achieved third layer
Networks (CNN) digit recognition. Convolutional neural network was 99.76% and
In these paper they explore for fourth layer of (CNN) we highest accuracy rate
the various option with for recognition of 99.76%.
stride size, kernel, padding For MINST dataset the model of CNN with three
number of layers and layers with optimizer provided best accuracy of
receptive. For testing 99.89%
MNIST dataset is used. Algorithm: Optimized CNN

IEEE[6 Strengthening Using two Q state it’s Accuracy according to two Q state model as
Handwritten Digit become very simple and follows: 1. on MNIST dataset accuracy
Recognition with Two easy to due to it required less is 99.0% 2. on USPS dataset accuracy is
State Q-Learning parameter to optimize and 99.7% 3. on MATLAB digital dataset
easy to function on MNIST accuracy is 100.0%.
Digital dataset, USPS Algorithm: Double Q Learning
dataset and MATLAB
dataset
JILSA[7] MCS HOG features and The Histogram of Directed In experiment the 10 fold cross and independent
handwritten digit Gradient (HOG) with a test set is used for validation to determine
recognition system based Support Vector Machine ( accuracy and the accuracy for both was founded to
on SVM SVM) algorithm is used in be 99.26 % and 99.36% respectively
this Multiple Cell Size Algorithm: SVM
(MCS) to identify digits
from the MNIST digital
dataset.

Copyright © 2020. Innovative Research Publication. All Rights Reserve 338

Electronic copy available at: https://ssrn.com/abstract=3692489
 International Journal of Innovative Research in Computer Science & Technology (IJIRCST)
ISSN: 2347-5552, Volume- 8, Issue- 4, July- 2020
https://doi.org/10.21276/ijircst.2020.8.4.16
www.ijircst.org

III. ALGORITHM SURVEY

We used various Machine Learning algorithms and
various models to get best accuracy by testing and training
with various digital datasets.
1. K-Nearest Neighbour is a basic algorithm for
classification. It is a kind of supervised learning and
mainly focused on pattern recognition and detection for
intrusion detection
2. K-Star is also called as instance base classifier. This
recognition is done by comparing instances of index of
pre recognized or classified sample [2].
3. Decision tree [3] is a kind of supervised algorithm for
learning and it is mostly used for a problem like
classification and solving problems of recognition. [8]In
these model we break problem in data by making
decisions like a questions in form of 0 and 1. Fig 3: CNN Digit Recognition

5. Double Q Learning [6],[15] used features with huge

dimensions along with sates and used in conventional
reinforcement network based objects. It is also used for
overestimated values. [11]Double Q Learning is faster
and efficient then Q Learning.

Fig 1: - Decision tree

4. Convolutional Neural Network (CNN) [4] constitutes a
type of deep neural network. Some advantages of its are
Hidden feature extraction and other is local field and
share weights. [5] In deep learning CNN improves the
research computer vision and practical improvement and
performance. [10],[13]It consists of three layers’ Input
layer, Hidden layer and classification layer.

Fig 2: CNN Architecture

Fig 4: Double Q Learning Recognition algorithm

Copyright © 2020. Innovative Research Publication. All Rights Reserve 339

Electronic copy available at: https://ssrn.com/abstract=3692489
 Handwritten Digit Recognition Using Various Machine Learning Algorithms and Models

6. Support vector Machine (SVM) [7],[14] is a supervised [5] Al-Wzwazy, Haider. (2016). Handwritten Digit
type of machine learning. It is a more accurate Recognition Using Convolutional Neural Networks.
classification tool. [12]It results in binary classification International Journal of Innovative Research in
or regression challenges. It has capability to handle Computer and Communication Engineering. 4.
multiple categorical and continuous variable. [6] Hafiz, Abdul & Bhat, Ghulam. (2020). Reinforcement
Learning Based Handwritten Digit Recognition with
Two-State Q-Learning. 2007.01193.
[7] Khan, H. (2017) MCS HOG Features and SVM Based
Handwritten Digit Recognition System. Journal of
Intelligent Learning Systems and Applications, 9,
21-33.
[8] M. Y. W. Teow, "Understanding convolutional neural
networks using a minimal model for handwritten digit
recognition," 2017 IEEE 2nd International Conference
on Automatic Control and Intelligent Systems
(I2CACIS), Kota Kinabalu, 2017, pp. 167-172, doi:
10.1109/I2CACIS.2017.8239052.
[9] Alsaafin, A. and Elnagar, A. (2017) A Minimal Subset
of Features Using Feature Selection for Handwritten
Fig 5: SVM Model Digit Recognition. Journal of Intelligent Learning
Systems and Applications, 9, 55-68.
III. CONCLUSION [10] Wu S., Wei W., Zhang L. (2018) Comparison of
From the survey, it can be achieved that the Machine Machine Learning Algorithms for Handwritten Digit
learning models and algorithms are very efficient to Recognition. In: Li K., Li W., Chen Z., Liu Y. (eds)
recognize or detect patterns with different writing style. Computational Intelligence and Intelligent Systems.
Testing various algorithms gives results in terms of lesser ISICA 2017. Communications in Computer and
the complexity more will be efficiency and accuracy for any Information Science, vol 874. Springer, Singapore
digital data sets. According to survey we have founded that [11] Jha, G., Cecotti, H. Data augmentation for handwritten
by using Convolutional neural network accuracy increase to digit recognition using generative adversarial
the 99.89% accuracy most among all Similarly, Double Q networks. Multimed Tools Appl (2020).
learning algorithm also given high accuracy but in [12] Kulkarni, S. R., & Rajendran, B. (2018). Spiking neural
MATLAB dataset only. SVM also given accuracy of networks for handwritten digit
99.36%. CNN produce most as it used layered architecture recognition—Supervised learning and network
which improve the computer vision and follow a optimization. Neural Networks, 103, 118–127.
hierarchical model which work on making network and give [13] Qiao, J., Wang, G., Li, W., & Chen, M. (2018). An
fully connected layers so neuron get connect to each other adaptive deep Q-learning strategy for handwritten digit
and output is processed. recognition. Neural Networks.
[14] S. Aly and S. Almotairi, "Deep Convolutional
REFERENCES Self-Organizing Map Network for Robust Handwritten
Digit Recognition," in IEEE Access, vol. 8, pp.
[1] Wang, Y., Wang, R., Li, D. et al. Improved
107035-107045, 2020, doi:
Handwritten Digit Recognition using Quantum
10.1109/ACCESS.2020.3000829.
K-Nearest Neighbor Algorithm. Int J Theor
[15] R. Jantayev and Y. Amirgaliyev, "Improved
Phys 58, 2331–2340 (2019).
Handwritten Digit Recognition method using Deep
[2] M. B. Abdulrazzaq and J. N. Saeed, "A Comparison of
Learning Algorithm," 2019 15th International
Three Classification Algorithms for Handwritten Digit
Conference on Electronics, Computer and
Recognition," 2019 International Conference on
Computation (ICECCO), Abuja, Nigeria, 2019, pp.
Advanced Science and Engineering (ICOASE), Zakho -
1-4, doi: 10.1109/ICECCO48375.2019.9043235.
Duhok, Iraq, 2019, pp. 58-63, doi:
10.1109/ICOASE.2019.8723702.
[3] Assegie, Tsehay & Nair, Pramod. (2019). Handwritten
digits recognition with decision tree classification: a
machine learning approach. International Journal of
Electrical and Computer Engineering (IJECE). 9. 4446.
10.11591/ijece.v9i5.pp4446-4451. K. Elissa, “Title of
paper if known,” unpublished.
[4] D. Ge, X. Yao, W. Xiang, X. Wen and E. Liu, "Design
of High Accuracy Detector for MNIST Handwritten
Digit Recognition Based on Convolutional Neural
Network," 2019 12th International Conference on
Intelligent Computation Technology and Automation
(ICICTA), Xiangtan, China, 2019, pp. 658-662, doi:
10.1109/ICICTA49267.2019.00145.

Copyright © 2020. Innovative Research Publication. All Rights Reserve 340

Electronic copy available at: https://ssrn.com/abstract=3692489