Handwritten Digit Recognition Using Encryption

Methods
Abdelmoumene Zebdi1, Lakhdar Laimeche 2, Abdelja1il Gattal 3
1
Mathematics and Computer Science, Larbi Tebessi Univesity, Tebessa, , 12002, Algeria,
2,3
Laboratory of Mathematics, Informatics, and Systems (LAMIS),
University of Larbi Tebessi, Tebessa, 12002, Algeria
1
[email protected]
2
[email protected]
3
[email protected]
2022 4th International Conference on Pattern Analysis and Intelligent Systems (PAIS) | 978-1-6654-6161-0/22/$31.00 ©2022 IEEE | DOI: 10.1109/PAIS56586.2022.9946886

Abstract— Over the past years, deep learning techniques has had more advanced recognition algorithms to manage handwriting
a strong impact on many areas of technical intelligence, including HDR tasks more accurately.
handwriting recognition. Handwriting recognition it is defined as In this paper, we propose an offline recognition system for
the domain which allow a computer to interpret intelligible handwritten digit recognition based on deep learning. To
handwritten inputs from different sources. It is plays an
improve the performance of the proposed method, we have
important role in many areas, such as authenticating bank
checking, exchanging remote computer files, and document played on the quality and diversity of training data. This
duration. In this paper, we present a novel handwritten digit process allows us to increase the size of the training data using
recognition method based on deep learning. In order to improve images that already exist in the training set.
the performance of recognizing handwritten digits, we The remainder of this paper is organized as follows:
have proposed a new data augmentation method by using Section 2 gives the related work to handwriting digits
different encrypted digit images and image fusion recognition; Section 3 introduces the background of the
techniques. The experimental results based on the CVL database encryption techniques and the image fusion method used in
shown that the proposed method achieved higher accuracy rates this work; Section 4 describes the proposed methodology
that were compared to the state of the art. followed by our conclusion in section 6.
Keywords— Handwriting Digit Recognition, Chaotic systems, II. RELATED WORK
Cryptography, Deep Learning.
Chawki et al. [2] proposed an isolated handwritten digit
I. INTRODUCTION recognition method based on oBIFs and Background features
extraction techniques. In this work, authors show how a
In the field of artificial intelligence, deep learning has had a
combination of the oriented basic image features (oBIFs) with
strong impact in many areas. Among these areas, handwritten
background concavity features can be used effectively to
digit recognition is considered as one of the most difficult
improve the performance of handwritten digit recognition
problems. Handwritten recognition attracted the interest of
systems. Using a one-for-all support vector machine (SVM)
researchers for several decades and continues to be a
and the CVL database, a series of experiments are performed
relatively open field of research due to its wide range of
using different configurations and feature sets which achieved
practical applications. Advances in this field are now evident
a higher recognition rate of 95.21%.
in a variety of applications, including the automatic reading of
Authors in [3] present a handwritten digit string recognition
bank checks, postal addresses, and form addresses.
method based on Convolutional Neural Network (CNN) deep
Systems of handwriting recognition can be grouped in two
learning technique. This method present a novel CNN-only
classes: offline recognition systems and online recognition
architecture applied to handwritten digit strings. Experiments
systems. The former works on scanned images obtained from
are conducted on three databases CVL, ORAND-CAR-A and
a scanner or camera. While the second, takes data directly
ORAND-CAR-B which achieved higher recognition rates of
from an electronic pen or digitizing tablet and transforms it
42.69%, 92.2% and 94.02%, for CVL, ORAND-CAR-A and
into digitized text.
ORAND-CAR-B, respectively.
The main challenges of these systems, particularly, the
Bouchoucha et al [4] propose a handwritten digit recognition
offline recognition systems are: the different levels of image
method based en image encryption and chaotic systems. In
quality which is critical for handwriting recognition, noisy
this work, to improve the performance of their recognizing
images, and the variety of individual handwritings which
handwritten digits method, they plays on data augmentation
makes recognition harder [1].
technique using RSA encrypted technique and two chaotic
To tackle these challenges, many handwriting recognition
systems including Arnold Cat Map and Henon techniques.
systems have been proposed based on :1) using higher quality
Using SVM machine learning technique and CVL database,
images, 2) removing the background noise, or 3) developing

978-1-6654-6161-0/22/$31.00 ©2022 IEEE

experimental results achieved higher rates of 85.99%, 61.15% processes used to achieve the encryption process: key
, 85.99%, and 85.92% for the unencrypted database, RSA generation, encryption and decryption [9].
encrypted database, Arnold Cat Map database, and for Henon
3) Arnold's Cat Map: In 1845, Pierre Verhulst proposed the
data set, respectively.
logistic map [10], which is a nonlinear dynamic map and
The proposed handwritten digit recognition system in [5] is
based on machine and deep learning algorithms. In this paper, which is considered one of the most popular chaotic maps.
authors are performed a handwritten digit recognition system The logistic map is a chaotic system whose complex behavior
based on MNIST databases using Support Vector Machines can come from nonlinear dynamic equations and very simple
(SVM), Multi-Layer Perceptron (MLP) and Convolution given by:
Neural Network (CNN) models. The main objective of this xn +1 = Γl ( xn , µ ) (1)
work consists to compare the accuracy of the models stated
above in terms execution time to choose the best possible µx n = (1 − x n ), µ ∈ [0,4], x n ∈ [0,1] (2)
model for digit recognition.
Hala et al. in [6] propose a handwritten digit recognition where xn is the system state with n=0, 1, 2 ... and µ is a control
system using a set of machine learning and deep learning parameter. Iteratively, this system generates a sequence from
algorithms. Experimental results performed on the MNIST x 0 ∈ [0,1] called the initial state. Depending on the µ , Γl
database shows that the CNN algorithm is efficient in the
values, it can be a convergent sequence, an oscillating
recognition process, achieving an accuracy of 96 % and the
sequence or a chaotic sequence. Thus, these systems are
results obtained have been very convincing and satisfactory.
considered chaotic µ ∈ [3.75,5] and purely chaotic if µ ≅ 4 .
Improved handwritten digit recognition method based on
Convolutional neural networks is proposed in [7]. The main 4) Henon map encryption: is developed by Henon in 1976,
objective of this work consist to comparable the performance the Henon map is a two dimensional iterated discrete time
of the proposed system using a pure CNN architecture Thus, dynamical system with a chaotic attractor. It can be expressed
CNN architecture is proposed in order to achieve accuracy using the following equations [11,12]:
even better than that of ensemble architectures, along with
reduced operational complexity and cost. Moreover, authors x ′ = y ′ − ax 2 + 1 (3)
present an appropriate combination of learning parameters in
designing the CNN that leads to reach higher rates in y ′ = bx (4)
classifying MNIST handwritten digits. Experimental results Where, n represent the image size, x and y are the original
achieved a recognition accuracy of 99.87% for a MNIST image's pixel position, x', y' are the pixel position after the
database. mapping, and a, b are a control parameters system.
III. BACKGROUND B. DWT fusion method
In this section, we describe briefly the different techniques DWT (Discrete Wavelet Transform) fusion of images it is
used in our method including the encryption techniques (AES the first transform used in the field of image fusion [13]. In
for symmetric encryption method), RSA for asymmetric the case of multi-resolution decomposition, at each level an
encryption method, Arnold Cat Map, Henon Map and DWT image is decomposed into an approximation and a detailed
fusion method. sub bands. Fusion in the plane of the wavelets consists first of
decomposing an image into two images I1 and I2.
A. Encryption techniques
Each image will then be represented by its last
1) AES encryption: the Advanced Encryption Standard approximation sub band and a set of wavelet coefficient
(AES) algorithm is known for its security as well as its speed. planes corresponding to the details of the image at different
NIST recommends it as a new encryption standard to replace scales. The generation of the merged image amounts, on the
DES. It encrypts 128-bit data blocks in 10, 12, or 14 rounds, one hand, to producing a plane of the last approximation and,
depending on the key size [8]. We note that each round on the other hand, to synthesizing its wavelet coefficients by
consists of the following four steps: 1) non-linear byte the use of a fusion model. The merged image IF is then
substitution, 2) shift row transformation, 3) mix columns produced by applying the inverse of wavelet transformation
transformation, and 4) round key transformation. (Inverse DWT-IDWT).
2) RSA encryption: RSA is one of the first widely used To synthesize the coefficients of each band (approximation
systems for secure data transmission. It is an asymmetric and details) of the merged image at scale i, it is necessary to
encryption algorithm. It uses two different keys, a public key define two fusion models allowing combining the wavelet
known to all while the private key is kept secret. Only coefficients of the images I1 and I2, one for the approximation
authorized users know how to open the message. The RSA and the other for the three details. Knowing that the two
algorithm has high encryption accuracy and is fast in fusion models are the same for each level of decomposition.
processing. The key length of this algorithm is more than
1024 bits. The block size of the RSA algorithm is 446 bytes
and one round of encryption. There are three different
 For a single level of decomposition, the images I1 and I2 can As shown in figure 2:
be represented by the respective sets of decomposition planes
(D1 and D2), such as for example: • In the first scenario, we have tested the performance of the
proposed method for each database including the original
{
D1 = I 1LL , I 1LH , I 1HL , I 1HH } (5) CVL database and the encrypted databases,
• In the second scenario, we have increased the size of the
D2 = {I 2 2
LL , I LH
2
, I HL 2
, I HH } (6) original database with the encrypted databases resulting
i i i four databases.
Where I LH , I HL , I HH are the wavelet coefficients
corresponding to the sub bands details of the images (i=1, 2), • In the last scenario, we have used the described DWT
i fusion method to merge the original images with the
and I LL are the respective approximation sub bands. The
encrypted databases separately resulting four databases.
preservation of the characteristics of the two images is
ensured by the average of the two approximations of the In order to improve the handwriting recognition, we have
input images: proposed three systems based on Alex Net deep learning
technique [14] and three protocols including original database,
1 data augmentation based on the encrypted databases, and the
I FLL ( x, y ) =
[ I1LL ( x, y ) + I 2 LL ( x, y )] (7)
2 databases based on the fusion of the original images and the
IFLL represents the approximation of the fused image. For the encrypted ones.
three detail sub bands, each sub band is divided into windows The proposed architecture of Alex Net showed in figure 3
of size 3×3. Next, we calculated the sum of the absolute contains five convolutional layers including three Max-
pooling layers, three fully connected layers, and a linear layer
values of all the pixels in each window. Finally, generate
with Soft-max activation in the output. Moreover, it uses the
Binary Decision Maps (DM), for the three detail sub bands, dropout regularization method to reduce over-fitting in the
using the following equation: fully connected layers and applies Rectified Linear Units
(ReLUs).
1 if max 3×3 ( I 1x ) ≥ max 3×3 ( I 2 x )
DM x =  (8)
 0 Else
where x ≡ [ HL, LH , HH ] are the binary decision maps for the
three detail sub bands. Based on these maps, the three bands
of the fused image details, IFX, are generated. Finally, the
inverse transform, IDWT, is then applied to the four merged
bands to generate the merged image.

IV. PROPOSED METHOD

In this paper, we have proposed three scenarios of handwriting
digits recognition. The former, is based on the original Fig 3 proposed Alex Net architecture for handwriting digits recognition
database and encrypted databases, the second is based on data
augmentation technique, and the last scenario is based on the V. EXPERIMENTAL RESULTS
fusion of original images with encrypted images as shown in
A. Database description
figure 2.
In our experiments we have used CVL Single Digit
database which is part of the CVL Handwritten Digit
database (CVL HD). It is contains samples from 303 writers
Database Encryption and was collected mostly among students at Vienna University
of Technology and an Austrian secondary school. There are
Fusion of original 10 classifications in the entire single-digit CVL data set (0–9).
Data augmentation images with The train set consisted of 7000 numbers (700 numbers per
encrypted images class) from 67 writers. A different group of 60 writers was
used to create an equal-sized verification group. The
Scenario 1 Scenario 2 Scenario 3 validation set can be used to estimate and validate parameters,
but not for supervised training. There were 2,178 numbers per
class in the testing set, resulting in 21,780 assessment
Alex Net samples from the remaining 176 writers [14].
Decision
Deep Learning B. Encrypted database generation
To test the performance of our handwriting digits
performance, we generate two databases based on the
Fig 2 Scenarios of the proposed handwriting digits recognition
encryption techniques: RSA and AES, and two databases generated database separately, we can learn from this table
based on chaotic systems including Arnold Cat Map and that proposed method achieves concurrent accuracies with the
Henon Map. Figure 4 shows samples of the generated different databases achieving a higher accuracy equal to
databases. 97.16%, except the database based on AES encryption
technique which gives a poor accuracy equal to 10.01%.
For the second protocol which based on the augmentation
of the original database with each encryption technique,
separately, we observe that the data augmentation technique
perform the performance of the proposed handwriting digits
recognition particularly for the generated database based on
(a) AES encryption achieving 52.38% compared to 10,01%
obtained with the first protocol. In overall, the obtained results
using the second protocol are higher than the first one.
Finally, the last protocol which based on the fusion of the
original image with the encrypted image achieves higher
handwriting digits recognition for all generated database
(b) particularly using the Arnold Cat Map logistic map which
achieves 98.21%.
Table II Comparative results of the proposed method and the state-of-the art

database approach Accuracy

Paper [2] CVL oBIFs + SVM 95.21%
CVL 42.69%
(c) Paper [3] ORAND-CAR-B CNN 92.20%
ORAND-CAR -A 94.20%
Paper [4] CVL SVM 85.99%
Henon Map
97.16%
database
CVL data
augmentation 97.07%
with Henon Map Alex Net model
Our work
Fusion of CVL
(d) and Arnold Cat 98.21%
Map images
Fig 4 Samples of generated databases based on : (a) RSA encryption, (b)
AES encryption, (d) Henon Map, and (d) Arnold Cat Map.
We have quantitatively compared the performance of our
C. Results and discussion handwritten digits recognition against three states of- the art
Many experiments are performed in this work to improve approaches as shown in table II. We can assert that our
the handwriting digits recognition based on three protocols: method outperforms and improves on three states of- the art
approaches. In the case of using each database separately, the
• Testing the original database and the generated databases proposed method achieves an accuracy rate equal to 97.16%
separately, using Henon Map database. Based on the data augmentation
• Testing the emerged original database with the generated technique, the proposed method achieves an accuracy rate
databases separately, equal to 97.07% increasing the CVL database by the Henon
Map database. Finally, fusion of CVL and Arnold Cat Map
• Using the fusion images that obtained by the original images gives the higher accuracy rate equal to 98.21%
images and the generated images, separately, using DWT compared to the proposed two first protocols as well as the
fusion method. state-of-the art.
Table I Comparative results of Alex Net based on the proposed protocols
VI. CONCLUSION
Accuracy
Encryption method Protocol 1 Protocol 2 Protocol 3
This paper introduces a data augmentation-based method
CVL Gray scale image 96.77% / / for off-line handwritten digits recognition. In this paper, we
RSA 96.70% 97.07% 95.53% proposed new databases generated from the CVL database
AES 10.01% 52.38% 95.60% using encryption techniques, chaotic systems, and DWT
Arnold cat map 94.58% 96.55% 98.21%
Henon map 97.16% 91.20% 97.69%
fusion method that are able to characterize the digits
handwriting style of the writers. Furthermore, the advantage
Table I gives the different results obtained with Alex Net of the proposed diversity of training data is that it captures
and the proposed protocols. In case of the first protocol which writer-specific details such as: direction and curvature of ink
based on the testing of the original database and each trace, preferred handwriting slant, and ink-trace width and
shapes. For each encrypted methods twelve handwriting [10] Laimeche L., Meraoumia A., Bendjenna H.: (2019), “Enhancing LSB
embedding schemes using chaotic maps systems”, In: Neural
databases are generated, were employed to test our proposed
Computing and Applications, Springer-Verlag Berlin Heidelberg, 2019.
method, whereas the digits recognition is carried out by using
the Alex Net deep learning technique. The comparisons of our
proposed method and the current existing handwriting digits
recognition, methods prove the data augmentation and images
fusion methodology.

REFERENCES
[1] Qi Wu, and Siyuan Zhang, Stanford University Shaohan Xu,
Application of Neural Network In Handwriting Recognition.
[2] Chawki Djeddi, Youcef Chibani, Imran Siddiqi Abdeljalil Gattal,
Isolated Handwritten Digit Recognition Using oBIFs and Background
Features, 2016.
[3] Shujing Lyu, Yue Lu, Hongjian Zhan, Handwritten Digit String
Recognition using Convolutional Neural Network, 2018
[4] Bouchoucha Mohammed Tayeb, Handwritten Digits
Recognition using Image Encryption, 2020.
[5] Rishika Kushwah, Samay Pashine Ritik Dixit, Handwritten Digit
Recognition using Machine and Deep Learning Algorithms, 2021.
[6] Hala Djerouni, Développement d’un système de reconnaissance de
chiffres manuscrits, 2021, Master Thesis, University Echahid Hamma
Lakhdar – Eloued, Algeria.
[7] Savita Ahlawat, Improved Handwritten Digit Recognition Using
Convolutional Neural Networks (CNN), Received: 25 May 2020.
[8] Abhishek Sachdeva Prerna Mahajan, A Study of Encryption Algorithms
AES, DES and RSA for Security.
[9] Atheer Sultan Almutiri, Bashaier Alqahtani, Rahaf Mohammed Alamri,
Hanan Fahhad Alqahtani, Nada Nasser Alqahtani, Ghadeer Mohammed
alshammari, and Azza. A. Ali Dalia Mubarak Alsaffar, Image
Encryption Based on AES and RSA Algorithms, March 24, 2020.
Computing and Applications, Springer-Verlag Berlin Heidelberg, 2019
[11] Laimeche L., Meraoumia A., Houam L., Bouchemha A., (2021), "An
Effective Framework for Secure and Reliable Biometric Systems based
on Chaotic Maps", International conference on Pattern Recognition and
Artificial Intelligence, Morocco.
[12] Niraj San,Baddigam Asha,Savvy Prasad,S.S.Sahu Rupesh Kumar Sinha,
Chaotic Image Encryption Scheme Based on Modified Arnold Cat Map
and Henon Map.
[13] Shekhar Karanwal, Davendra Kumar and Rohit Maurya, “Fusion of
Fingerprint and Face by using DWT and SIFT”, International Journal of
Computer Application, Jun 2010.
[14] Amit Choudhary, Anand Nayyar , Saurabh Singh , Byungun Yoon
Savita Ahlawat, Improved Handwritten Digit Recognition Using
Convolutional Neural Networks (CNN), Received: 25 May 2020.
[15] F. Kleber, S. Fiel, M. Diem, R. Sablatnig CVL-Database: An off-line
database for writer retrieval, writer identification and word spotting
International Conference on Document Analysis and Recognition, IEEE,
Washington, DC, USA (2013), pp. 560-564,