Offline Signature Verification Using Ensemble Learning

Avijit Bose, Aniket Paul, Debanjan Bhattacharjee, Tusharika Mandal, Rene Ghosh,
Satyajit Chakrabarti

Institute of Engineering and Management, Kolkata, India

Abstract. Signature Verification is crucial in its role in many real-

world applications, such as banking, in some legal documents etc.
However, conventional approaches that have been followed over many
years are susceptible to forgeries. The accuracy of these types of mod-
els should be sufficiently high since it is being used by the banks, courts
where signatures act as a prominent evidence in validating the identity
of an individual. This paper proposes a novel ensemble learning ap-
proach to signature authentication using two robust deep learning mod-
els. The ensemble learning influences the complementary strengths of
CNN and Siamese Network to achieve a higher accuracy and robust-
ness in signature verification. It seamlessly accommodates images re-
ducing the likelihood of false positives and false negatives. This growth
in the development of signature verification systems could mitigate the
potential for forgery and involvement of any third-party person who
judges the validity of the signatures.

Keywords: Convolutional Neural Network, Siamese Network, Ensem-

ble Learning

1 INTRODUCTION

Over the past years, signature verification has served as a critical feature in ensuring
the authenticity and security of various transactions, documents, and systems. Tradi-
tional methods often faced challenges in accurately distinguishing between genuine
and forged signatures, thereby encouraging the need to develop a robust solution.
Moreover, signature verification can be classified into two types, offline signature
verification which is carried out by analyzing the fluency of the pattern found in the
signature or by the visual comparison of the signature pattern with previously collect-
ed samples [1]. The other is the online signature verification or the dynamic signature
verification method where the signatures are verified after analyzing pen pressure,
pen inclination, position of pen and many more features [1]. In response to the above
problem, many deep learning techniques or algorithms have evolved to serve the pur-
pose. Among several techniques, ensemble learning has emerged as a promising ave-
nue to reinforce the accuracy and robustness of signature verification systems. Deep
Learning, characterized by its ability to autonomously learn even the minute represen-
tations from the data, has shown substantial success in many diverse domains includ-
2

ing image and sequential data analysis. Within the spectrum of deep learning, various
architectures such as Convolution Neural Network as shown in Fig. 1 is a class of
deep, feed-forward artificial neural network mostly applied to analyzing images [5].
Conversely the Recurrent Neural Network excels in capturing sequential patterns
aligning with the dynamic nature of signature writing. Additionally, the Siamese Neu-
ral Network plays a vital role in identifying the distinct similarities between different
signatures. In context to genuine and forged signatures, forgery has been classified
into three different types namely, random forgeries which are produced without
knowing the name of the signer nor the pattern of the signature [3]. Then comes the
simple forgeries that are produced by having the knowledge of the signer’s name but
not knowing the pattern of their signature [3]. Lastly, the skilled forgeries which are
produced by an individual who after examining an authentic signature, tries to imitate
it with utmost precision [3]. In order to build a robust software, ensemble learning has
significantly evolved. Ensemble learning combines multiple models into a single
model. The inter-dependent utilization of multiple deep learning models, each of
which are specialized in distinct aspects of signature analysis leading to increased
overall accuracy and reliability of the signature verification system. Various pre-built
Convolution Neural Network (CNN) models have been proposed such as [8] had used
VGG16, VGG19, ResNet50, InceptionV3. The above pre-built models were trained
on a particular dataset and results were compared.

Fig. 1. CNN Architecture

 3

Researchers have utilized [3] many feature extraction methods like structural,
texture, geometric, and global transformation combined with neural networks to
achieve an accuracy of 96% along with an error rate of approximately equal to
0.05. In [13], researchers applied the hidden Markov Model (HMM) with Dis-
crete Cosine Transform (DCT) for feature extraction, achieving an accuracy of
99.2%. Our proposed work approach relies on the concept of ensemble learning
where we combine two diverse model’s predictions namely, Convolution Neural
Network (CNN) and Siamese Neural Network and generate the results based on
majority voting.

2 SECTION TWO

2.1 Related Work

With the progress of Deep learning and increase in the compute power of edge
devices, there has been a significant development in the field of Signature Veri-
fication System. The researchers in [1] utilized various feature extraction meth-
ods — structural, texture, geometric— combined with neural networks to attain
an accuracy of 96% with an average error rate of approximately 0.05. The re-
searchers in [2], [13] employed a fusion of integrated wavelet transform, dis-
crete cosine transform and principal component analysis before training a neural
network, achieving an accuracy of approximately 96%. Manoj Kumar in [3]
used backpropagation neural network for signature recognition by retrieving,
preprocessing and extracting features from signature. Rajib Ghosh in [4] pro-
posed a RNN-based model for verifying handwritten signatures, including fea-
ture extraction methods like trajectory slope, change of directory and cent of
mass. The performance of the RNN model outperformed the CNN model in this
case. In [5], the researchers focused on improving accuracy using synthetic sig-
nature dataset, employing Google-Net architectures (Inception v1 and Inception
v3). The Inception v1 showed a lower EER value as compared to Inception v3.
The researchers in [6] developed a CNN model using TensorFlow, achieving a
remarkable accuracy of 99.7% when the dataset is split into 80:20 ratio. In [7],
the researchers designed a model based on DAG-CNN, especially designed to
prevent forgery by just looking at the signature structure and contours, and
achieved a high accuracy in identifying genuine signatures. In [8], the research-
ers proposed a combination of CNN-LSTM which outperformed both CNN and
LSTM individually, achieving an accuracy of 70% - 80%. Zihan Zeng, and Jing
Tian in [9] proposed a VGG16 model classifying signatures in the SIGCOMP
2011 dataset as genuine and forged, achieving an accuracy of 88 % to 94%.
Gabe Alvarez, Blue Sheffer, and Morgan Bryant in [10] designed a mutual sig-
nature dense net (MSDN) that helps extract minute details from the signatures.
Researchers in [11] utilized an ANN for offline signature verification, using a
low pass filter, binary image conversion and features like area, centroid, kurtosis
and skewness. It achieved an accuracy of approximately 93%. In [12], the re-
searchers proposed a static based Support Vector Method (SVM) framework
that compares their model with various other methods like template matching
4

methods, structural patterns etc. This trained the model on various dataset name-
ly, MNIST, UCOM etc. This model showed an accuracy of approximately 88%.
Dr S. Adebayo Daramola, and Prof. T. Samuel Ibiyemi in [14] proposed a Ge-
netic Algorithm (GA) which selects features for Support Vector Mechanism
classification. The model was trained on GPDS and CEDAR dataset and
achieved low error rates 4.17, 5.42 respectively. S. Rokade, S. K. Singh, S.
Bansod and P. Pal in [16] created a CNN model and trained it for two distinct
modes much as Writer-Dependent (WI) and Writer-Independent (WI). Mosaher,
Q.S.- ul and Hasan, M in [17] used a deep convolutional neural network and
trained on a completely new dataset achieving an accuracy of 95.5%. [18] used
a learning-based neural network and attained the most promising results for the
OfSV systems on public datasets. Indrajit Bhattacharya, Prabir Ghosh, Swarup
Biswas in [20] introduced an off-line signature verification and recognition sys-
tem utilizing the pixel matching technique. They assessed the performance of
their proposed model by comparing it with the existing back-propagation meth-
od of Artificial Neural Network (ANN) and Support Vector Machine (SVM)
technique. Dewi Suryani, Edy Irwansyah, Ricki Chindra in [21] proposed an of-
fline signature recognition and verification system employing the efficient fuzzy
Kohonen Clustering Networks (EFKCN) algorithm. The accuracy achieved with
the EFKCN algorithm demonstrated improvement compared to the accuracy re-
ported in previous research, which stood at 53%. In [22], the researchers have
proposed a hybrid method of feature extraction that is histogram of oriented
gradient (HOG) that produces records of angular introduction in a specific area
and CNN that uses convolution and pool to extract and recognize feature from
image. The model achieved an accuracy of approximately 95.4%. Tayeb Bah-
ram in [23] used Modified Local Binary Pattern and In-track width and space to
extract features from a preprocessed image. The researcher in [24] proposed a
handwritten verification system using OC-SVM for WI parameters. The exami-
nation was conducted on CEDAR and GPDS datasets. In [25], the researchers
studied the biometric classification error and introduced an adapted fuzzy snake
approach to solve practical problems in offline signature verification. In [26],
researchers have proposed a Trans-OSV model for writer dependency. It is used
to extract features and learn in writer-independent-way and for classification,
they use Support Vector Machines using linear kernel and radial basis function
of the kernel. The model has achieved 3.24% EER. Researchers in [27] have
proposed the concept of Paired Contrastive Feature (PCF) for signature verifica-
tion. They also used Siamese Network on Sig-Net dataset to extract the features
and then applied PCF to represent similarity between the two paired signatures.
In [28], the researchers proposed a multi-loss-snapshot ensemble (MLSE) to
generalize the need for signature verification. The researchers in [29] analyzed
the results of dealing with various feature selection techniques. In [30], re-
searchers have proposed a signature verification system combining online and
offline features, comparing results using SVM.
 5

2.2 Proposed System

We have designed our system keeping in mind the below three points. Our
work has been done using a dataset of around 5435 images which has been
trained on a convolution neural network (CNN) and on a Siamese Neural Net-
work independently. The result of each model is combined and the final result
is generated based on the average prediction of the two models.
1. It should have very high accuracy
2. The signature verification model should be robust.
3. The model can predict accurately even with small datasets.
Moreover, we have used a pre-trained convolutional neural network, VGG16
which consists of 16 convolutional and fully connected layers [10]. The archi-
tecture is characterized by using a series of convolutional layers with small 3x3
filters, followed by max-pooling layers [10]. We adapted the VGG16 model by
incorporating a flatten layer, a dense layer and using ReLU and sigmoid as an
activation function.

3 PRIMARY MODEL DEPLOYMENT

In this section the generalized training of the model is discussed which will help for
identification of genuine and forged signatures. There can be various solutions to the
above problem, but we choose the ensemble learning approach where we will be
combining two diverse deep learning models, each of which are initially trained and
tested on a set of signatures. The predictions of the ensemble learning model would be
based on combined voting, that is if the average prediction of both the models is
above a certain cutoff level then it will report genuine signatures otherwise forged.
Fig 2. represents the architecture of a general Siamese network.

Fig. 3. General Siamese Network

6

3.1 Dataset

We collected about 5435 images each belonging to two categories – Genuine and
Forged signatures. The images are of medium to high resolution. The images have
been collected using Scraping, and from various websites, and then manually cropped
and cleaned so that no image contains any overlapped images, incomplete signatures
etc. The entire dataset is split into two primary segments, namely training set and
testing set. The train set contains around 4000 images both belonging to genuine and
forged classes. Similarly, the test set contains around 1435 images belonging to the
same classes.

3.2 Data Preprocessing

Our dataset contains about 5435 images belonging to both genuine and forged classes.
In order to enhance the quality of the data and facilitate the effective training of our
deep learning models, we preprocess the images by performing various operations
such as image resizing, normalization, noise reduction, data augmentation, conversion
to grayscale images as shown below in Fig 3. We have resized our dataset images to a
standardized size that is (224,224), this helps maintain uniformity across the dataset,
making it easier for the model to learn patterns. We have normalized the pixel value
to a common scale that is [0,1], this ensures that the model is not biased towards high
intensity or low intensity pixel values. We have applied a blurring effect to reduce the
noise present in the dataset that may hinder the performance of the model. We have
augmented, scaled the images by applying transformations such as cropping, rotation
as shown in Fig. 4, Fig. 5, which help increase the diversity of the training set, pre-
venting overfitting and improving model generalization. We also took care of the
computational complexity by converting the images to grayscale as shown in Fig 5.
Converting the color images to grayscale reduces the computational complexity and
also improves the model training speed.

Fig. 4. Preprocessing Stages

 7

3.3 Training

Our motive was to test and review how our model is able to predict whether the in-
putted signature is genuine or forged correctly or not. Our system consists of two
diverse deep learning models: Convolutional Neural Network (CNN) and Siamese
Network. Each model is tailored to extract specific distinct features from the signature
images. We loaded a pre-existing CNN model that is VGG16 as shown in Fig. 6
which is pre-trained on ImageNet. Then a new sequential model is created and the
VGG16 base model is added to the new model. Supplementary layers were added to
facilitate additional feature extraction and classification. These include a flatten layer,
three dense layers, with one employing the ReLU activation function and other utiliz-
ing sigmoid activation function for binary classification. Similarly, a Siamese Net-
work was constructed using a base model consisting of convolutional layers, max
pooling, flattening and dense layers to generate feature embeddings. The Siamese
architecture takes pairs of genuine and forged signature images as input to learn and
compute similarities between paired images. The models were evaluated using Adam
optimizer, binary cross-entropy loss function and accuracy metric. Training is execut-
ed using fit function with the training and validation data for a specified number of
epochs that is 50. After both the models are trained and tested individually, the two
models are combined together on the basis that if the average prediction is above a
certain range, the model generates true value; else it generates false.

Fig. 5. Original and Scaled Images

8

Fig. 6. Grayscale image and Augmented images

4 EXPERIMENTAL RESULTS

The accuracy achieved following the implementation of each method was assessed
using approximately 4000 signature images. The performance of the overall model
was judged based on the accuracy achieved after each model was tested on around
1435 images. Table 1 lists the accuracy of each model and the accuracy of the ensem-
ble model. Table1 summarizes the experimental results of each model which revealed
classification accuracies for each model and the ensemble model. The VGG16 model
provides an accuracy of approximately 98.18% whereas the Siamese Network gener-
ates an accuracy of approximately 99.43%. Such performance of individual models on
the same dataset would help increase the robustness and accuracy of the software.
Additionally, we evaluated the effectiveness of the proposed model by considering
the following metrics such as False Acceptance Rate (FAR), False Rejection Rate
(FRR) and Equal Error Rate (EER) as shown in Table 2. A low value of EER is a
good sign for a model because it suggests that the algorithm less frequently makes
mistakes. We also analyzed the train accuracy, validation accuracy, validation loss,
training loss with the help of graphs as shown in Fig. 7, Fig. 8 respectively.
 9

Table 1. Comparison of the accuracy of the VGG-16, Siamese Network and ensemble
model.

Model Input Training Test Accuracy Validation

Accuracy Accuracy

VGG16-Convolution Images 98.79 ± 0.21 99.60 ± 0.40 98.18±0.27

Neural Network

Siamese Network Images 99.70±0.3 99.68 ± 0.2 99.43±0.17

Ensemble Model Images 99.65±0.2 99.74 ± 0.2 99.27±0.20

Table 2. Comparison of FAR, FRR, ERR of different models.

Model FAR % FRR % ERR %

Convolution Neural Network 0.80 0.008 0.00001

Siamese Network 0.65 0.0041 0.00056

10

Fig. 7. Model Accuracy

Fig. 8. Model Loss

5 DISCUSSION

In this paper we have discussed and analyzed the performance of our ensemble learn-
ing model which is used to classify the signatures as genuine and forged. The ensem-
ble of convolutional neural networks (CNN) and Siamese Network in our signature
 11

verification system contributes greatly to its overall effectiveness. The CNN excels at
capturing minute details, shapes and patterns that are vital for distinguishing between
genuine and forged signatures. On the other hand, Siamese Network excels in learn-
ing pairwise similarities between two images. This combination made the model more
robust and high accuracy freak. As a future work, we would explore the integration of
additional functionalities such as behavioral dynamics to further enhance the robust-
ness. We would also implement a continuous learning mechanism to adapt the model
over time, accommodating changes in signature patterns and staying effective against
any forgery that may evolve over time.

6 CONCLUSION

We have demonstrated the fusion of Convolutional Neural Network (CNN) and Sia-
mese Network to improve the precision and resilience of signature verification sys-
tems. With an accuracy of approximately 98.18% for CNN model, 99.43% for Sia-
mese model and 99.27% for the ensemble model, the test results proved that our pro-
posed model performs effectively, which is considered as highly valuable [22]. As a
future work, the robustness can be made better and continuous learning mechanisms
can be invoked making the model adapt over time and new skilled forgeries.

REFERENCES

1. M. Muzaffar Hameed, Rodina Ahmad, Miss Laiha Mat Kiah, and Ghulam Murta-
za “Machine Learning-based Offline Signature Verification Systems: A System-
atic Review”.
2. Sunil S Harakannanavar, Jayalaxmi H, Asha C N, and Kabballi Prashanth Priya
Hudedavar “Biometric Trait: Offline Signature Identification and Verification
based on Multimodal Fusion Techniques”.
3. Manoj Kumar “Signature Verification Using Neural Network”, International
Journal on Computer Science and Engineering (IJCSE)
4. Rajib Ghosh “A Recurrent Neural Network-based deep learning model for offline
signature verification and recognition system”.
5. Jahandada, Suriani Mohd Sam, Kamilia Kamardin, Nilam Nur Amir Sjarif, and
Norliza Mohamed “Offline signature verification using deep learning convolu-
tional neural network (CNN) architectures Google-Net inception-v1 and incep-
tion-v3”.
6. Eman Alajrami, Belal A. M. Ashqar, Bassem S. AbuNasser, Ahmed J. Khalil,
Musleh M. Musleh, Alaa M. Barhoom, and Samy S. Abu-Naser “Handwritten
Signature Verification using Deep Learning”.
7. Javier O. Pinz´on-Arenas, Robinson Jim´enez-Moreno, and C´esar G. Pach´on-
Suesc´un “Offline signature verification using DAG-CNN”
8. Atefeh Foroozandeh, Ataollah Askari Hemmat, and Hossein Rabbani “Offline
handwritten signature verification and recognition based on deep transfer learning
using convolutional neural networks (A Literature Review)”.
9. Zihan Zeng, and Jing Tian “Deep Learning Methods for Signature Verification”.
12

10. Gabe Alvarez, Blue Sheffer, and Morgan Bryant “Offline Signature Verification
with Convolutional Neural Networks”.
11. Li Liu, Linlin Huang, Fei Yin, and Youbin Chen “Offline signature verification
using a region-based deep metric learning network”.
12. Ali Karounia, Bassam Dayab, and Samia Bahlakb “Offline signature recognition
using neural networks approach”.
13. Prof. Sathish, and Yasir Babiker Hamdan “Construction of static SVM based
recognition model for handwritten character recognition”
14. Dr S. Adebayo Daramola, and Prof. T. Samuel Ibiyemi “Offline Signature
Recognition using Hidden Markov Model (HMM)”.
15. Muhammad Sharif, Muhammad Attique Khan, Muhammad Faisal, Mussarat
Yasmin, and Steven Lawrence Fernandes “A framework for offline signature ver-
ification system: Best features selection approach”
16. S. Rokade, S. K. Singh, S. Bansod and P. Pal, "An Offline Signature Verification
Using Deep Convolutional Neural Networks,".
17. Mosaher, Q.S.- ul and Hasan, M. 2022. “Offline Handwritten Signature Recogni-
tion Using Deep Convolutional Neural Network.”
18. Harish Srinivasan, Sargur N. Srihari, and Matthew J. Beal “Machine Learning for
Signature Verification”
19. Yuchen Zheng, Brian Kenji Iwana, Muhammad Imran Malik, Sheraz Ahmed,
Wataru Ohyama, Seiichi Uchida: “Learning the micro deformations by max-
pooling for offline signature verification”
20. Indrajit Bhattacharya, Prabir Ghosh, Swarup Biswas: “Offline Signature Verifica-
tion Using Pixel Matching Technique
21. Dewi Suryani, Edy Irwansyah, Ricki Chindra: “Offline Signature Recognition
and Verification System using Efficient Fuzzy Kohonen Clustering Network
(EFKCN) Algorithm
22. Fadi Mohammad Alsuhimat and Fatma Susilawati Mohamad “A Hybrid Method
of Feature Extraction for Signatures Verification Using CNN and HOG a Multi
Classification Approach”
23. Tayeb Bahram. “A texture-based approach for offline writer identification”.
24. Yasmine Guerbai, Youcef Chibani, Bilal Hadjadji “The effective use of the one-
class SVM classifier for handwritten signature verification based on writer-
independent parameters.
25. José Vélez, Ángel Sánchez, Belén Moreno, José L. Esteban “Fuzzy shape-
memory snakes for the automatic off-line signature verification problem”
26. Huan Li, Ping Wei, Zeyu Ma, Changkai Li, Nanning Zheng “Trans-OSV: Offline
Signature Verification with Transformers”
27. Xiaotong ji, Daiki Suehiro, Seiichi Uchida “Paired contrastive feature for highly
reliable offline signature verification”
28. Saeed Masoudnia, Omid Mersa, Babak Nadjar Araab, Abdol-Hossein Vahabie,
Mohammad Amin Sadegh, Majid Nili Ahmadabadi “Multi-Representational
Learning for Offline Signature Verification using Multi Loss Snapshot Ensemble
of CNNs.
29. Ebrahim, AY, Kolivand, H, Rehman, A, Rahim, MSM and Saba “Features selec-
tion for offline handwritten signature verification: State of the art”
30. K S Radhika, Gopika S “Features selection for offline handwritten signature veri-
fication: State of the art.