MTAP Bank Cheque Verification Using Image Processing
MTAP Bank Cheque Verification Using Image Processing
Prateek Agrawal
University of Klagenfurt, Austria
Lovely Professional University, India
E-mail: [email protected]
Deepak Chaudhary
Lovely Professional University, India
E-mail: [email protected]
Vishu Madaan
Lovely Professional University, India
E-mail: [email protected]
Anatoliy Zabrovskiy
University of Klagenfurt, Austria
Petrozavodsk State University, Russia
E-mail: [email protected]
Radu Prodan
University of Klagenfurt, Austria
E-mail: [email protected]
Dragi Kimovski
University of Klagenfurt, Austria
E-mail: [email protected]
Christian Timmerer
University of Klagenfurt, Austria
Bitmovin Inc., CA, USA
E-mail: [email protected]
2 Prateek Agrawal et al.
task of cheque clearance using image processing and deep learning methods.
These components include the bank branch code, cheque number, legal as well
as courtesy amount, account number, and signature patterns. our innovation
aims at benefiting the banking system by re-innovating the other competent
cheque-based monetary transaction system which requires automated system
intervention. For this research, we used institute of development and research
in banking technology (IDRBT) cheque dataset and deep learning based con-
volutional neural networks (CNN) which gave us an accuracy of 99.14% for
handwritten numeric character recognition. It resulted in improved accuracy
and precise assessment of the handwritten components of bank cheque. For
machine printed script, we used MATLAB in-built OCR method and the ac-
curacy achieved is satisfactory (97.7%) also for verification of Signature we
have used Scale Invariant Feature Transform (SIFT) for extraction of features
and Support Vector Machine (SVM) as classifier, the accuracy achieved for
signature verification is 98.10%.
1 Introduction
Although cheque systems in banks for transactions is one of the most pow-
erful tool, primarily due to its slow processing time, considerable complexities
in cheque clearing process and gradually with an immense growth in the tech-
nology, the electronic-transactions have created a big shadow on the cheque
system to cater the need of the banking sector. Therefore, we have proposed a
mechanism to make the entire process of bank cheque verification not only fast
but also reliable by using deep learning methods and image processing tech-
niques. Due to various technical constraints, it was not feasible for the banking
sector to implement the automation of the bank cheque clearing which required
sophisticated measures for authentication and verification for the transfer of
4 Prateek Agrawal et al.
these features in the highlighted using bounding boxes. There are various tech-
niques already available for the identification, authentication and verification
of the bank cheques as shown in the research works presented by Dhanva et
al (2018) [7], Singh and Tigga (2012) [8], Gonge and Ghatol (2014) [9], Gonge
and Ghatol (2016) [10], Menon et al (2017) [11]. Unfortunately, the efficiency
of the works are not up to mark. For the task of verification, we considered
matching of Indian Financial System Code (IFSC), cheque number, account
number, legal amount, courtesy amount and signature of cheque issuer as input
components.
2 Related Work
In order to perform the task of getting a logical response for the input of an
image, it requires a meaningful interpretation of the characters present in the
Title Suppressed Due to Excessive Length 7
image and, thus, we need to perform the character recognition for the various
patterns present in an image. Singh and Sachan (2018) [12], Wankhede and
Mohod (2017) [13], and Ramanathan et al. (2009) [14] presented their work to
retrieve the characters from the image Optical Character Recognition (OCR).
The segmentation of digits for which there are several techniques devised is
the major challenge faced in the recognition of characters. Chen (2017) [15]
proposed a general approach for handwritten digit segmentation using spectral
clustering and achieved an accuracy of over 97%.
One of the earliest researches on font recognition was done by Dickinson
(1960) [16], Liu and Shelton (1966) [17], Nagy and Shelton (1966) [18], Wang
et al. (1972) [19], Zramdini and Ingold (1998) [20] in which they suggested
a statistical approach for the Latin Language font recognition at the text
line level depending on the global typographical features. They were able to
achieve approximately 97% of recognition rate. For English language also, due
to its widespread usage and commercial usability, Zhu and Wang (2002) [21],
Dhande and Kharat (2017a) [22], Pal et al. (2006) [23], Dhande and Kharat
(2017b) [24], and Saenthon and Sukkhadamrongrak (2014) [25] proposed few
approaches for character recognition. Font recognition was done by using a
wide array of techniques and they were able to achieve a satisfactory accuracy
on text blocks.
Similarly, Ramanathan and Sharma (2017) [26] devised a method using
Support Vector Machine (SVM) for classification and feature extraction by
proposing a SVM-based multi-knowledge-based system, achieving an accuracy
rate of 93.54%, which marked the continued focus on the OCR or segmentation
for the character recognition in image processing methods. However, Stewart
et al. (2018) [27] proposed a segmentation free recognition techniques in which
they presented that for small training datasets, it is possible to manipulate
either manually or automatically that segmentation and stitching improves
handwriting recognition on dataset with few samples. In another attempt by
Peres et al. (2018) [28] in 2018 proposed a novel handwritten methodology in
which they used a hybrid CNN and SVM classifier with which they were able
to achieve an overall accuracy of 96.7% using the MNIST dataset [29].
For intelligent character recognition to create and match dynamic tem-
plates as well as the conversion of characters into text, Kajale et al. (2017) [30]
implemented the supervised machine learning methodology and achieved the
accuracy close to 95% for the alpha numerals and special characters. The pres-
ence of the noise in the form of lines, watermarks, diagrams, patterns or all of
these at the background is one of the important factors to create the perfor-
mance variation in different character recognition techniques. Goto and Aso
(2002) [31], Brisinello and Gibric (2018) [32], Brisinello et al. (2017) [33], and
Feng and Gao (2010) [34] performed research involving extraction of the infor-
mation from the colored or noisy background with satisfactory level of output.
Contrary to the notion that output will drop significantly if there is a change
in the background with varying levels of color change in the texts or intro-
duction of noise in the form of patterns and figures. Jia and Wei (2010) [35]
extracted the legal amount from the Chinese bank cheques and proposed a
8 Prateek Agrawal et al.
multiple channel binarization approach for extracting legal amount from color
Chinese bank check images with a combination of global and local threshold.
Sneha Singh et al. (2015) [36] proposed a method to recognise the legal amount
of Indian bank cheques using correlation coefficient method. They achieved an
average 76.4% word recognition accuracy on a self created dataset of 19215
word samples of 61 different words. Another researchers M Badrul Alam et
al. (2015) [37] developed a tool to recognise the handwritten legal amount
of Bangladesh bank cheques using ANN and achieved only 93.4% accuracy.
Similarly, Jayadevan et al. (2010) [38] proposed a lexicon-driven segmentation
recognition technique for the recognition of the legal amount words from the
Indian bank checks written in English using Modified Quadratic Discriminant
Function (MQDF) and claimed 97.20% accuracy.
Raghevendra and Danti (2015) [39] proposed a method to identify the bank
details associated with the cheques using bank after removing the background
noise. Researchers used the invariant geometrical features for accomplishing
the recognition of the checks with 93.8% of accuracy only. To improve the
security for verifying the cheque, another important feature of it is Magnetic
Ink Character Recognition (MICR) code. Even though if we are able to per-
form the pattern recognition as well as the legal amount recognition using the
above stated techniques one more component of a bank check that needs to be
properly identified for the sake or security as well as determining the validity
of the check is the Magnetic Ink Character Recognition (MICR), Shirai et al.
(2012) [40] proposed a three pronged solution to the problem starting with
the character color estimation and conversion into binary form to remove the
background, to separate each character using the character pitch estimation
and the removal of the unwanted noise & extraction of the required MICR
code using the sign fragments removal by the stroke tracing.
Lu et al. (2017) [41] proposed techniques to deal with noisy images as the
image that we have, may be a copy of the original image with some consider-
able noise such as shadow along with a spatial noise due to improper copying
of this image. Researchers used local adaptive threshold algorithm in order to
transform the gray-scale image into the binary image to capture the contours
of the texts in the image and applied a double-filtering algorithm to remove
the noise between the texts and the characters. Wiraatmaja et al. (2017) [42]
proposed a deep convolutional de-noising auto-encoder and claimed the im-
provement in accuracy by 26.78% as compared to a simple OCR in average
accuracy testing. Void Pantograph is another important feature which ensures
whether an original image of the cheque was taken for verification or not. It
refers to the process of creating a pantograph which is otherwise invisible to
naked eyes on the original print. However, when copied with a printer, it is
clearly visible making it a very important security features for security print-
ing toolkit. Aronoff et al. (2011) [43] formulated a method to automate the
optimization of void pantograph settings. Peymani and Soryani (2017) [44]
proposed a model to identify machine generated as well as hand written char-
acters for Farsi language images with an accuracy of 68% using an Arabic
language dataset [45]. Bhadwal (2019) [46], Singh (2018) [47] and Agrawal
Title Suppressed Due to Excessive Length 9
(2012) [48] also developed different applications ranging from object recogni-
tion to edge detection by using image processing and computer vision methods.
Ignat and Aciobniei (2016) [49] proposed method of feature extraction for of-
fline handwritten digit recognition by using rotation and filtering of edges for
the extraction of the digit characteristics. KNN and SVM classifiers were used
for classifying the features and claimed to achieve the accuracy about 99% on
the MNIST dataset.
Hou et al. (2017) [50] implemented a model using convolutional neural Net-
work for learning the handwritten numeric characters and perform scratchy
or non-scratchy decisions with error rate of 0.40% on MNIST database. Li et
al. (2018) [51] used grab cut segmentation based on image region by consid-
ering image texture and boundary information which helps them to perform
segmentation with minimal user interaction. Mityakov et al. (2017) [52] in ap-
plication of modern architectures of deep neural networks for solving practical
problems explained the advent of sophisticated networks like GoogleNet [53]
capable of automatic feature extraction and classification. Evidently watershed
segmentation is one of the most widely used algorithm in terms of achieving
feasible results however in cases of adhesion among components of the image,
it becomes difficult to obtain remarkable results. Ji et al. (2015) [54] proposed
an improved watershed algorithm based on distance transform for cell image
segmentation.
ance process without any human intervention. While in our research, we pro-
posed the solution starting from pre-processing of scanned cheque image to
verification of cheque amount and signature. In further section, we described
the entire methodology to implement the model and presented the results
which are found satisfactory.
3 Methodology
The bank cheque verification tool required fulfillment of some key stages in
sequential manner. At first, the tool ensured the genuineness of IFSC present
on cheque then it verified the cheque number to ensure whether the cheque
was from the set of cheque leaflets assigned to the account holder or not.
After that, the amount was verified by checking whether the customer was
having sufficient amount in his account for which he released the cheque, and
finally the signature of the cheque issuer(s) were verified. Only after the com-
pletion of these necessary verification steps, the cheque clearance was passed
for transfer or withdrawal of funds.
- A unique number assigned to the user of the bank services, used for
Account number
monetary transaction related services.
ple, to extract information from the machine typographic script, we use OCR
method as it tends to produce results with higher level of accuracy whereas
for hand-written words and numerical values, we can use deep learning based
CNN. Similarly, for authenticating the signature(s), we extracted the features
using SIFT algorithm and for classification, SVM produced optimum results.
To implement these methods, we required noise free image therefore the better
idea was to segment the image so as to acquire only the relevant information.
Fig. 4 shows the broad steps using a flow-chart, involved in our research
process of cheque verification. These steps encompassed certain sub-steps each
of which were essential to meet the standards so as to extract the required
information from the bank cheque. The entire methodology for bank cheque
verification is summarized in Fig. 5.
The image acquisition of a bank cheque is crucial for the CTS. Generally,
flatbed scanners are used to acquire such images. Due to orientation and ir-
regularities presented in the scanned image(s), we were unable to use the
acquired image(s) directly for the image processing operations therefore it
required some pre-processing steps.
We used the cheque scanned images for our research. A sample scanned image
is shown in Fig. 6. As a scanned image obtained from the scanner can not be
used directly thus we needed to pre-process it, which involved two primary
operations, i.e, rotation and removal of unnecessary background information.
In this step, we rotated the scanned image with respect to the ‘Date Box’
(which is a common feature presented at the same part of every bank cheque)
and then removed the background noise and extra information. Efficiency of
the parameter identification considerably improves with removal of extra back-
ground information.
3.2.1 Rotation
As the scanned images may vary in terms of orientation therefore we used the
date box present in all standard cheque leaflets and the relative invariant na-
ture of the position was utilized. In order to perform the rotation of image, we
determined the point of rotation and degree of rotation. The primary compo-
nent essential for rotation to work was contour extraction, as we were able to
determine the position values of date box by using it which used as our anchor
for any set of operation related to length mapping. The process of contour
extraction was explained in the forthcoming segmentation sub-section. Also,
in order to perform the rotation, we used the midpoint of the image as the
rotation point, and we used date box in order to determine the angle necessary
for the rotation. Fig. 7 shows the same process.
Title Suppressed Due to Excessive Length 15
There was lot of extra information in the scanned image which was to be
removed. For the task of removing of background information, we used the
date box present in the standard cheque template. The cheque dimensions of
any bank in a country are standard and are followed by the norms decided
by the government financial body. In India, RBI monitors and controls the
16 Prateek Agrawal et al.
functioning of all Indian banks. A sample cheque leaf with it’s standard di-
mensions regulated and decided by RBI, is shown in Fig. 8 [80]. All Indian
banks are bound to follow these standards and geometric dimensions strictly.
These geometric dimensions may vary for banks of different countries and can
easily be calculated as per the respective country agency standards. Using the
date box coordinates range and with the implementation of length mapping
between the date box and the whole cheque image, we determined the range of
the cheque as per the schematic representation of edge removal as mentioned
in Fig. 9. In reference to data box, we extracted the regions of interest for
courtesy amount, legal amount, signatures etc.
(yA −yD )
AD
(AD+a) = y2 −yD
AD = A −yD )
(y
(AD+c) yA −y1
AB (xB −xA ) (1)
(AB+d) = xB −x1
AB = (xB −xA )
(AB+b) x2 −xA
The parameters involved in the equations are explained in the Fig. 9. Here,
dimension AB and AD are representing the length, breadth of the date box.
Whereas, a, b, c, d are the distance from the edges of the cheque leaflet. Also,
x and y are the horizontal and vertical coordinates of the date box. The key
parameters of these equations are the boundary of cheque leaflet which is the
coordinate range for unnecessary background information removal.
18 Prateek Agrawal et al.
Here RGB represents the Red, Green and Blue channels of the image and
Gray Image represents the pixel value of the grayscale image.
After converting the image into grayscale format, next pre-processing step was
to remove the noise. For this, we applied Gaussian de-noising method which is
used in a linear form of smoothing filter for a signal. This method of filtering
is a form of convolution between the normal arrangement of values and the
image. The formula for 2-D Gaussian function is explained in Eq. (4).
2
−(x +y 2)
1
G(x, y) = ( 2
) × e 2σ2 (4)
2πσ
Here G(x, y) is the normal arrangement of the values. Whereas, σ repre-
sents the standard deviation of the arrangement.
We used the binary image format of the gray-scale image for precised and
accurate contour extraction, i.e., we converted the image into binary format
and then determined the boundary points for the region using the formula as
described in the Eq. (5).
f (i, j − 1) = 0
f (i, j) = 1
(5)
f (i, j) ≥ 1
f (i, j + 1) = 0
In this equation, f (i, j) function for binary image represents the pixel value
of the required point, and the functions f (i, j − 1) and f (i, j + 1) are repre-
senting the pixel values of the neighboring points for f (i, j).
After the removal of noise from the image, last step of pre-processing was to
perform morphological operations. There are much chances to have certain ad-
ditional values present in the image even after applying filters which hamper
the process of identification by using CNN. Therefore, we performed erosion
and dilation for morphological operations which were able to determine the
local minimum and local maximum. We applied dilation first followed by ero-
sion. This closing operation was capable of closing small cracks in the image
while not changing the shape of the it.
Title Suppressed Due to Excessive Length 19
3.3 Segmentation
To use only the desired part of the image for various operations, the good idea
is to do image segmentation. For this, we separated each pattern locally in the
optimum way. Thus, we performed segmentation to ensure that the tool would
use only the required information of image for its processing, while accessing
the entire process. We separated each key parameter of the cheque leaflet us-
ing segmentation for correct identification and verification. Other than contour
extraction in order to determine the position values for the date box as ex-
plained in previous section(s), we identified region of interest (ROI) as per the
CTS-2010 cheque leaf standard dimensions as shown in 2. In order to remove
the extra information from the desired region, we created a standard template
using pixel values as per RBI guidelines on CTS-2010 for bank cheques as
shown in Fig. 10. After the segmentation of bank cheque image, we separated
each segment of key parameters for OCR to determine the patterns for verifi-
cation. We also performed transfer learning approach in order to recognize the
handwritten numeric digits present in the date and courtesy amount of cheque.
For Indian banks, image segmentation work is considered automated as the
cheque dimensions of different banks are standardised as per CTS-2010 stan-
dards while for different country cheques, it should be done either manually
due to variation in cheque dimensions or separate dimensions will be defines
as per the country or bank guidelines. Therefore we consider our approach for
bank cheque leaf image segmentation as semi-automatic.
ture verification, we used SIFT for feature extraction and SVM for classifica-
tion.
3.4 CNN model for Legal amount and Courtesy amount identification
After extracting the image segments, we used CNN for handwritten numeric
digits recognition of courtesy amount as well as to convert the legal amount
into string. We used Deep Learning Toolbox (a MATLAB toolbox) for CNN
implementation on two convolution layers with six and twelve filters as shown
in Fig. 11. Whereas, in max pooling process, we used a window of 2x2 with one
thousand epochs and one stride. After that, we received numeric output from
the courtesy amount of bank cheques and converted it into string to compare
with the legal amount string. We followed IPV system so as to be written
in the words IPV format. For handwritten courtesy amount identification, we
used transfer learning for digit identification. We used MNIST dataset [82] of
handwritten digits which has sixty thousands examples in training set and ten
thousands examples in test set. All the images in the dataset are normalized
and centered in a fixed size. We used a dataset of two thousands samples (two
hundred for each digit number ranging from 0 to 9) from training dataset and
five hundred (fifty for each digit) samples from testing dataset. We distributed
the entire dataset in 80% for the training and 20% for the testing using CNN.
We trained with the training loss occurring with one thousand epochs, 0.01
base learning rate and seven iterations per epoch. Also for handwritten charac-
ters used in legal amount recognition, we used EMNIST dataset [83] of English
letters identification. EMNIST dataset contains 88800 training examples and
14800 testing examples. We used a dataset of 4160 training samples (160 for
each letter with mixed uppercase and lowercase alphabets in twenty six classes)
and 1040 (forty for each letter) samples for testing dataset in our work. For
CNN training, we distributed this training and testing dataset in a ratio of
80:20. Rest all the CNN training parameters are kept same as it was used for
MNIST dataset training. Table 3 shows the training parameters for MNIST
and EMNIST sample dataset training.
After identifying the digits from courtesy amount image segment and charac-
ters from legal amount image segment using CNN, we arranged the acquired
numeric digits into the Indian Place Value (IPV) system for matching and
comparing it with the legal amount identified text. For this, we developed
an algorithm ”transfer learning method” to identify different nuances of hand-
written strings of numerals in accordance to the IPV chart. Then we converted
courtesy amount into strings for comparing it with the legal amount. It is very
essential to have a clear match between both the strings as only after that the
process of funds transfer is supposed to be initiated. For this we convert the
Title Suppressed Due to Excessive Length 21
Parameters Value
Number of convolution layers 2
Number of Max pooling layer 1
Number of filter at convolution layer 6 and 12
Kernel window size at convolution layer 3X3
Kernel window size at max pool layer 2X2
Number of stride at convolution layer 1
Number of stride at max pool layer 1
Number of neurons in output layer 2
Learning rate 0.01
Number of epochs passed 1000
Number of iteration (per epoch) 7
Number of input attributes for digits data (MNIST dataset) 784
Number of output attributes for MNIST dataset 1
Training data for MNIST dataset 2000
Testing data for MNIST dataset 500
Input attributes for letters data (EMNIST dataset) 784
Output attributes for EMNIST dataset 1
Training data for EMNIST dataset 4160
Testing data for EMNIST dataset 1040
Training / testing data 80%/20%
Activation function (before and after max pooling layer) ReLU
Batch size 64
Fig. 11 Proposed CNN architecture for handwritten digit recognition of courtesy amount.
numeric values into strings and then they are compared with the legal amount
strings. As the way of referring the amount IPV system be different due to
22 Prateek Agrawal et al.
local influence thus a dictionary function was used which kept all the possible
ways, a word can be written.
Algorithm 1: Algorithm for verifying legal amount with courtesy
amount.
Input: Input Courtesy amount (number), Legal amount (legal amount)
Output: (1) Courtesy amount to words (courtesy in words)
Output: (2) If Courtesy amount in words equal to Legal Amount
// Indexing of arrays ones[] and tens[] starts from one
1 ones[ ] ← {”One”, ”T wo”, ”T hree”, ...”N ineteen”}
2 tens[ ] ← {”T wenty”, ”T hirty”, ”F orty”, ...”N inety”}
3 S ← null
4 output ← null
5
6 Function convert(number):
7 if number > 19 then
8 if number%10 = 0 then
9 S = tens[(number/10) − 1]
10 else
11 S = tens[(number/10) − 1] + ” ” + ones[number%10]
12 else
13 S = ones[number]
14 end
15 return S
16 End Function
17
18 Function num2words(number):
19 if len(number) > 7 then
20 output ← output + crore
21 if len(number) > 5 then
22 output ← output + lakh
23 if len(number) > 3 then
24 output ← output + thousand
25 if len(number) > 2 then
26 output ← output + hundred
27 if number > 100 and (number%100) then
28 output ← output + ”and”
29 output ← output + convert(number%102 )
30 if len(number) = 2 and (number > 19) then
31 output ← convert(number)
32 if (len(number) = 2 and number ≤ 19) or (len(number) = 1) then
33 output ← ones[number − 1]
34 return output
35 End Function
36
37 Function compare(string 1, string 2):
38 if string 1 = string 2 then
39 Result ← ”M atching”
40 else
41 Result ← ”N ot M atching”
42 end
43 return Result
44 End Function
45
46 crore ← output + convert(number/107 ) + ”Crore”
47 lakh ← output + convert((number/105 )%100) + ”Lakh”
48 thousand ← output + convert((number/103 )%100) + ”T housand”
49 hundred ← output + convert((number/102 )%10) + ”Hundred”
50
// Courtesy amount to words
51 courtesy in words ← num2words(number)
52
// Comparing Courtesy amount in words with Legal Amount
53 result ← compare(courtesy in words, legal amount)
Title Suppressed Due to Excessive Length 23
Next, we moved to signature verification step. In the dataset 114 images used
for our experiments, four images do not contain signatures. Therefore, we used
110 handwritten offline signatures from scanned cheque images for signature
verification task. These 110 signatures were created by 50 different users. Out
of these signature samples, 80 were genuine while rest 30 were forged sig-
natures. Each user’s signatures were found in one to three different cheque
images. Fig. 12 shows the overall signature verification sub-process of bank
cheque. For feature extraction, we normalized each signature image segment
in order to extract the features like resizing, thinning, rotating and cropping.
The normalization experimental results are represented in the Fig. 13. From
the figure, it is clearly shown that we identified the region of interest (of ac-
tual signature) by normalizing the signature image. We applied SIFT method
on these normalized images for feature extraction and contour identification.
SIFT works on a principle to reflect the structure of the image regions us-
ing key points and preventing the scale and orientation information of these
structures. We applied SIFT method into four steps: i) Scale-space extrema
counstruction (SSE), ii) Key point localization (KPL), iii) Orientation assign-
ment (OA), iv) Key point descriptor extraction (KDE).
SSE breaks down each image segment into Gaussian pyramid at different
levels. It further breaks each level into small sub-levels with different variations
using Laplacian of Gaussian (LoG) filters to get the prominent keypoints. Later
in next part, secured locations and key points are detected and calculated. OA
is responsible for calculating the scales and orientations of finalized keypoints.
KDE is responsible to generate a SIFT descriptor against each selected key
point.
After extracting SIFT descriptors and scales & orientations for each signa-
ture image, we applied SVM classifier on these two features dataset to match
and classify the signature patterns. SVM is a machine learning method under
supervised learning. It is used for both classification as well as regression. To
verify the genuineness of the signatures, we applied SVM classifiers on SIFT
extracted features. We distributed genuine and forged signatures data in uni-
form manner between training and testing dataset in ration of 80:20. For this,
88 signature data (with 66 genuine and 22 forged signature data) were taken
in training dataset while 22 signature data (14 genuine and 8 forged signature
24 Prateek Agrawal et al.
Fig. 12 Account holder Signature classification process for bank cheque verification.
Fig. 13 Normalization of Signature: (a) Original segment of Signature. (b) resized image.
(c) morphological Operation: Thinning. (d) rotation of Image. (e) cropping.
Title Suppressed Due to Excessive Length 25
We experimented the problem stated in the research with the procedure elu-
cidated above in terms of performance and accuracy using MATLAB software
(deep learning toolbox and image processing toolbox). We used 114 bank
cheque images in our research work. Out of which, 112 images were from
IDRBT bank cheque dataset and 02 cheque images were scanned by us [85]
[86]. We performed training and testing on the developed system with these
cheque leaflets. We used different key parameter segments obtained for bank
cheque leaflet for training and testing.
For handwritten digit recognition using CNN, we achieved the accuracy
with a mean value of 99.14% as shown in Table 4. Similarly, the learning
progress of training networks for character recognition on different epochs
and iterations are shown Table 5. From the table, we can see that the mini
batch loss is reduced up to 0.0077 with 99.94% accuracy at 850 iterations. The
experimental results are represented in Fig. 17.
We achieved the collective loss less than 1% for the trained network. Af-
ter extracting the numbers from the courtesy amount images, we converted
these numbers into words using IPV system. Once we acquired the result in
words for the courtesy amount, we compared it with the legal amount using
the algorithm mentioned above and in case of matching the strings of cour-
tesy amount and legal amount, we proceed with the verification of signature.
We first normalized the segmented image, extracted the features using SIFT
method and then we applied SVM classifier on extracted features to match
and classify the patterns. Using SVM, we were able to achieve 98.1% accuracy.
Fig. 14 Segmentation of a bank cheque image: (a) Original scanned cheque, (b) cropped
cheque leaflet, (c) gray-scale cheque image, (d) IFS code, (e) account number, (f) cheque
number, (g) courtesy amount, and (h) signature.
ferred in [87] for recognizing the signature patterns. Another sample cheque
with leaflet segmentation is presented in the Fig. 15. Considering a sample
cheque leaflet for experimenting the verification procedure, we got the exper-
imental outcome as shown in the Fig. 16. We executed the segmented images
from figure 15(a), 15(b), and 15(c) by OCR for text identification while for
figure 15(d) and 15(e), we implemented CNN respectively.
There are advances in the field of forensics still handwritten signatures are
considered as one of the most reliable feature for certifying authenticity due to
its unique style of writing. We can perform online as well as offline verification
of signatures and both have their benefits and shortcomings. Some of the highly
efficient methods for signature verification are summarized in the Table 1.
Module wise system performance comparison in terms of accuracy with the
state of the art methods, is summarised in Table 6.
Title Suppressed Due to Excessive Length 27
Fig. 15 Segmented regions of SBI bank cheque: (a) Original scanned cheque, (b) cropped
cheque leaflet, (c) gray-scale cheque image, (d) IFS code, (e) account number, (f) cheque
number, (g) legal amount, (h) courtesy amount, and (i) signature.
28 Prateek Agrawal et al.
Fig. 16 Image segments on left hand size and their OCR results (shown as value of s1) for
machine printed and handwritten characters on right hand side for (a) IFS code, (b) cheque
number, (c) account number, (d) legal amount, and (e) courtesy amount.
Title Suppressed Due to Excessive Length 29
We developed the model to verify the bank cheques using OCR, CNN, SIFT
and SVM. We used OCR method to identify the machine typographic char-
acters with desirable accuracy and efficiency, whereas, we performed CNN to
give precise output for the handwritten digits written on the cheque leaflet.
We proposed and implemented the algorithm to convert numbers into words
to verify the cheque which is one of the major reason of its bounce-off and of
halt the monetary transaction. In order to achieve this, we applied OCR tech-
nique to recognise the machine printing digits and achieved 97.7% accurate
matching. Then, we used separate database to train the network and after
30 Prateek Agrawal et al.
Table 6 Module wise performance analysis of proposed system with other existing works.
Bank cheque Machine typo- Handwritten
Signature
Work details leaf image graphic text & text & digit
verification
segmentation digit recognition recognition
CNN & SVM method
Peres et al. [28] Manual NA for handwritten text NA
(96.5% accuracy)
Template matching &
correlation coefficient
Sneha et al. [36] manual NA NA
for handwritten text
(76.4% accuracy)
Modified quadratic
discriminat function
Jayadevan et al. [38] Manual NA NA
for handwritten text
(97.04% accuracy)
Pixel & geometrical
based CNN method
Ahlawat et al. [88] manual NA NA
for handwritten digit
(99.89% accuracy)
Kullback - Leiber
Mondal & Kundu [69] Manual NA NA Divergence method
(97% accuracy)
Model Agnostic
Hafemann & et al. [79] Manual NA NA Meta Learning method
(94.84% accuracy)
ANN method
Karanjkar et al. [77] Manual NA NA
(85% accuracy)
Gabor Filter &
Ramanathan et al. [14] Manual SVM classifier NA NA
(97% accuracy)
CNN method
Semi-automatic (standard FIST feature extraction
OCR method & transfer learning
Proposed system cheque template & SVM classifier
(97.7% accuracy) algorithm (99.14%
dimensions) (98.1% accuracy)
& 99.94% accuracy)
achieving a desired level of accuracy after training, we used different data sets
to test the trained model for matching the numbers into the words. Evidently,
we achieved an accuracy of 99.14% for the digit recognition which is an im-
provement over the previous accuracy of 99.05% for the CNN used for digit
recognition. Similarly, for character recognition using CNN, we were able to
achieve the accuracy up to 99.94%. For signature recognition, we used SIFT
and SVM classifier to determine the relevancy for verification of signature and
achieved 98.1% accuracy. our innovation will help the banking organizations
to automate the process of cheques clearance in faster and easier way in com-
parison to conventional and manual approach by saving time and manpower.
In our research work, we performed the verification of bank cheques using the
CNN, in an effort to automate the process of verification of the bank cheques
based on the identification of the handwritten numeric digits. However, in or-
der to successfully perform the clearance of bank cheque, we need to perform
the validation of bank cheques as well as ability to function online using a
database resolving the required queries in real time.
Title Suppressed Due to Excessive Length 31
Acknowledgements
This work has been partly supported by the European Union Horizon 2020 Re-
search and Innovation Programme under the ARTICONF Project with grant
agreement number 644179.
References
13. P A Wankhede and W S Mohod. A different image content based retrieval using ocr
techniques. International conference of Electronics, Communication and Aerospace
Technology (ICECA), pages 155–161, 2017.
14. R Ramanathan, S Ponmathavan, L N V Thaneshwaran, A S Nair, and K P Soman.
Optical character recognition for english and tamil using support vector. International
Conference on Advances in Computing, Control, and Telecommunication Technologies,
pages 610–612, 2009.
15. C Chen. A general approach for handwritten digits segmentation using spectral clus-
tering. 14th IAPR International Conference on Document Analysis and Recognition,
pages 547–552, 2017.
16. W E Dickinson. A character-recognition study. IBM Journal of Research and Develop-
ment, pages 335–348, 1960.
17. N Liu, C and L Shelton, G. An experimental investigation of a mixed-font. IEEE
Transactions on Electronic Computers, EC-15(6):916–925, 1966.
18. G Nagy and G Shelton. Self-corrective character recognition system. IEEE Transactions
on Information Theory, 12(2):215222, 1966.
19. P Wang and C R Shiau. Machine recognition of printed chinese characters via trans-
formation algorithms. Pattern Recognition, 5(4):303–321, 1972.
20. A Zramdini and R Ingold. Optical font recognition using typographical features. IEEE
Transactions on Pattern Analysis and Machine Intelligence, 20(8):877–882, 1998.
21. Zhu Yong, Tan Tieniu, and Wang Yunhong. Font recognition based on global tex-
ture analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence,
23(20):1192–1200, 2002.
22. P Dhande and R Kharat. Recognition of cursive english handwritten characters. In-
ternational Conference on Trends in Electronics and Informatics ICEI, pages 199–203,
2017.
23. U Pal, F Kimura, K Roy, and T Pal. Recognition of english multi-oriented characters.
International Conference on Pattern Recognition (ICPR’06), pages 1–6, 2006.
24. P Dhande and R Kharat. Character recognition for cursive english handwriting to recog-
nize medicine name from doctors prescription. International Conference on Computing,
Communication, Control and Automation (ICCUBEA), pages 1–5, 2017.
25. A Saenthon and N Sukkhadamrongrak. Comparison the training methods of neural
network for english and thai character recognition. Signal and Information Processing
Association Annual Summit and Conference (APSIPA), pages 1–4, 2014.
26. TT Ramanathan and D Sharma. Multiple classification using svm based multi knowl-
edge based system. International Conference on Advances in Computing & Communi-
cations, (ICACC’17), pages 307–311, 2017.
27. S Stewart, L Pinto, and Barrett B. Segmentation and stitching improves handwriting
recognition on datasets with few samples. 16th International Conference on Frontiers
in Handwriting Recognition (ICFHR), pages 465–470, 2018.
28. A A Peres, S M Vieira, and J R C Pinto. Hybrid neural models for automatic handwrit-
ten digits recognition. International Joint Conference on Neural Networks (IJCNN),
pages 1–8, 2018.
29. Y. LeCun, L. Bottou, Bengio. Y., and P. Haffner. Gradient-based learning applied to
document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998.
30. R Kajale, S Das, and P Medhekar. Supervised machine learning in intelligent char-
acter recognition of handwritten and printed nameplate. International Conference on
Advances in Computing, Communication and Control (ICAC3), pages 1–5, 2017.
31. H Goto and H Aso. Character pattern extraction from colorful documents with com-
plex backgrounds. Object recognition supported by user interaction for service robots
(ORSUISR), pages 180–183, 2002.
32. M Brisinello and R Gibric. Optical character recognition on images with colorful back-
ground. IEEE 8th International Conference on Consumer Electronics - Berlin (ICCE-
Berlin), pages 1–6, 2018.
33. M Brisinello, R Grbic, M Pul, and T Andelic. Improving optical character recognition
performance for low quality images. 59th International Symposium ELMAR-2017, pages
167–171, 2017.
Title Suppressed Due to Excessive Length 33
34. Wanli Feng and Shangbing Gao. A vehicle license plate recognition algorithm in night
based on hsv. 3rd International Conference on Advanced Computer Theory and Engi-
neering(ICACTE), 4:53–56, 2010.
35. H Jia, Shui and G Wei, Dong. Extraction of legal amount from color bank check images.
International Conference on Web Intelligence and Intelligent Agent Technology, pages
253–256, 2010.
36. Sneha Singh, Tharun Kariveda, Jija Das Gupta, and Kallol Bhattacharya. Handwritten
words recognition for legal amounts of bank cheques in english script. Eighth IEEE
International Conference on Advances in Pattern Recognition (ICAPR), pages 1–5,
2015.
37. Mohammad B. A. Miah, Mohammad A. Yousuf, Mohammad S. Mia, and Mohammad P.
Miya. Handwritten courtesy amount and signature recognition on bank cheque using
neural network. International Journal of Computer Applications, 118(5):21–26, May
2015.
38. R Jayadevan, U Pal, and K F. Recognition of words from legal amounts of indian bank
cheques. 12th International Conference on Frontiers in Handwriting Recognition, pp.
166-171. IEEE, pages 166–171, 2010.
39. S P Raghevendra and Danti A. A novel recognition of indian bank cheques based on
invariant geometrical features. International Conference on Trends in Automation,
Communications and Computing Technology (I-TACT’15), pages 1–5, 2009.
40. K Shirai, M Akita, M Okamoto, K Tanikawa, T Akiyama, and T Sakaguchi. Removal
of background patterns and signatures for magnetic ink character recognition of checks.
10th IAPR International Workshop on Document Analysis Systems, pages 190–194,
2012.
41. H Lu, B Guo, J Liu, and X Yan. A shadow removal method for tesseract text recognition.
International Congress on Image and Signal Processing, BioMedical Engineering and
Informatics (CISP-BMEI), pages 1–5, 2017.
42. C Wiraatmaja, K Gunadi, and IN Sandjaja. The application of deep convolutional
denoising autoencoder for optical character recognition preprocessing. International
Conference on Soft Computing, Intelligent System and Information Technology (IC-
SIIT), pages 72–77, 2017.
43. Jason Aronoff, Steven Simske, and Margaret Sturgill. Automated optimization of void
pantograph settings. HP Laboratories, pages 690–693, 2011.
44. K Peymani and M Soryani. From machine generated to handwritten character recog-
nition; a deep learning approach. 3rd International Conference on Pattern Recognition
and Image Analysis (IPRIA), pages 243–247, 2017.
45. Farsi-datatset. https://github.com/Kianenigma/DeepConvFarsiOCR, 2017. [Online; ac-
cessed October 4, 2019].
46. Neha Bhadwal, Prateek Agrawal, Vishu Madaan, Awadhesh Shukla, and Anuj Kakran.
Smart border surveillance system using wireless sensor network and computer vision.
International Conference on Automation, Computational and Technology Management
(ICACTM19), pages 183–190, 2019.
47. Sandeep Singh Chauhan, Prateek Agrawal, and Vishu Madaan. E-gardener: Building
a plant care taker robot using computer vision. 4th International Conference on Com-
puting Sciences (Feynman100-ICCS’18), pages 137–142, 2018.
48. Prateek Agrawal, Simranjeet Kaur, Harjeet Kaur, and Amita Dhiman. Analysis and
synthesis of an ant colony optimization technique for image edge detection. International
Conference on Computing Science (ICCS), pages 127–131, 2012.
49. A Ignat and B Aciobniei. Handwritten digit recognition using rotations. 18th Inter-
national Symposium on Symbolic and Numeric Algorithms for Scientific Computing,
pages 303–306, 2016.
50. J Hou, H Zeng, L Cai, J Zhu, J Cao, and J Hou. Handwritten numeral recognition
using multi-task learning. International Symposium on Intelligent Signal Processing
and Communication Systems (ISPACS), pages 155–158, 2017.
51. Y Li, J Zhang, P Gao, L Jiang, and M Chen. Grab cut image segmentation based on
image region. 3rd IEEE International Conference on Image, Vision and Computing,
pages 311–315, 2018.
34 Prateek Agrawal et al.
72. T Handhyani, A R Yohannis, and L Hiryanto. Hand signature and handwriting recogni-
tion as identification of the writer using gray level co-occurrence matrix and bootstrap.
Intelligent Systems Conference (IntelliSys), pages 1103–1110, 2017.
73. D Morocho, A Morales, J Firrrez, and J O Garcia. Humans in the loop: Study of semi-
automatic signature recognition based on attributes. International Carnahan Confer-
ence on Security Technology (ICCST), pages 1–5, 2017.
74. A Beresneva, A Epishkina, and D Shingalova. Handwritten signature attributes for its
verification. IEEE Conference of Russian Young Researchers in Electrical and Elec-
tronic Engineering (EIConRus), pages 1477–1480, 2018.
75. D Morocho, A Morales, J Fierrez, and R V Rodriguez. Human- assisted recognition
based on comparative attributes. 14th IAPR International Conference on Document
Analysis and Recognition, pages 5–9, 2017.
76. F Boudamous, H Nemmour, Y Serdouk, and Y Chibani. Offline handwritten signature
identification using histogram of templates and artificial immune recognition system.
IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineer-
ing (EIConRus), pages 1–5, 2017.
77. Shubhangi L. Karanjkar and P. N. Vasambekar. Signature recognition on bank cheques
using ann. IEEE International WIE Conference on Electrical and Computer Engineer-
ing (WIECON-ECE), pages 44–47, 2016.
78. Luiz G. Hafemann, Robert Sabourin, and Luiz S. Oliveira. Characterizing and evaluat-
ing adversarial examples for offline handwritten signature verification. IEEE Transac-
tions on Information Forensics and Security, 14(8):2153 – 2166, January 2019.
79. L. G. Hafemann, R. Sabourin, and L. S. Oliveira. Meta-learning for fast classifier adap-
tation to new users of signature verification systems. IEEE Transactions on Information
Forensics and Security, 15:1735–1745, 2020.
80. CTS-2010 cheque leaf dimensons and specifictions. https://rbidocs.rbi.org.in/
rdocs/content/PDFs/SCFR220210.pdf, 2010. [Online; accessed July 17, 2020].
81. Kanan Christopher and W. Cottrell Garrison. Color-to-grayscale: Does the method
matter in image recognition? PLoS ONE, 7(1):1–7, 2012.
82. Yann LeCun, Corinna Cortes, and Christopher J.C. Burges. The mnist database of
handwritten digits. http://yann.lecun.com/exdb/mnist/, 1998.
83. Gregory Cohen, Saeed Afshar, Jonathan Tapson, and Andr van Schaik. Emnist: an
extension of mnist to handwritten letters, 2017.
84. V. Madaan and A. Goyal. Predicting ayurveda-based constituent balancing in human
body using machine learning methods. IEEE Access, 8:65060–65070, 2020.
85. P. Dansena, S. Bag, and R. Pal. Differentiating pen inks in hand-written bank cheques
using multi-layer perceptron. 7th International Conference on Pattern recognition and
Machine Intelligence, pages 655–663, 2017.
86. Cheque Market 2018. http://www.idrbt.ac.in/icid.html, 2017. [Online; accessed
October 4, 2019].
87. Zhang Hongyang, Han Jianfeng, Jia Hui, and Zhang Yan. Extraction and matching
of binocular image based on sift algorithm. International Conference on Intelligent
Transportation, Big Data & Smart City (ICITBS), pages 665–668, 2018.
88. S Ahlawat, A Choudhary, A Nayyar, S Singh, and B Yoon. Improved handwritten digit
recognition using convolutional neural networks (cnn). Sensors, 20:3344, 2020.