First International Conference on Industrial and Information Systems, ICIIS 2006, 8 - 11 August 2006, Sri Lanka

Online Character Recognition System using

Elastic Matching
Vikas Kumar1, Rakesh Ahuja2
'Department of Computer Engineering & Information Technology, Moradabad Institute of Technology (Uttar Pradesh
Technical University), Moradabad, Uttar Pradesh, India
2Department of Computer Engineering & Information Technology, Moradabad Institute of Technology (Uttar Pradesh
Technical University), Moradabad, Uttar Pradesh, India
x F; -;_1I
2

Abstract - In this work we are presenting a simple online variety of different writing styles which are present in the
character recognition system using elastic matching. In our target user group. Even more difficult for online
system, we collected our test data from twenty different recognition in writer-independent environment, a writing
writes for uppercase, lowercase and digits. We used the which looks similar in a graphical (i.e. offline)
nearest neighbor classifier with elastic matching as between
patterns distance measure. We achieved very good representation, can have a different sequential (i.e. online)
recognition accuracy 95.99% for uppercase letters, 92.21% representation.
for lowercase letters and 99.11% for digits. We have also
shown that putting a rejection threshold may improve the
reliability of the recognition system. II. ONLINE HANDWRITING RECOGNITION
Handwriting recognition is the task of transforming a
I. INTRODUCTION
language represented in its spatial form of graphical marks
into its symbolic representation [4]. Online handwriting
The field of personal computing has begun to make recognition refers to the situation where recognition is
transition from desktop to handheld devices, thereby preformed using the dynamic stroke information along
requiring input paradigms that are more suited for single with spatial shape information.
hand entry than a keyboard. Also the small size of these
devices makes the inclusion of keyboard difficult. A. Online Handwriting Data
Handwritten and Speech inputs may be the attractive Online handwriting data is typically a dynamic, digitized
alternatives. For many applications [6] and situations representation of the pen movement, generally describing
handwritten input is preferable over the speech as it is sequential information about position velocity,
relatively insensitive to environmental noise. In many acceleration, pen angle etc. as a function of time. For our
situations like note-taking, annotations on a document, experiment we have considered only (x, y) coordinate
form filling etc. writing with pen is more natural than positions of sample points. We have represented writing as
keyboard entry. Data entry for many natural languages a sequence of strokes and a stroke consists of a set of
having large number of symbols (like Kanji, Chinese etc.)
has the great potential for handwriting recognition. sample points between a consecutive pen-down and pen-
Handwriting recognition can be broken down in two up. Duplicate samples are removed at the time of recording
categories: offline and online. Offline handwriting the data. This handwritten data is then segmented using
recognition focuses on the recognition of words that have user intervention such that each segmented sequence of
been written at some previous point of time on the paper. points represents a character pattern. This pattern is then
Information is presented to the system in the form of a assigned the class label of the class to which it belongs.
scanned image of the paper document. In contrast, online Patterns for training and testing are collected from 20
handwriting recognition focuses on the tasks when writers in the categories of uppercase letters, lowercase
recognition can be performed at the time of writing. The letters and digits. Test and training patterns are collected at
information presented to the recognition system is the different time so as to absorb variation of handwriting due
sequence of (x, y) coordinates of sample points, which to situational factors.
record the trace of pen's movement through time on the
surface of digitizing tablet. Handwriting recognition B. Data Preprocessing
system can further be broken down into the categories of Preprocessing [3] usually addresses the problems of data
writer-dependent and writer-independent. A writer- reduction, elimination of imperfections (smoothing), and
dependent system achieves higher recognition accuracy as size normalization. We have performed size normalization
it works on data with smaller variability. A writer- on the character patterns so as to bind the pattern in a box
independent system has to discriminate among a large of standard size. Aspect ratio was maintained during this

1-4244-0322- 7/06/$20. 00 c2006 IEEE

84
First International Conference on Industrial and Information Systems, ICIIS 2006, 8 - 11 August 2006, Sri Lanka

size normalization. When patterns are normalized to unknown against a known pattern stored in a database that
smaller sizes, we may get some duplicate points because of we call a model database with the aim of minimizing the
the compression of pattern. These duplicate points are distance between unknown and model. Figurel illustrate
removed from the pattern. the process of elastic matching as given by formula 3
below. The definition given in the formula ensures that all
the points of the unknown are compared against a point of
III. STRING MATCHING MEASURE the model. Scott [5] used a variant of this technique.
A string matching technique [2] is used to provide a
distance measure between character pattern pairs. A [D(i- -1,j)
written stroke is represented as a sequence of events,
corresponding to sequence of sample points in the stroke. rmin D(i- -1,1- -1) j>2
This sequence forms a variable length string. The distance |D(i- -1,j--2)J
between two strings, say A and B, involves computing the (3)
D(i,j)=d(i,j)+
distance between the corresponding pair of events eA and
D(i-l, j)
eBB This requires an alignment of the events between the -2
nD(i-l,j -1)J
.

two strings, and calculating the distance between strings

based on that alignment and the distances between the min{D(i-lj)}j=l
individual pairs of aligned events [5]. In our approach each where
sample point is an event and is represented with two 1 < i < NA,1 < j< NB
measurements: the x and y coordinates. Given an
alignment of events between two strokes, the distance Dist(A,B)= D (NA,NB )
dE (i, j) , between a pair of events eA from stroke A and
eB from stroke B is defined as the Euclidian distance. V. ITERATIVE ELASTIC MATCHING
Elastic matching as described above is computationally
intensive, especially if the recursion is carried out directly.
dE (i j) = J(Xi -xj)2 +0 _ yB)2 We used an alternate iterative formula that is not recursive
and nearly identical to the recursive description. Its main
The distance between two patterns is then the sum advantage is the reduction in computational complexity.
distances between each pair of corresponding points
the strings, given some alignment of points. An aliginment Idr,j
of the events between two strings takes the form of a Iset of d(i,j)

pairings of the events between the strings: mintd(i,j+1) j<m-1

ld(i, j+2)0 (4)

{e(tA (1) Ie
tB(1)
I
(tA (2)' etB (2) )
e
¢(tA (L)' etB (L) )
e
(2) DI(i, j) =DI(i -j)+
{d(ij)}
where
tA (1)< tA (2)< ...< t (L),L < NA holds. min{d(i,j)}I=
tB (1) < tB (2) < ...< tB (L),L < NB
where DI (1,1) = d (1,1) and jI is chosen from
Here NA and NB are the lengths of the string A
{ j, j + 1, j + 2 } depending upon which distance among
respectively, and the sequence of events from each
in the alignment are defined by the functions tA (i {d(i,j),d(i,j+1),d(i,j+2)} is minimum. This j' will
tB (j) be used as j in next iteration. To find the distance between
two patterns formula 4 above is evaluated iteratively for
DI (1,1) up toDI (NA, NB )
IV. ORIGINAL ELASTIC MATCHING
The application of string matching theory in cha Despite the similarities between the original and iterative
recognition is known as elastic matching. It match formulae, they are not identical. When we compare similar
patterns, both formulae will yield the same results. The two

1-4244-0322- 7/06/$20. 00 c2006 IEEE

85
First International Confereice on Industriatl and Informnation Systems, ICIIS 2006, 8 - 11 August 2006, Sri Lanka

formulae may yield different results, when the patterns are Table I - Results with increasing Look ahead Deaths
far apart. When this is the case, the resulting distance will Look lTest Category (0 Recognition)
aheacd UL ppercase Lowercase Diigits
be high with either formula and does not lead to a Deptlh
misclassification. Thus, in the cases where the two 68.75 68.60 64. 2
formulae are likely to differ, the results do not influence 2 86.56 85.04 88.65
recognition accuracy and two formulae can he considered 3
4
84.08
81.01
83.34
80.27
88.48
8 59
equivalent for our purpose. 5 76.06 7671 87.41

VI. EXPERIMENTS & RESULTS Table 2- Recognition Results without Stroke Information
Experiments were performed on the patterns (both Test Category Recognition Result
training and testing) collected from 20 writers. A total of Uppercase 92.330
Lowercase 89.2100
1696 uppercase, 1825 lowercase and 564 digit character
pattems were used as test patterns. We used 1-nearest- i Digits i98.5800
neighbor classifier i.e. the class of a test pattern is
determined to be the class of its nearest neighbor in the Table 3- Recognition Results with Stroke Intormation
model base where iterative elastic matching provides the Test Category Recognition Result
between -patterns distance measure We first performed the Uppercase 95 050
experiment to find the best look-ahead depth for the Lowercase 90.300o
1~~~~aloraS Digits Oh Ih
98.940o
patterns available in our model base Formulae 3 and 4 are
oiven at the look-ahead depth of two as our results have
shown that two is the best choice. Our recognition results In our system, we used nearest neighbor classifier along
for different look-ahead depth are shown in Table 1 and with elastic matching which tries to match the unknown
best results are shown as bold. with the model with minimum alignment distance. Thus,
we expect most of correct recognitions with lower
In all further experiments we used the look-ahead, depth distances, as the unknowns will get matched, with similar
of two We then performed our experiment by using models. We also performed an Occurrence-Area Analysis
category-wise model bases thereby decreasing the of correct and wrong recognitions in different distance
confusion that may be present in model base due to similar ranges. This analysis (see Figure 2) clearly shows that
shapes of many lowercase and, uppercase letters. As wrong recognitions are having increasing contribution in
expected, better results are obtained (see Table 2). higher distance ranges. It makes us sure that putting a
reject threshold will certainly improve the reliability of the
In previous two experiments, we used only x and y system An unknown pattern is rejected by the system
coordinates of sample points and their sequence in the when the lowest matching score is higher than the reject
pattern. In further experiment, we also used the stroke threshold as there are more chances for the recognition to
information. We found improved results (see Table 3) be wrong. Table 4 shows the improved recognition results
This improvement is impressive in uppercase letters in by setting a reject threshold By setting the lower reject
comparison to other categories. It is due to the fact that thresholds we may find a more reliable system but it can
majority of uppercase letters are made of more than one only be done at the cost of higher reject rates. Moreover,
stroke, thus, leading better improvement while using stroke increase in re ect rate is nearly exponential with respect to
informatton. almost linear increase in recognition accuracy.

Analysis of Occurrence-Area
100%
C
0 80%-
60%
Q-0
L.
C
4O0o No. Correct
0
20%-
0%
0-20 21-40 41-60 61-80 81-100 > 100
Mod Uoikcown
Distance Range

Fig. 1. Illustration at Elastic Matching Fig. 2. Analysis of the Correct and Wrong recognitions in
different distance ranges

1-4244-0322- 7/06/$20. 00 C2006 IEEE

86
First International Conference on Industrial and Information Systems, ICIIS 2006, 8 - 11 August 2006, Sri Lanka

Table 4 - Results of using the Reject Threshold For the matching of patterns we demonstrated strength of
Reject Recognition Accuracy Reject Rate iterative elastic matching. This technique reduces the time
Threshold (Complete Set) (%)
complexity of string matching to linear order while giving
None 93.46 No Reject good recognition accuracy. We have also shown that the
80 93.87 0.86 use of stroke infornation leads to better recognition
70 93.92 1.25 accuracy especially for uppercase letters. It is reconfirned
60 94.12 2.18 that slope-angle feature appropriately represents the curved
50 94.44 3.94
40 94.90 8.69 shapes well. Also, an appropriate reject threshold may
improve reliability of the recognition system. Our
Table 5- Recognition Results with slope-angle feature recognition results are better than the best seen results [1]
Test Category Recognition Result for the mixed style characters in the literature.
Uppercase 95.99%
Lowercase 92.210%
Digits 99.110%
Complete Set 94.74% REFERENCES
[1] C. Bahlmann, H. Burkhardt, "The Writer Independent Online
Handwriting Recognition System frog on hand and Cluster
In further experiment concentrated our attention towards Generative Statistical Dynamic Time Warping", IEEE
the lower case characters and observed that most of then Transactions on Pattern Analysis and Machine Intelligence, Vol.
are having curves as their shape constituents. Thus, we 26, No. 3, March 2004.
decided to include a slope-angle feature in our feature set
[2] J. B. Krushkal, "An overview of sequence comparison: Time
as this feature represents curved shapes well. In place of warps, string edits and macromolecules", SIAM review, vol. 25,
fornula 1 we used following fornula to compute the No. 2, pp. 201-237, 1983.
Euclidian distance of two aligned events in the patterns to
be matched. [3] M. Blumenstein, C. K. Cheng, X. Y. Liu, "New preprocessing
techniques for handwritten word recognition", Proc. of
International conference on Visualization, Imaging, and Image
Processing, pp. 480-484, 2002.
dE(i,j) = fXj xB)2 +0< -y)2 + UQI< _$)2 (5)
[4] R. Plamondon, S.N. Srihari, "On-line and Off-line Handwriting
Recognition: A Comprehensive Survey", IEEE Transactions on
where w is the coefficient which decides the weight of Pattern Analysis and Machine Intelligence, Vol. 22, No. 1, January
slope-angle difference and (D is the slope-angle of 2000.
respective pattern. We repeated our recognition experiment
and found further improved results (see Table 5) in all [5] Scott D. Connell, "Online Handwriting Recognition using Multiple
Pattern Class Models", Ph.D. Thesis, Michigan State University,
categories. 2000.

VII. CONCLUSION [6] V. Hiroshi Tanaka, V. Naomi Iwayama, V. Katsuhiko Akiyama,

"Online Handwriting Recognition Technology and Its
In this work, we have described our writer-independent Applications", FUJITSU Science Tech. Journal, vol. 40, No. 1,
character recognition system based on template matching. pp. 170-178, June 2004.

1-4244-0322- 7/06/$20. 00 c2006 IEEE

87