Handwriting to Text Conversion: A Robust

Approach Using KNIME and AWS

Rekognition
Shreeya Kosireddy
Department of Electronics and Communication Engineering,
National Institute of Technology, Warangal,
Warangal, Telangana, India
Email: [email protected]

Ravi Kishore Kodali

Department of Electronics and Communication Engineering,
National Institute of Technology, Warangal,
Warangal, Telangana, India
Email: [email protected]

Abstract—The digitization of handwritten docu- I. I NTRODUCTION

ments plays a crucial role in modern document man-
Handwritten documents are integral to many
agement, from educational assessments to archival
preservation. This paper presents a solution for fields, including education, administration, and his-
handwriting-to-text conversion, aimed at automating torical record-keeping. Despite the growing adop-
time-consuming processes such as examination paper tion of digital tools, many processes continue to
grading and document analysis. The system, imple- rely on handwritten records, such as examination
mented on the KNIME analytics platform, integrates
grading, administrative file management, and pre-
with AWS Rekognition, a cloud-based Optical Char-
acter Recognition (OCR) service, to extract text from serving historical documents. These processes are
images of handwritten documents stored in an S3 often manual and require significant time and ef-
bucket. A custom Python script, utilizing the boto3 fort, leading to errors and inefficiencies, particularly
library, automates interaction with AWS Rekognition, when dealing with large volumes of data.
ensuring efficient text extraction by filtering out low- Existing OCR technologies primarily excel at rec-
confidence outputs. KNIME handles preprocessing
tasks such as path manipulation and data integration, ognizing printed text, but they face significant chal-
while join nodes combine the extracted text with rele- lenges when applied to handwritten content. Factors
vant metadata, producing a comprehensive, structured like varying handwriting styles, image quality, and
output. inconsistent spacing between characters often result
The proposed system reduces manual effort, min- in incomplete or inaccurate text extraction. While
imizes errors, and saves time, demonstrating strong
accuracy in processing diverse handwriting styles. recent advancements in OCR technology have im-
Key applications include digitizing archival records, proved accuracy, there is still a gap in handling com-
automating educational assessments, and improving plex handwritten documents efficiently and reliably.
administrative document handling. Future enhance- This paper proposes a scalable system for
ments will focus on improving OCR accuracy for com- handwriting-to-text conversion that integrates AWS
plex handwriting, expanding multilingual support, and
integrating advanced analytics. By leveraging cloud- Rekognition, a cloud-based OCR tool, with the KN-
based technologies, this scalable solution highlights the IME analytics platform. AWS Rekognition extracts
potential of modern tools in transforming document text from handwritten document images stored in an
analysis workflows. S3 bucket, while KNIME processes and organizes
the data into a structured format. A Python script is strengths in feature extraction and classification.
used to filter out low-confidence OCR results, ensur- These models, while accurate, demand extensive
ing high-quality outputs. This combination of tools training and fine-tuning for handwriting-specific
addresses the challenges of handwriting digitization tasks [?], [?]. Additionally, a literature review on
and provides a reliable, automated solution. handwriting recognition systems highlighted the ef-
The proposed system is designed to reduce man- fectiveness of combining CNNs with OCR frame-
ual effort and improve efficiency in document pro- works for classification and feature extraction [?].
cessing. Its applications span across various do-
C. Cloud-Based and Hybrid Systems
mains, such as digitizing archives, automating grad-
ing systems, and streamlining administrative work- Cloud-based OCR solutions, such as AWS
flows. The modular nature of the workflow allows Rekognition and EasyOCR, are increasingly popular
for easy adaptation and future enhancements, such due to their scalability and ease of integration. A no-
as support for multiple languages and advanced data table example demonstrated converting handwritten
analytics. trip sheets to CSV using EasyOCR combined with
By leveraging the strengths of cloud-based OCR regular expressions [?]. These tools offer practical
and data analytics platforms, this solution demon- alternatives to custom-built systems by leveraging
strates the potential to transform handwritten doc- pre-trained models for rapid deployment and mini-
ument management. It not only addresses current mal maintenance.
limitations but also opens new possibilities for au- D. Applications of Handwriting Recognition
tomating and improving document workflows in
Applications of handwriting recognition include
diverse fields.
digitizing archival records, real-time grading sys-
II. R ELATED W ORK tems, and improving accessibility for visually im-
paired users. For instance, a system combining
The digitization of handwritten text has been
text-to-speech (TTS) capabilities with handwriting
extensively studied, with diverse methodologies re-
recognition was developed to provide audio outputs
flecting the evolution of technology in handwriting
for handwritten content [?]. Additionally, handwrit-
recognition systems. The following research efforts
ing detection frameworks have been employed for
provide critical insights into the current state of the
diverse languages using hybrid CNN-RNN models,
field, laying the foundation for the design and im-
showcasing versatility [?], [?].
plementation of the system described in this paper.
E. Key Differentiators of the Proposed System
A. Deep Learning and Handwriting Recognition
While many studies focus on resource-intensive
Several studies have demonstrated the effective- deep learning models, the proposed system adopts a
ness of combining Convolutional Neural Networks practical approach using AWS Rekognition for OCR
(CNNs), Long Short-Term Memory (LSTM) net- and KNIME for data preprocessing. This hybrid
works, and Connectionist Temporal Classification solution minimizes the need for extensive training
(CTC) layers in handwriting recognition tasks. Sys- and model development, ensuring adaptability to di-
tems trained on various databases, such as IAM verse handwriting styles [?]. By integrating Python-
Handwriting, have shown high accuracy due to ad- based automation, the system addresses challenges
vanced preprocessing techniques like segmentation. such as low-confidence outputs and ensures efficient
For example, a CNN-based deep learning model us- processing.
ing LSTM and CTC loss was proposed for handwrit-
ten text recognition, achieving reliable performance F. Insights and Opportunities for Enhancement
across diverse datasets [?], [?]. The review of related work highlights areas for
future development, including multilingual support
B. Architectural Comparisons in Neural Networks and real-time analytics. Hybrid approaches combin-
Comparative studies on architectures such as ing cloud-based tools with localized models may
AlexNet, ResNet-50, and VGG-16 emphasize their balance accuracy and scalability [?], [?].
 R EFERENCES This threshold strikes a balance between retaining
useful data and avoiding low-quality results. AWS
III. M ETHODOLOGY Rekognition’s parallel processing capabilities make
A. Introduction to Workflow it efficient for large datasets, reducing processing
time. The extracted text is then formatted into a
This project outlines a process to transform hand-
structured output for the next stage of the workflow.
written text into structured digital text by combining
AWS Rekognition, a cloud-based Optical Character
Recognition (OCR) tool, with KNIME, a platform D. Text Vectorization
for data preprocessing, analysis, and visualization. After text extraction, the next step is text vec-
The modular workflow allows customization at each torization, where the text is transformed into a
step to meet specific requirements. numerical format suitable for analysis or machine
As shown in Figure 1, the process consists of learning models. Techniques like one-hot encoding
several stages: dataset loading, OCR processing, fil- or word embeddings are used, depending on the
tering outputs by confidence levels, text conversion application.
into numerical form, and result generation. Each
Vectorization prepares the data for advanced tasks
stage ensures the system’s accuracy and reliability.
such as clustering, summarization, or sentiment
The modularity of the workflow allows for easy
analysis. By converting text into numerical vectors,
substitution of components. For instance, AWS
the system leverages mathematical models to gain
Rekognition can be replaced with a more advanced
insights and make predictions.
OCR tool, or additional preprocessing steps can be
incorporated based on the dataset. This flexibility
makes the system adaptable and scalable for various E. High Confidence Table Visualization
applications. To ensure OCR result quality, KNIME’s Table
View node displays detected text alongside its confi-
B. Accessing the Dataset and Preprocessing
dence score, categorizing it into ”High Confidence”
The input consists of images of handwritten text (50
stored in an Amazon S3 bucket. This step involves The table also serves as a diagnostic tool dur-
accessing the dataset, performing basic preprocess- ing testing and development, providing immediate
ing, and removing unsuitable files such as corrupted feedback on how the system handles different input
or empty ones. types and facilitating iterative improvements.
Metadata is crucial for tracking and indexing the
dataset. Each image is assigned a unique identifier
F. Final Results
along with metadata, including upload date, source,
and document type. This structure improves data The final output pairs the original handwritten im-
management and traceability, making downstream ages with the extracted text, showcasing the system’s
processing more efficient. KNIME nodes are used ability to accurately digitize handwritten documents.
to automate this preprocessing, ensuring that only The modular design allows the system to handle
clean, structured data is passed to the OCR stage, a range of use cases, including digitizing archives,
reducing the potential for errors. processing exam papers, or analyzing handwritten
records.
C. Optical Character Recognition (OCR) The results can be further enhanced by incorpo-
The core of the workflow is OCR, where hand- rating Natural Language Processing (NLP) tools for
written images are analyzed, and text is extracted tasks like summarization or grammatical correction.
using AWS Rekognition. Each detected word or Machine learning can also be added to provide
line is assigned a confidence score. A Python script additional insights, such as sentiment analysis or
in KNIME filters out text with a confidence score topic modeling, directly from the handwritten data.
below 50
 Fig. 1. Complete Workflow in KNIME

IV. C ONCLUSION results are considered for analysis.

Future work could expand the system by inte-
This project presents an efficient and scalable so- grating advanced NLP techniques for tasks like au-
lution for converting handwritten text into structured tomatic summarization, grammatical correction, or
digital format using AWS Rekognition and KNIME. sentiment analysis. Additionally, machine learning
The system’s modular workflow, which includes models could be incorporated to further improve
preprocessing, OCR extraction, text vectorization, OCR accuracy, particularly for challenging hand-
and result generation, ensures accurate and efficient writing styles. Overall, this project lays a robust
data processing. By leveraging AWS Rekognition’s foundation for building intelligent systems capable
powerful OCR capabilities and KNIME’s robust of processing handwritten text for a wide range of
data processing environment, the system can han- practical applications. ./cite(
dle large volumes of handwritten data, making it
suitable for various applications, such as digitizing V. ACKNOWLEDGMENT
historical documents, automating exam paper cor- I would like to thank the Department of Electron-
rections, and analyzing handwritten records. ics and Communication Engineering, NIT Warangal,
The system’s flexibility allows easy adjustments for providing resources and support for this project.
to components like OCR tools and preprocessing ./cite(
methods, making it adaptable to diverse use cases.
The final results demonstrate the system’s ability to R EFERENCES
convert handwritten text into digital format, ready [1] N. Teslya and S. Mohammed, ”Deep Learning for Hand-
for further analysis or integration into other appli- writing Text Recognition: Existing Approaches and Chal-
lenges,” 2022 31st Conference of Open Innovations Associ-
cations. Confidence-based filtering further enhances ation (FRUCT), Helsinki, Finland, 2022, pp. 339-346, doi:
OCR accuracy by ensuring only high-confidence 10.23919/FRUCT54823.2022.9770912.
 Fig. 2. Basic workflow of an OCR engine

[2] B. Vidhale, G. Khekare, C. Dhule, P. Chandankhede, A. Editable Text And Audio,” 2024 International Conference
Titarmare, and M. Tayade, ”Multilingual Text Handwritten on Advances in Modern Age Technologies for Health and
Digit Recognition and Conversion of Regional languages Engineering Science (AMATHE), Shivamogga, India, 2024,
into Universal Language Using Neural Networks,” 2021 pp. 1-7, doi: 10.1109/AMATHE61652.2024.10582076.
6th International Conference for Convergence in Tech- [7] J. Memon, M. Sami, R. A. Khan, and M. Uddin, ”Hand-
nology (I2CT), Maharashtra, India, 2021, pp. 1-5, doi: written Optical Character Recognition (OCR): A Com-
10.1109/I2CT51068.2021.9418106. prehensive Systematic Literature Review (SLR),” in IEEE
[3] B. Debnath et al., ”Automatic Handwritten words on Touch- Access, vol. 8, pp. 142642-142668, 2020, doi: 10.1109/AC-
screen to Text file converter,” TENCON 2018 - 2018 IEEE CESS.2020.3012542.
Region 10 Conference, Jeju, Korea (South), 2018, pp. 0219- [8] K. Vijay, A. Mukundh, S. Pooja, T. N. Aravind,
0223, doi: 10.1109/TENCON.2018.8650269. and R. Manoj Adhithya, ”Hand Written Text Recogni-
[4] P. Mishra, P. Pai, M. Patel, and R. Sonkusare, ”Extraction tion and Global Translation,” 2023 International Confer-
of Information from Handwriting using Optical Character ence on Computer Communication and Informatics (IC-
recognition and Neural Networks,” 2020 4th International CCI), Coimbatore, India, 2023, pp. 1-5, doi: 10.1109/IC-
Conference on Electronics, Communication and Aerospace CCI56745.2023.10128327.
Technology (ICECA), Coimbatore, India, 2020, pp. 1328- [9] J. Jebadurai, I. J. Jebadurai, G. J. L. Paulraj, and S. V.
1333, doi: 10.1109/ICECA49313.2020.9297418. Vangeepuram, ”Handwritten Text Recognition and Conver-
[5] S. N, K. S, V. C, D. N, and S. G, ”Handwritten trip sion Using Convolutional Neural Network (CNN) Based
sheet to digital conversion using OCR,” 2023 International Deep Learning Model,” 2021 Third International Confer-
Conference on Advances in Electronics, Communication, ence on Inventive Research in Computing Applications
Computing and Intelligent Information Systems (ICAE- (ICIRCA), Coimbatore, India, 2021, pp. 1037-1042, doi:
CIS), Bangalore, India, 2023, pp. 7-10, doi: 10.1109/ICAE- 10.1109/ICIRCA51532.2021.9544513.
CIS58353.2023.10170030. [10] Z. Zhou, F. Shao, and H. Xu, ”A Survey on Hand-
[6] V. R. Vaddadi, C. Bharathi, A. K. Rout, and A. K. Tiruna- written Text Recognition Techniques Using Deep Learn-
gari, ”A Handwriting Recognition System That Outputs ing,” 2018 International Symposium on Signal Processing
 Fig. 3. High confidence table

Systems (SSPS), Beijing, China, 2018, pp. 154-159, doi:

10.1109/SSPS.2018.8476140.
[11] K. Patil, A. Shinde, and S. Khan, ”Handwritten Script
Recognition Using Deep Neural Networks,” 2019 In-
ternational Conference on Recent Trends in Electron-
ics, Information Communication Technology (RTEICT),
Bangalore, India, 2019, pp. 165-170, doi: 10.1109/RTE-
ICT.2019.8749985.
Fig. 4. Output handwritten images and detected text