CHAPTER 2

RELATED WORK

2.1 LITERATURE REVIEW

There have been numerous studies over the years that contribute to the development and
understanding of Automated Exam Grading Systems (AEGS). These studies have explored
various methods, technologies, and challenges involved in automating the grading of handwritten
exam responses. The following reviews some of the significant studies and their findings:

[1] Automated Evaluation of Handwritten Answer Script Using Deep Learning Approach
(Kumar et al., 2021):
This study presents an innovative approach to handwritten text recognition and grading
using a combination of Convolutional Neural Networks (CNNs) and Bidirectional Long
Short-Term Memory (BiLSTM) networks. The model aims to address the challenges in
automatically grading handwritten exam responses. The model performs the following key
tasks:

 Feature Extraction using CNN: The CNN is used to extract features from handwritten
text by identifying character patterns. This approach has been proven effective in
handling the complex variations seen in handwritten characters.

 Context-Aware Sequence Processing using BiLSTM: The BiLSTM network processes

the sequence of text to capture the contextual relationships between words and characters.
This is crucial for understanding the structure of sentences and paragraphs in handwritten
responses.

 Optical Character Recognition (OCR): The system uses OCR for digitizing the
handwritten text and converting it into a machine-readable format.

 Automated Grading with NLP: The model uses Natural Language Processing (NLP)
techniques to evaluate and score the responses based on predefined grading criteria.
Despite the promising results, the study highlights several challenges in handwriting
recognition, particularly with variations in handwriting styles, overlapping characters, and
noise in scanned images. The authors emphasize the need for large-scale annotated datasets
to enhance model performance and propose the exploration of transformer-based models
for better contextual understanding and error handling.

[2] Automated Grading Recommendation System (Srivastava et al., 2022):

This study focuses on evaluating and comparing various essay grading systems that are
already in use. The authors conducted a comprehensive review of different grading models,
comparing them based on their techniques, performance, application, and accuracy. Key
findings include:

 Latent Semantic Analysis (LSA)-based Grading: LSA-based models were found to

provide a 0.85% aggregation accuracy, primarily by focusing on content evaluation. This
approach allows the system to understand the latent meaning behind the words, not just
keyword matching.

 C-Rater: Another NLP-based system, C-Rater, achieved an accuracy of 0.8% by

evaluating both style and content. The system incorporates several linguistic features,
such as grammar, syntax, and coherence, which improve the overall grading process.

 Semantics and Feedback: The study emphasizes the importance of semantic similarity in
enhancing grading accuracy and effectively detecting the correctness of responses.
Furthermore, it discusses how AI-driven feedback mechanisms can be integrated into
automated grading systems to provide constructive and personalized feedback to
students.

The study points out that while these models have shown significant improvement in grading
accuracy, there are still limitations with keyword-based models that fail to capture the
nuances in a student's writing style. Future work should focus on incorporating deeper
semantic attributes and AI-powered feedback for more accurate and insightful evaluations.

[3] AI-Based Automated Grading for Open-Book Exams (Janda et al., 2023):
This research proposes an AI-driven grading system specifically designed to assess open-
 book exams. It focuses on evaluating handwritten essays by considering factors like
semantic similarity and coherence of the response. The model incorporates a graph-based
relationship between the content of the essay to understand the logical flow and contextual
meaning. Key elements include:

 Graph-Based Relationship Development: The study uses a graph structure to represent

the relationship between different ideas in the essay, enabling a deeper understanding of
the response.

 Rule-Based Grammar and Syntax Checks: In addition to AI, the system also uses rule-
based checks to ensure the grammatical accuracy and logical consistency of the essay.

 Performance Evaluation: The system achieved an average quadratic weighted kappa of

0.793, which is slightly lower than SBLSTMA's kappa score of 0.801.

This study highlights the complexity of evaluating open-ended responses and underscores
the need for more robust NLP techniques to maintain fairness and accuracy in grading. The
authors point out the scalability challenges when implementing such systems on a large
scale and suggest the adoption of hybrid approaches that combine AI with human review
mechanisms to improve reliability and accuracy.

Works Cited

[1] Kumar, S., et al., "Automated Evaluation of Handwritten Answer Script Using Deep
Learning Approach," Journal of AI Research, 2021.
[2] Srivastava, K., Dhanda, N., and Shrivastava, A., "Automated Grading Recommendation
System," International Conference on NLP, 2022.
[3] Janda, H. K., Pawar, A., Du, S., and Mago, V., "AI-Based Automated Grading for
Open-Book Exams," IEEE Transactions on Learning Technologies, 2023.

2.2 EXISTING TECHNOLOGY

Several technologies are currently used in the field of Automated Exam Grading Systems
(AEGS). These technologies include traditional OCR-based systems, machine learning
 models, and hybrid AI solutions. Below is an overview of existing technologies commonly
employed in automated grading systems:

1. Optical Character Recognition (OCR)

OCR technology has been widely used to digitize handwritten text in automated grading
systems. OCR extracts the text from scanned exam papers and converts it into machine-
readable content. Popular OCR systems like Tesseract and Google Cloud Vision are
frequently employed for text recognition in automated grading systems. However, OCR
alone does not address the complexity of grading subjective, open-ended responses, which
require deeper understanding and contextual analysis.

2. Natural Language Processing (NLP)

NLP plays a critical role in grading open-ended responses. It is used for understanding the
meaning, structure, and context of student answers. Current NLP models, such as Latent
Semantic Analysis (LSA), BERT, and GPT-3, are used to evaluate responses based on
content, grammar, coherence, and style. These models analyze the contextual meaning of
words and phrases rather than just matching keywords. NLP techniques are essential in
addressing challenges like semantic gaps and contextual understanding, which are difficult
to achieve with traditional keyword-based models.

3. Machine Learning Models

Machine learning (ML) techniques, including supervised learning, decision trees, random
forests, and support vector machines, are commonly applied in grading systems. These
models are trained on datasets consisting of correctly graded responses, allowing the
system to make predictions about the quality and correctness of new responses. ML models
are particularly useful for automated feedback generation and error detection in student
answers. Deep learning techniques, including CNNs and BiLSTMs, are used for complex
tasks like handwritten text recognition and grading of free-text responses.

4. Hybrid Approaches
 Many existing systems combine traditional OCR and machine learning models with rule-
based techniques to create hybrid systems. These approaches can combine the strengths of
both models—machine learning for pattern recognition and rule-based methods for
grammar checking and ensuring consistency. Hybrid systems are increasingly popular in
automated essay grading systems because they improve grading accuracy and consistency.

5. Existing Platforms and Systems

 Grammarly: An AI-powered tool that helps assess written content, focusing on grammar,
style, and plagiarism. It offers an automated system for grading essays and other forms of
writing based on predefined rubrics.

 Turnitin: While primarily known for plagiarism detection, Turnitin also provides tools for
grading and providing feedback on student essays by utilizing machine learning and
NLP.

 Automated Essay Scoring (AES) Systems: Various AES systems, including e-rater and
IntelliMetric, focus on scoring essays based on content, grammar, and style. These
systems use algorithms to compare student responses against predefined standards or
rubrics.

6. Challenges in Existing Technologies

While current systems are advanced, they still face several challenges:

 Handwriting Variability: OCR-based systems often struggle with diverse handwriting

styles, leading to inaccuracies in text extraction and grading.

 Contextual Grading: Grading open-ended responses using AI still requires better

understanding of context and nuance, which current systems struggle with.

 Bias and Fairness: Automated grading systems must be designed to minimize bias.
Current models may inadvertently reinforce biases based on the data they are trained on,
particularly in relation to grammar and content evaluation.
  Scalability: Implementing these systems on a large scale, particularly for exams with
thousands of responses, poses challenges in terms of resource management and
processing power.

7. The Gap and Our Approach

Existing technologies primarily focus on automating one part of the grading process, such
as handwriting recognition or semantic analysis. However, they often lack an integrated
solution that combines these elements into a single, seamless workflow. The proposed
Automated Grading System for Handwritten Exam Papers will bridge this gap by
integrating OCR, NLP, and AI-driven grading models into a cohesive system capable of
handling diverse handwriting styles, providing accurate assessments, and minimizing bias.

2.3 INFERENCE OF LITERATURE REVIEW

Aspect Inference

Handwriting Recognition CNN and BiLSTM-based models have shown significant

improvements in text recognition. However, handling diverse
handwriting styles and overlapping characters remains a
challenge. Further work is required to address these variations.

Semantic Analysis for NLP-based models like BERT and LSA offer substantial
Grading improvements by understanding contextual meaning. These
systems outperform keyword-based models, but limitations in
training data and semantic gaps need to be addressed for better
 accuracy.

Bias Mitigation and Integrating fairness algorithms into AI models helps reduce
Fairness biases in grading. However, these algorithms are still in their
infancy and need refinement to ensure unbiased evaluations.

Balancing Techniques for Techniques like SMOTE, ADASYN, and Gradient Boosted
Data Distribution Trees improve the performance of grading models by
addressing issues with imbalanced datasets, thus enhancing
prediction accuracy.

Automated Feedback AI-driven feedback helps students understand their mistakes

Mechanisms and improves their learning outcomes. However, these systems
need to be enhanced to provide personalized, context-aware
feedback to ensure deeper learning.

2.4 EXTRACTION FROM LITERATURE REVIEW

Key observations extracted from the literature review include:

1. Handwriting recognition using deep learning techniques has evolved significantly but still
requires further research to address complex handwriting styles and other variations.
2. NLP-based grading models using semantic analysis show superior performance
compared to traditional keyword-based methods. However, they still face challenges due
to data scarcity and semantic gaps.

3. Hybrid AI models, which combine rule-based evaluations with machine learning, offer
promising improvements in grading accuracy and consistency.

4. Fairness and bias mitigation are critical in automated grading systems, and more refined
techniques need to be developed to ensure objective evaluations.

5. Adaptive learning feedback systems improve student engagement by providing detailed

and context-aware feedback on their responses.

6. Scalability challenges in AI-based grading systems call for efficient resource

management and hybrid AI-human review approaches to handle large volumes of data.

Our proposed Automated Grading System will integrate OCR for text extraction, NLP for
semantic understanding, and AI-driven models for grading. By doing so, it will enhance the
grading process's efficiency, reduce bias, and ensure accurate assessments of handwritten
responses.