International Journal of Engineering Research & Technology (IJERT)

ISSN: 2278-0181
Vol. 3 Issue 3, March - 2014

Evaluating Student Descriptive Answers Using

Natural Language Processing

Ms. Shweta M. Patil Prof. Ms. Sonal Patil

M.E. (CSE) student, Dept of CSE Assistant Professor,Dept of Computer Engineering
G.H.Raisoni Institute of Engg and Management Jalgaon , G.H.Raisoni Institute of Engg and Management,
Maharashtra, India Jalgaon, Maharashtra,India

Abstract— Computer Assisted Assessment of free-text answers performance. This paper also discusses various techniques
has established a great deal of work during the last years due to underpinned by computer assisted assessment system as well as
the need of evaluating the deep understanding of the lessons’ current approaches of CAA and utilizes it as a framework for
concepts that, according to most educators and researchers, designing our new framework.
cannot be done by simple MCQ testing. In this paper we have
reviewed the techniques underpinned this system, the description The techniques for automatic marking of free-text
of currently available systems for marking short free text responses are basically categories into three main kinds,
response and finally proposed a system that would evaluate the Statistical, Information Extraction and Full Natural Language
descriptive type answers using Natural Language Processing. Processing [2].

A. Statistical Technique
Keywords—Computer Assisted Assessment,Short free text response,
Descriptive type answer, Natural Language Processing. It is only based on keyword matching, hence considered as
poor method. It cannot tackle the problems such as synonyms
RT
I. INTRODUCTION in student answers, nor does it takes into account the order of
words, nor can it deal with lexical variability.
“Computer Assisted Assessment (CAA) is a common term
for the use of computers in the assessment of student learning
B. Information Extraction (IE) Technique:
IJE

[1]”. The idea of using computers to assist learning process has

surprisingly changed the field of learning system. The study in Information Extraction consists in getting structured
the field of CAA started nearly in 70’s. The CAA systems information from free text. IE may be used to extract
developed so far are capable of evaluating only essay and short dependencies between concepts. Firstly, the text is broken into
text answers such as multiple choice questions, short answer, concepts and their relationships. Then, the dependencies found
selection/association, hot spot and visual identification. Most are compared against the human experts to give the student’s
researchers in this field agree on the notion that some aspects score.
of complex achievement are complicated to measure using
objective type questions. Learning outcomes implying the C. Full Natural laguage processing(NLP):
ability to recall, organize and integrate ideas, the ability to It invloves parsing of text and find the semantic meaning of
express oneself in writing and the ability to supply merely than student answer and finally compare it with instructors answer
identify interpretation and application of data, require less and assign the final scores.
structuring of response than that imposed by objective test
items [5]. Due to these students will be evaluated at higher II. RELATED WORK
level of Bloom’s (1956) Taxonomy (namely evaluation and
synthesis) that the essay question or descriptive questions Jana and John developed C-rater [3] to score student’s
serves it’s most useful purpose. answers automatically for evidences of what a student knows
about the concepts already described by C-rater. It is
Many researchers claim that the essay’s evaluated by underpinned by NLP and Knowledge Representation (KR)
assessment tools and by human graders leads to great variation techniques. In this system model answers are generated with
in score awarded to students. Also many evaluations are the help of concepts already given and later student’s answers
performed considering specific concepts. If that particular are processed by NLP technique. Later on only concepts
concepts are present then only award grades otherwise answer detection is done and finally scores are assigned. But there are
is marked as incorrect. So to overcome this problem new some disadvantages of this system as No distinct concepts
system is proposed. specified, incorrect spelling mistakes unexpected similar
Purpose of this paper is to present the new system that can lexicons and many more. Then Raheel, Christopher and
evaluate student’s performance at higher level of Bloom’s Rosheena created IndusMarker [4] an automated short answer
taxonomy by considering the assessment of descriptive type marking system for Object Oriented Programming (OOP)
questions. The system can perform grading as well as provide course. The system was used by instructors to assist the overall
feedback for student’s to make improvement in their performance of student and to provide feedback to the students
about their performance. It exploits the structure matching i.e.

IJERTV3IS031517 www.ijert.org 1716

 International Journal of Engineering Research & Technology (IJERT)
ISSN: 2278-0181
Vol. 3 Issue 3, March - 2014

matching the pre specified structure with the contents of question in number of ways. The system basically focuses on
student response text. Automated Essay Grading (AEG) system multiple sentences response.
was developed by Siddhartha and Sameen in the year 2010 [5].
The aim of the system is to overcome the problems of The basic architecture of proposed system is depicted in fig
influence of local language in English essays while correcting [1] below. It is basically composed of following components:
and by giving correct feedback to writers. AEG is based on
NLP and some of the Machine Learning (ML) techniques. A. Student Module:
Auto-assessor was developed in year 20111 by Laurie and It consists of question editor where question will be
Maiga [6] with an aim to automatically score student short displayed and response editor to enter student response.
answers based on the semantic meaning of those answers.
Auto-assessor is underpinned by NLP technique. This system B. Tutor Module:
consists of component based architecture. The components are In this module question as well as correct response to
created in order to reduce the sentences to their canonical form respective question is entered by tutor. Tutor will also identify
which are used in preprocessing of both supplied correct and enter the keywords from correct answer with their
answers as well as student response. Later evaluation of student respective weights.
response with the correct answer takes place where each word
from correct answer in canonical form is compared with the
C. Processing Module:
word from student response which is in canonical form and
finally scores are awarded for student response. Ade-Ibijola, Both answers i.e. student response and correct answer will
Wakama and Amadi developed Automated Essay Scoring be processed by initially dividing them into token i.e. words.
(AES) an Expert System (ES) for scoring free text answers [7]. Later on noun phrase and verb grouping will be assigned to
AES is based on Information Extraction (IE). This ES is each and every word with the help of Part-Of-Speech (POS)
composed of three primary modules as: Knowledge Base, tagger. This task is accomplished by NLP technique.
Inference Engine and Working Memory. Inference engine uses
shallow NLP technique to promote the pattern matching from D. Answer comparison and Grade Assignment Module:
the inference rules to the data contained in knowledge base. Following text processing module, that actual evaluation of
The NLP module contains: a Lexical Analyzer, a Filter and a student response with correct answer takes place. Each and
Synonyms Handler module. The correctness evaluation is every word of student response is compared with correct
performed by fuzzy model which generate the scores for answer. If exact match is found in word as well as POS tag and
RT
student answer with the help of two parameters: the percentage word position in sentence the scores are assigned.
match and the mark assigned.
After score assignment Final scores are calculated by
III. PROBLEM DEFINITION making summation of assigned scores of all words.
IJE

There are a number of commercial assessment tools E. Projection of Final Scores:

available on the market today; however these tools support
objective type question such as multiple choice Questions or Final calculated scores assigned to student response are
short one-line free text responses. This will assess student’s given in report.
depth of knowledge only at lower level of Blooms taxonomy of Now steps for evaluating the descriptive type answer of the
educational objectives. They fail to assess student’s proposed system are:
performance at higher level of taxonomy of educational
objective. Also these systems fail to check spelling & Step 1: Start.
grammatical mistakes made by students. As well as they Step 2: Form correct answer and store it in table.
were unable to check the correct word order. Even the answers
with wrong word order were awarded assigned scores by mere Step 3: Identify keywords, tag keywords with the help of
presence of words in student response. So to overcome the POS tagger and assigned weights depending upon importance
encountered problems the system is going to be developed that of it presence in sentence.
evaluated students descriptive answers by considering the Step 4: Store synonyms and antonyms into another table.
collective meaning of multiple sentences. Also system will
mark spelling mistakes made and finally scores will be Step 5: set student score to 0. Input student response and
assigned to student answer. The proposed system will try to store it in another table i.e.SR table.
provide feedback to the students so to help them to improve
their performance in academics. Step 6: Now check whether keyword present in SR table if
present assigned score=score + already assigned weight.
IV. PROPOSED SYSTEM Step 7: If keyword not found check in synonyms table and
The proposed system will implement CAA for descriptive on finding assign score=score + already assigned weight.
type answer. The existing system checks single line text Step 8: Check if antonyms present in SR table if present
response without considering word order. So the proposed then score=score * (-1).
system will try to avoid this problem by considering collective
meaning of multiple sentences. The primary focus of this Step 9: Check the position vector of noun and verb
newly proposed system is to determine the semantic meaning combination in input answer and compare it to that of correct
of student answer with a consideration that student responses to answer to verify dependencies of noun and verb in answer.

IJERTV3IS031517 www.ijert.org 1717

 International Journal of Engineering Research & Technology (IJERT)
ISSN: 2278-0181
Vol. 3 Issue 3, March - 2014

Step 10: if grammatical mistakes are present deduce 2 REFERENCES

marks from net score. [1] Perez D. 2007, “Adaptive Computer Assissted Assessment of Free Text
Students answers: an approach to automatically generate students
Step 11: Now make summation of assigned scores of conceptual models.” PhD Thesis, computer science Department
Student response. Universidad Autonoma de Madrid, 2007.
Step 12: If scores calculated are negative then answer [2] Mitchell T, Russell Broomhead P, and Aldridge N. 2002, “Towards
robust computerised marking of free text responses.” In proceeding of
entered by student is INCORRECT, else if scores are in same the 6th computer Assissted Assessment Conference, 2002.
range with already assigned scores then student response is [3] Jana Z. Sukkarieh, and John Blackmore,” c-rater:Automatic Content
marked to be CORRECT. Scoring for Short constructed Responses.” In proceeding of the 22nd
International FLAIRS Conference,2009.
V. CONCLUSION AND FUTURE SCOPE [4] Raheel Siddiqi, Christopher J. Harrison and Rosheena
Siddiqi,”Improving Teching and Learning through Automated Short-
CAA has been an interesting research area since70’s. CAA Answer Marking.” IEEE Transactions on learning technologies, vol.3,
helps in evaluation of student performance accurately and No.3, July-September 2010.
without wastage of time. Most of the CAA developed provides [5] Siddhartha Ghosh and Dr. Sammen S Fatima,”Design of an Automated
promising results compared with results provided by human Eassy Grading (AEG) system in Indian Context.” Internation Journal of
graders. Computer Application (0975-8887), vol.1, No.11, 2010.
[6] Laurie Cutrone, Maiga Chang and Kinshuk, “Auto-Assessor:
But the problem of existing CAA developed so far is that Computerized Assessment System for Marking Student's Short-Answers
they evaluate student performance at lower level by making Automatically”, IEEE International Conference on Technology for
assessment of only objective type questions or essay questions. Education,2011.
So the proposed system will try to overcome this problem by [7] Ade-Ibijola, Abejide Olu, Wakama, Ibiba, Amadi, Juliet Chioma, “ An
Expert System for Automated Essay Scoring (AES) in Computing using
evaluating students at higher level by considering assessment Shallow NLP Techniques for Inferencing”, International Journal of
of descriptive type question consisting of multiple sentences. Computer Applications (0975 – 8887) Volume 51– No.10, August 2012.
The proposed system will consider the collective meaning of
multiple sentences. It will also try to check grammatical as well
as spelling mistakes made in student response.
The proposed system will try to provide Report to student
giving them detail feed back.
RT
IJE

Student Module Student Response (SR)

Processing Module SR
DB
Tutor Module Correct Answer (CA)
Answer Comparison
CA
DB
Grade assignment

Projection of Final Score

Fig. 1. Architecture of proposed system

IJERTV3IS031517 www.ijert.org 1718