See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/343437088

Enhancing College Chat Bot Assistant with the Help of Richer Human Computer
Interaction and Speech Recognition

Conference Paper · July 2020

DOI: 10.1109/ICESC48915.2020.9155951

CITATION READS

1 279

4 authors, including:

Sangeeta Kumari Zaid Naikwadi

Vishwakarma Institute of Technology 2 PUBLICATIONS   1 CITATION   
16 PUBLICATIONS   52 CITATIONS   
SEE PROFILE
SEE PROFILE

Akshay Akole
Vishwakarma Institute of Technology
2 PUBLICATIONS   1 CITATION   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Anatar Manager System View project

Open Simulator for Digital Circuit based Virtual Experiments View project

All content following this page was uploaded by Sangeeta Kumari on 11 August 2020.

The user has requested enhancement of the downloaded file.

Proceedings of the International Conference on Electronics and Sustainable Communication Systems (ICESC 2020)
IEEE Xplore Part Number: CFP20V66-ART; ISBN: 978-1-7281-4108-4

Enhancing College Chat Bot Assistant with the

Help of Richer Human Computer Interaction
and Speech Recognition
Sangeeta Kumari1 ,Zaid Naikwadi1 , Akshay Akole 1 , Purushottam Darshankar2
1
Department of Computer Engineering,
Vishwakarma Institute of Technology, Pune
1
[email protected],2 Persistent India Pvt. Limited

Abstract: Chat bots are the expert systems that The implemented chatbot will solve queries of the
understands and responds to the query asked by users users, provide information to users as they require,
in their own language. Chat bot responds in improve quality of service time and make customers
conversation just like how a human interact with each
happy by providing smart solutions. It also improves
other. It works as a virtual assistant and its accuracy is
determined by finding correlation between user’s
productivity by providing 24/7 service, reduces
queries and answers provided by chat bots. crowd at help desk and also reduces human efforts.
Implemented Chat bot provides two modes like text Students can interact with chatbot on the web via
mode and audio mode for better user experience. In laptop or smartphones. Students ask miscellaneous
audio mode it facilitates an interactive way of answering queries regarding admission details in natural
through voice messages. During Institute’s Academic language and both can respond to their queries with
Admission procedure there is a huge queue at the correct answers. Proposed application is accessible to
enquiry window. S ituation is even more difficult for the customers easily and gives response to users
parents who reside in different cities, states, and
anywhere anytime. Chatbot not only gives response
countries. The goal of this system is to provide a
platform for student and parents to ask queries and
but also self-learn and improve itself in order to
clear doubts through simple English language text improve the quality of service.
messages or audio commands. S tudents and parents will
amalgamate with bot instead of making queue at
II. Related Work
enquiry desk to ask queries related to admission
Over the past one decade’s chat bots gained a lot of
procedure.
attention in almost every online domain, e.g. e
Keyword: Chat Bot, College bots, Interactive bots, commerce platform, travel sites, government sites,
Dialogue based bots, NLP, Text based bots College websites, company’s website, online ticket
booking site etc. Authors Bayu Setiaji and Ferry
I. Introduction Wahyu Wibowo[2] implemented a chatbot based on
After 12th or completion of diploma students are pattern matching using relational databases. They
going to take admission to the engineering. Before have also used normalization of spell check after
taking admission, they faced many problems. checking of spelling to increase the accuracy.
Students and their parents are concerned about In paper [1] authors Karthick Sowndarajan, R John
various queries related to admission. Students get Victor and S Manikandan uses Natural language
confused when selecting a good engineering college. processing to make an android based chat application
They are selecting colleges based on various using java language. They have removed words
parameters like fees, last year cutoff of admission appearing at the starting and at the end using
procedure. So some students contact me via e-mail or stemming process after preprocessing. Preprocessing
make phone calls. Result is an unnecessary crowd for is used to remove special characters and numbers.
enquiry. Admission department also faces problems After stemming processing, remaining words are
solving repeated queries of students. And the college matched with the keywords that are stored in
admission department required extra manpower and dictionary.
money to serve all the queries.

978-1-7281-4108-4/20/$31.00 ©2020 IEEE 427

Authorized licensed use limited to: Carleton University. Downloaded on August 08,2020 at 05:04:57 UTC from IEEE Xplore. Restrictions apply.
Proceedings of the International Conference on Electronics and Sustainable Communication Systems (ICESC 2020)
IEEE Xplore Part Number: CFP20V66-ART; ISBN: 978-1-7281-4108-4

Dungeon Lee, Kyo-Joong Oh and Ho-Jin Choi [5] III. Architecture

designed and implemented an artificial intelligence
based chatbot (referring as college bot) application Interaction between chat bot and user is
for college website, which provides the answer of all defined by process of chatbot design. The chatbot
the college related queries. Authors have user designer defines overall interaction, chatbot
facebook like user interface to make it more personality and the questions that will be asked by
interactive. users. Dedicated chatbot design tools is used b y
designers to speed up the process that allow team
Authors Borah B., Pathak D., Sarmah P., Som B., collaboration, immediate preview and video export.
Nandi S. have done extensive survey on text based User testing is important part of chatbot design.
chat bots. Their focus are on chat bots based on
artificial intelligence and natural language process
based chat bots. They have beautifully listed down
the problems in implementation of text based
chatbots for various types of application.

AM Rahman, Abdullah Al Mamun, Alma Islam

discuss about promising domains of chatbot
applications in detail along with general architecture
of interactive bots. Their work mainly focus on
various existing API’s available for implementation
of chat bots.

Chabot also called interactive agent is a software

which interacts with human via textual or auditory
methods. Programs like chat bot are smart and
designed in such a way that they could work like
conversational partner of human being. Dialog driven
systems included in various fields like information
Figure 1: Architecture
acquisition and customer service uses these type of
smart bot. Simple chat bot systems scan keyword 3.1. BOT Client
within input, and give response with similar pattern
or matching keyword while some chatbot uses User can interact with the system with the
NLP(Natural Language Processing. help of GUI. He/she will enter the query or doubt in
text box provided. Once the query is submitted, that
Guruswami Hiremath developed a text based chatbot request will be further handled by conversation
for education system [10]. Preprocessing, interface (hmi.jar).
tokenization and pattern matching used in this paper
is pretty much similar as used by Bayu Setiaji and 3.2. Conversational Interfaces
Ferry Wahyu Wibowo [2]. Author Guruswami
Conversational interfaces are platforms that
Hiremath used machine learning and NLP for
can have conversation like a real human. Generally,
template matching.
users use Graphical user interfaces (GUI) to give
But all of the above proposed bots lacked filling the commands to the computer. The computer then
gap between how a real human would interact and interprets the meaning of that command and perform
how a bot would respond for the same user queries the desired action. In case of Conversational
thus users having an unpleasant experience. Natural interfaces the user can communicate with computer
Language Understanding is a key aspect in chat bots in their natural language instead of giving command
by which they can understand a human error while or using GUI. For this to be possible, there is use of
having a conversion and can try to correct them and Natural Language Processing (NLP) so that the
extract meaning out of those incorrect sentences. computer will be able to understand the meaning of
Thus, while using these techniques, it has been the input given by the user and perform the task
attempted to work on giving users a pleasant accordingly. Due to the ambiguous nature of
experience while interacting with bots. languages it is difficult for computer to always

978-1-7281-4108-4/20/$31.00 ©2020 IEEE 428

Authorized licensed use limited to: Carleton University. Downloaded on August 08,2020 at 05:04:57 UTC from IEEE Xplore. Restrictions apply.
Proceedings of the International Conference on Electronics and Sustainable Communication Systems (ICESC 2020)
IEEE Xplore Part Number: CFP20V66-ART; ISBN: 978-1-7281-4108-4

understand the correct meaning of the input given by MongoDB is used at the backend which is a
human, which is known as Natural Language cross-platform document oriented database.
Understanding (NLU). MongoDB is a NoSQL database. It uses JSON-like
documents with schemata.
With the help of Natural Language
Understanding it is possible for conversational 3.3.4 Representational State Transfer
interfaces to understand the correct meaning of the (REST API)
query which contains spelling mistakes, wrong
grammar, etc. For example, if a user asks “What is RESTful web services is used which allows
intake of Computer Engineering?” After this the user requesting systems to access and manipulate textual
asks “Who is HOD”, the search will carry forward representations of web resources by using a uniform
the context of Computer Engineering and provide the and predefined set of stateless operations.
appropriate response. RESTful systems are fast and reliable in
3.3. Service Gateway terms of performance and have ability to grow.

3.3.1 Dialogue Definition

Computer System which is intended to IV. PROPOSED SYSTEM

converse with a human is called a Dialog Definition. A. Classification of Questions
Phases in a dialogue system are: Questions submitted by user through web
The user with the help of systems application can be divided into two parts either it is
microphone asks queries in plain English. With the FAQ question or it is transactional question.
help of Automatic Speech Recognizer (ASR) input is Frequently Asked Questions [FAQ] are the
converted into plain text. commonly listed questions and answers, asked in
Natural language understanding unit (NLU) some context. BOT can answer these types of
is used for analyzing the converted text. questions without asking any further question. For
example: Where the college is located? Answer for
The dialog manager is responsible for this question is address of college and this questio n
analyzing the semantic information. The history and does not required any clarification or additional
state of the dialog is kept by dialog manager itself. question.

The task managers have knowledge of

specific task domain, which is contacted by the
dialog manager. User Query
Question
Output generator is used by dialog manager
to produce output.
Conversational
At the last step, the output is rendered with FAQ Question
Question
the help of an output renderer and displayed as
response to the user.
Conversational
FAQ Server
Server(ITOs)
3.3.2 Knowledge Extraction

Structured relational database or XML and Figure2: Classification of Question

documents are used for creation of Knowledge
extraction.

3.3.3 Backend Connectors and Database Conversational Questions : In conversational

questions slots are filled based on the Abstract
Question Descriptor. BOT may asked fallback

978-1-7281-4108-4/20/$31.00 ©2020 IEEE 429

Authorized licensed use limited to: Carleton University. Downloaded on August 08,2020 at 05:04:57 UTC from IEEE Xplore. Restrictions apply.
Proceedings of the International Conference on Electronics and Sustainable Communication Systems (ICESC 2020)
IEEE Xplore Part Number: CFP20V66-ART; ISBN: 978-1-7281-4108-4

question or clarify question in the case if slot is not  useIntentEngine–if set to true , the intent
filled. will be identified using internal intent engine
trained separately (see appendix -3), else
Figure2: Classification of Question
will use BOW (Bag Of Words) specified in
B. Dialog Definition File (DDF) <selector> part of dialogue definition file.
 failureAttempts–is an integer number which
Dialog Definition File (DDF) consists of is number of consecutive attempts that will
Global Flags for defining Dialog behavior. It consists be allowed to user to answer specific ITO.
of different tasks that BOT needs performs. Some In case he/she fails to answer, he/she will be
task are reserved and they are used for special redirected to “helpTask”
purposes.

 Cancel task: Cancelling current task. User

explicitly want to cancel the task.
 Help task : Providing help to user in case
there are consecutive failed attempts in
answering ITO
 Handover task: Coming out of current task
and to handover the chat from bot to human
respectively
 Exit task: If user says (bye, exit or bye bye)
Figure3: Flags to define Dialog Definition
this task will get final confirmation from
user before logout. D. DDF-Task definition:
C. Flags to define Dialog Definition: Task definition consist of unique task name
as an intent. The intent are map to all the possible
 startTaskName -This specify the initial task
utterances using bag of word model after removal of
to be executed by BOT. Initial task may be
stopwords.
other than ”start” but after execution the
initial task BOT will return to start task. In Information Transfer Entity (ITOs) the
 globalLanguage –This specifies the BOT slots are filled on the basis of AQD (Abstract
language. This has to be in ISO language Question Descriptor). A fallback question may be
code. asked by the BOT to the user if slot is not filled and if
 useSODA(System Of Dialogue Act) –Each required flag is true. Clarify question is used in case
utterance will be classified using Max if user fails to provide correct answer.
Entropy Classifier to find whether user is
E. DDF-Action
seeking the information or Providing
information or issuing the defined command DDF-action supports REST api. The API
(e.g. switch ON, Switch OFF) result will be always stored in the result variable and
 allowSwitchTasks–if set to true it will allow accessed using #result.
sub-dialogues from other tasks , if set to
false only sub-dialogues of same task will be V.USER INTERFACE
executed. (e.g. for trip.xmlDDF us e case -
User interface design (UI) is the design of
User can only answer questions specific to
user interfaces for BOT so that user usability can be
getTripInformationand will not allowed to
maximized as well as user experience. Making user’s
switch to getWeatherInformationtask)
interaction simple and efficient is the goal of user
 allowOverAnswering- if set to true the user
interface design.
is allowed to provide more than the
information that has been asked for (but at User can interact with BOT and ask queries
least the current question). using three different ways.
 allowCorrection-if set to true user can
change a value of an already asked question

978-1-7281-4108-4/20/$31.00 ©2020 IEEE 430

Authorized licensed use limited to: Carleton University. Downloaded on August 08,2020 at 05:04:57 UTC from IEEE Xplore. Restrictions apply.
Proceedings of the International Conference on Electronics and Sustainable Communication Systems (ICESC 2020)
IEEE Xplore Part Number: CFP20V66-ART; ISBN: 978-1-7281-4108-4

A. Keyboard Interface

User can ask queries by giving text input gives input through microphone of system, such as
through keyboard. User type their query in the voice commands (e.g. "What is address of college")
provided textbox and hit submit button to submit as shown in figure 4. The bot reply query by
query to BOT. providing address of location.

B. Card Interface

With the help of card interface user can give

input to the BOT by selecting values of the cards.
Card interface provide option to the user to select
particular card and submit query.

C. Speech Recognition

Speech recognition gives BOT ability to

Figure3: Flags to define Dialog Definition

recognize words and sentences in spoken language

and convert them to a machine-readable format.
Speech recognition is used when user

Figure 5: Screenshot of BOT while interacting

with Card Interface

Figure 5 is a snippet of a query of a user enquiring

about courses and intake. In General a variety of
course are offered by a college so bot provides card
interface to user to avoid the ambiguity of fees of
different courses. After that user can provide extra
inputs by selected course type and department from
dropdown option to or by typing message to bot. This
is a transactional type of question where bot require
more information to reply back to the user. Therefore
bot provide a card interface for more user details. So
Chabot is offering to choose a type of course

D. Voice Commands

1) Manual Mode: In manual mode user will have to

interact with BOT for every voice command. User
will input a command the BOT gives reply and stops.
Again if the user wants to ask new question he should
again press the voice command button.

2) Auto Mode: Auto mode is like a real conversation

Figure4: Screenshot of BOT while interacting UI. When auto mode is on user can have a back-to-
with User Interface. back conversation with BOT like he is talking to a

978-1-7281-4108-4/20/$31.00 ©2020 IEEE 431

Authorized licensed use limited to: Carleton University. Downloaded on August 08,2020 at 05:04:57 UTC from IEEE Xplore. Restrictions apply.
Proceedings of the International Conference on Electronics and Sustainable Communication Systems (ICESC 2020)
IEEE Xplore Part Number: CFP20V66-ART; ISBN: 978-1-7281-4108-4

real human being. The BOT uses Text-to-speech to 3) Install scipy module
answer the question through speakers.
4) NLTK packages
.
5) Flask module

VI. CONCLUS ION

Chat bot can be used by any

College and Universities on their website so that
external stakeholders can ask their queries anytime.
And it would be very much effective to simplify the
admission process as well as this chatbot is not only
providing college information but also able to answer
queries related to state wise cut off, Categories wise
off, gender specific cut offs and shift wise cut offs.

User can provide the feedback by pressing

like and dislike buttons based on the replied received
to their query. This data is stored in backend and can
be viewed by Administrator for analyzing the type of
questions asked and improvement of answer if
required.

To check the accuracy of chatbot, same

query has been asked in different form by changing
the wordings of sentences and appending various
especial characters and irrelevant words.

VII. Future Scope

Figure 6: Screenshot of BOT while listening to
Speech Commands Chat bots are ever green area of research of
interest in field of computer science. Bots still
As shown in figure 6, user is trying to interact with
sometimes cannot understand what is asked if
bot with speech command mode so voice option is
highlighted in output screen shot. User will ask there are any glitches in input due to human
spoken language, like a grammar error or a
question and speech recognition model of Chabot
context error.
will try to extract information via audio received
from users. Misinterpretation of any commands may be
because of misspelling can be further improved
V1. Hardware and Software Requirements
with the help of Powerful Natural Language
Below steps describe the deployment of Processing [NLP] techniques. Improvement can
Chat BOT on windows machine. be done till the BOT passed Turing test.

Required configuration of Windows Accuracy of BOT for recognizing a

machine: particular query for different contexts in which
user of asking can be improved, so that the User
 RAM 8GB Minimum can get a more realistic feel like talking to a real
 64 bit processor counselling person. At later stages this bot can
be implemented in local language and
Following software are required for
multilingual as well.
implementation of chat bot:

1) Scikit-learn pip3

2) Numpy module

978-1-7281-4108-4/20/$31.00 ©2020 IEEE 432

Authorized licensed use limited to: Carleton University. Downloaded on August 08,2020 at 05:04:57 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of the International Conference on Electronics and Sustainable Communication Systems (ICESC 2020)
IEEE Xplore Part Number: CFP20V66-ART; ISBN: 978-1-7281-4108-4

VIII.REFERENCES
[1] S. Karthick, R. J. Victor, S. Manikandan and B. Goswami,
"Professional chat application based on natural language
processing," 2018 IEEE International Conference on Current
Trends in Advanced Computing (ICCTAC), Bangalore, 2018, pp.
1-4.
[2] Bayu Setiaji ; Ferry Wahyu Wibowo “ Chatbot Using a
Knowledge in Database: Human-to-Machine Conversation
Modeling.” In IEEE 2016 7th International Conference on
Intelligent Systems, Modelling and Simulation (ISMS)
DOI: 10.1109/ISMS.2016.53
[3] Naz Albayrak ;” An overview of artificial intelligence based
chatbots and an example chatbot application.” In IEEE 2018 26th
Signal Processing and Communications Applications Conference
(SIU),DOI: 10.1109/SIU.2018.8404430
[4] Chaitrali S. Kulkarni, et.al. “ BANK CHAT BOT – An
Intelligent Assistant System Using NLP and Machine Learning.”
In International esearch Journal of Engineering and T echnlgy
(IRJET ) e:ISSN:2395-0056
[5]Dungeon Lee, Kyo-Joong Oh, Ho-Jin Choi. “The ChatBot Feels
You – “ A Counseling Service Using Emotional Response
Generation.” In 2017 IEEE International Conference on Big Data
and Smart Computing (BigComp), DOI:
10.1109/BIGCOMP.2017.7881752
[6] A. Argal, S. Gupta, A. Modi, P. Pandey, S. Shim and C. Choo,
"Intelligent travel chatbot for predictive recommendation in echo
platform," 2018 IEEE 8th Annual Computing and Communication
Workshop and Conference (CCWC), Las Vegas, NV, 2018, pp.
176-183.
[7] Borah B., Pathak D., Sarmah P., Som B., Nandi S. (2019)
Survey of T extbased Chatbot in Perspective of Recent
T echnologies. In: Mandal J., Mukhopadhyay S., Dutta P.,
Dasgupta K. (eds) Computational Intelligence, Communications,
and Business Analytics. CICBA 2018. Communications in
Computer and Information Science, vol 1031. Springer, Singapore
[8] A M Rahman ; Abdullah Al Mamun ,et.al,” Programming
challenges of chatbot: Current and future prospective.” 2017 IEEE
Region 10 Humanitarian T echnology Conference (R10 -HT C)
[9] Ashay Argal ,et.al “Intelligent travel chatbot for predictive
recommendation in echo platform.” In 2018 IEEE 8th Annual
Computing and Communication Workshop and Conference
(CCWC)
[10] Guruswami Hiremath, et.al,” Chatbot for education system.”
In International Journal of Advance Research, Ideas and
Innovations in T echnology, ISSN: 2454-132X
[11] Milla T Mutiwokuziva, et.al “A neural-network based chat
bot.” In 2017 2nd International Conference on Communication
and Electronics Systems (ICCES)
[12] Godson Michael D'silva , et.al “Real world smart chatbot for
customer care using a software as a service (SaaS) architecture.” In
2017 International Conference on I-SMAC (IoT in Social, Mobile,
Analytics and Cloud) (I-SMAC).

978-1-7281-4108-4/20/$31.00 ©2020 IEEE 433

Authorized licensed use limited to: Carleton University. Downloaded on August 08,2020 at 05:04:57 UTC from IEEE Xplore. Restrictions apply.
View publication stats