Proceedings of the International Conference on Sustainable Computing and Data Communication Systems (ICSCDS-2022)

IEEE Xplore Part Number: CFP22AZ5-ART; ISBN: 978-1-6654-7884-7

A Natural Language Processing based Intelligent

Bot Application
Vindhya Vasini K Tejasvi K Pravallika K
Department of Information Technology Department of Information Technology Department of Information Technology
V. R Siddhartha Engineering College V. R. Siddhartha Engineering College V. R. Siddhartha Engineering College
2022 International Conference on Sustainable Computing and Data Communication Systems (ICSCDS) | 978-1-6654-7884-7/22/$31.00 ©2022 IEEE | DOI: 10.1109/ICSCDS53736.2022.9760998

Vijayawada, India Vijayawada, India Vijayawada, India

[email protected] [email protected] [email protected]

Ch. Nanda Krishna

Department of Information Technology
V. R. Siddhartha Engineering College
Vijayawada, India
[email protected]

Abstract— In the modern day technology, superior models Following the lead, Microsoft delivered Cortana into life.
can be designed due to technology like Natural Language Taking this a step forward, clever audio system had been
Processing, Open CV, python etc. and bot assistant is one delivered which made voice co mmunication v iable among
among them. Personal robot assistants assist lowering the human beings and the bots. Amazon A lexa & Google Ho me
guide efforts being placed by humans for their daily tasks. In
this paper, a voice-managed non-public assistant has been represent a specific class of conversational UI.
proposed. The human voice instructions are given to the robot Basic definitions and Back ground:
assistant remotely and the bot can carry out distinct
movements, turns, start/prevent operations and relocate an  Artificial Intelligence (AI), the capacity of a virtual
item from one area to another. The speech sign instructions laptop or laptop-managed robot to carry out duties
transformed to textual content are communicated to the robot typically related to human beings.
over a Bluetooth network. The non-public assistant robot is  Natural Language Processing (NLP) refers back to
built primarily on a micro-controller based platform and is
the department of AI giv ing computer systems the
aware of the modern location. The effectiveness of the voice
manipulated communication over a distance is measured via
capacity to apprehend textual content and spoken
numerous experiments. Performance assessment is completed phrases in a whole lot, identical as humans can.
with encouraging effects of the preliminary experiments.  Python is a famous programming language. It was
Moreover, possible enhancements are mentioned towards the created by Guido van Rossum, and launched in
capability packages in home, hospitals and industries. 1991. Python is an interpreted high-degree
generalised programming language. Its layout
Keywords— NLP, OpenCV, Python. philosophy emphasizes code clarity with its use of
I. INT RODUCT ION massive indentation. Its language constructs in
addition to its object-orientated method, intends to
While many might agree that Chabot has end up a buzzword assist programmers write clear, logical code for
recently, however the idea has its life fro m the t ime whilst small and large-scale tasks.
human beings began out growing approaches to engage with  Co mputer Vision (CV) is the subcategory of AI that
computers. The first-ever Chabot became delivered even makes a speciality of constructing and using virtual
earlier than the release of private computers. It became structures to process, examine and interpret visib le
evolved via way of means of Artificial Intelligence data.
Laboratory by Joseph Weizenbaum in 1966 and was named
Eliza. Eliza tested the key phrases acquired as entered, after Hardware requirements:
which the output is precipitated in step with a described set
of rules. This technique of producing output continues to be PROCESSOR : Intel core i5
utilized by some chatbots. The subsequent chatbot became SYSTEM TYPE :64-bit operating system
Parry, written by psychiatrist Kenneth Colby on the Stanford PRIMARY STORAGE : 8GB RAM
University in an try to simu late someone with paranoid SECONDARY STORAGE: 1TB
schizophrenia. Then the A.L.I.C.E, wh ich was evolved in
1995 by Richard Wallace. While A.L.I.C.E gained the
Loebner prize thrice, it did not skip the Turing check. A II. LITERATURE REVIEW
Turing check examines whether now or no longer a device
A. Virtual Assistant for blind people:
is capable of working intelligently like hu man beings. After
this, numerous digital assistants had been released. Siri v ia  The task of this virtual assistant provide operations
Apple became the primary one to introduce convers ational like greet ing user. Different applications can be
assistants. The idea received recognition and shortly after, opened and provide operations of multimedia and
Google released their Google Assistant for Android. so on. Particularly this assistant helps the people

978-1-6654-7884-7/22/$31.00 ©2022 IEEE 391

 Proceedings of the International Conference on Sustainable Computing and Data Communication Systems (ICSCDS-2022)
IEEE Xplore Part Number: CFP22AZ5-ART; ISBN: 978-1-6654-7884-7

who are b lind to co mplete their tasks [1]. The III. PROPOSED WORK
advantages in this approach are searching in
A. Existing Method:
Wikipedia, p laying music and telling time and the
disadvantages in this methodology is that it cannot
perform all the operations , i.e., it can just do small
operations like searching.
B. Pravas Sarthi- A Convience MultiLingual Virtual
Assistants:
 Helps in co mmun ication by providing other desired
languages for the users while travelling [2]. The
advantages in this approach is that it helps the
people who have language problem and the
disadvantage in the approach is that it requires
large amount of data.
C. Next- generation of virtual personal assistants Microsoft
Cortana, Apple Siri, Amazon Alexa and Google Homes:
 Helpfu l in industrial sector for the people in their
Fig. 1. Personal Assistance
daily work activ ities [3]. Advantage in this approach
is the interactive conversational systems that helps
Proposed Method:
the humans to interact with the machines and the
disadvantage is that sometimes every operations like The main mot ivation for the proposed work is to
dialogue can't be heard clearly and doesn't respond provide a voice controlled personal assistant with a
to what is asked. secure login, so that the users can manage their
everyday activities by having an assistant bot in their
D. IRJET Voice Recognition (AI): Voice Assistant Robot: desktops [5]. By provid ing face recognition, the users
 Every co mpany need their clients to know about can have a secure login and after proper validation, the
their co mpany and about innovating new projects. user can give the commands to the system, and if the
For that purpose, to have best interactive session, commands are recognized by the system, it will provide
advanced website for digital marketing. Having an the desired output by replying with human voice, so
advanced website is not a big plat form to solve the that the users can feel like they are talking to a normal
queries of clients because clients wont able to wait person. It will be mo re helpful for b lind people since it
too long for responses , because assigned humans can be managed through voice commands.
can't answer everyone at the same time. Hence a
chatbot which have enough intelligent to solve all
the queries of clients is required. The advantage in
the approach is best intelligent chatbot for solving
client’s queries in co mpanies [4]. The disadvantages
is that as many previous chatbots already exist, there
are still some problems in making data-driven
system due to a large amount of data required for
development.
E. Problem Statement:
Currently everything is in its advanced version
because of technologies and bot assistant is one of
them. Personal robot assistants assist by decreasing
the efforts being placed by human beings to carry
out their daily tasks. In this applicat ion, a voice-
managed non-public assistant, advanced on a micro-
controller based platform aware about the modern
location has been proposed. The effectiveness of the
voice manipulate co mmunicated over a d istance is
measured by numerous experiments.

978-1-6654-7884-7/22/$31.00 ©2022 IEEE 392

 Proceedings of the International Conference on Sustainable Computing and Data Communication Systems (ICSCDS-2022)
IEEE Xplore Part Number: CFP22AZ5-ART; ISBN: 978-1-6654-7884-7

spoken through someone and changing them to a layout ,

which is comprehensible through a machine [15].
Depending at the end goal, if it's far, then it is transformed
to textual content or voice or every other required layout.
C. Rules Edit, rule based machine translation and rule
based programming:
 Rules take the frame of: "{In the event that: At that
point} expression" (eg.
{IF'condition'THEN'result'}, or more particu larly
{IF'red'AND'octagon' THEN' stopsign. }). The
single run itself isn't a show, as the run applies it is
the case that the condition is met.
 The RBMT framewo rk with the input sentence
(source dialect) produces them based on
morphological, syntactic, and semantic
examination of both the source and target
languages involved with in the important specific
translation assignment and yield the art iculation
(within the target dialect [17].
 This programming may be a programming
worldview based fundamentally on fo rmal
rationale. Any program co mposed in a rationale
programming d ialect, may be a set of sentences in
consistent shape, communicating actualit ies and
rules appro ximately a few issue space [19]. Majo r
rationale programming d ialect families incorporate
Prolog, Answer Set Programming (ASP) and
Information log. In all of these languages, rules are
written in the form of clauses:
H :- D1, …, Dn and are read declaratively as
logical implications.
Fig. 2. Desktop Personal Assistant with Face recognition H if D1 and … and Dn. H is called the head of the
authentication
rule and D1, ..., Dn is called the body. Facts are
The proposed architecture shows the whole process how rules that have no body, and are written in the
the project works. First, the user have to go through the face simplified form: H.
recognition process for access. After the valid face D. Algorithm
authentication, the user can get access to the bot to give
commands using system mic , and then the command is
processed in backend process which is done in python. Here Input: Voice commands
in the backend process , the system first recognizes the user Output: Required Task Performed
command and compares it to the set of rules that are Algorithm:
programmed in bot [6]. Then the output is displayed as per function takeCommand(){
the input. To perform the who le process, tools like speech r = speech_recognition.Recognizer()
recognition, face recognition, mo ment recognition, pyttsx3 with speech_recognition.Microphone() as source:
etc. have been used. print("listening....")
r.pause_threshold = 1
B. Description of Algorithm:
r.adjust_for_ambient_noise(source)
#audio = r.listen(source)
Each type of approach makes use of a set of rules to audio = r.listen(source)
generate policies fro m the pattern statistics. These policies try:
are then carried out to new statistics. print("Recognizing....")
Set of rules offer mechanisms that generate policies by: query = r.recognize_google(audio, language = "en-
1. Focusing on a particular elegance at a time. in")
2. Maximizing the possibility of the preferred type. except Exception as e:
In this utility the set of ru les used is primarily based set of #print(e)
rules [9]. These algorith ms extract knowledge fro m the return "None"
model, which might be easy to recognize and is expressive. return query
This set of rules is appropriate for studying statistics }
containing a combination of nu merical and qualitative function mainTask(){
attributes. Main topic of this utility is Speech reputation that while(true){
refers to a personal computer deciphering the phrases textCommand = takeCommand();

978-1-6654-7884-7/22/$31.00 ©2022 IEEE 393

 Proceedings of the International Conference on Sustainable Computing and Data Communication Systems (ICSCDS-2022)
IEEE Xplore Part Number: CFP22AZ5-ART; ISBN: 978-1-6654-7884-7

if("required task" == textCommand){  Face Recognition: The technology may change

performTask() when it comes to facial recognition. In this project,
} to get started, the camera authenticates the face
} with the previous registered images. The face is
} best identified when the person is looking directly
Steps: at the camera, as this facilitates facial recognition
 Input: voice commands, user face perfectly. With the advancement in technology, it
 Output: responds back in speech mode by showing has been improved to be able to recognize the face
required output that user wants. with a slight variat ion in their posture facing the
 login using face recognition camera. The photo of the face is then captured and
 take user commands as input analysed.
 if
V. RESULTS AND OBSERVATIONS
the command matches the functions that
are present in code, then the output will be A. Stepwise Description of Result:
displayed.  This section consists of results of creating the user
 else interface using the flask.
A voice with dialogue like sorry co mmand  User face recognition using open cv for secure
is heard. access is provided.
 Once the user accesses the interface through this
IV. DESCRIPTION OF TOOLS USED face recognition, the process starts.
This section discusses about the tools used in this work.  In this process, there are two sides: input side and
output side.
A. Tkinters:
 The function happening in input side is that the
 Tkinter – It is a de facto standard GUI (graphical user can give input commands and these
user interface) package for Python. This is a thin commands are displayed in that interface.
object-oriented layer on top of Tcl / Tk. Tkinter is  The user commands are processed in backend
not the only GUI programming toolkit for Python , process which is python that includes NLP fo r
this framework provides Python users with an easy word extraction.
way to create GUI elements using the widgets that  After co mparison of ru les that are already stored in
come with Tktoolkit.tkinter [10]. Moreover, it is code with user command, if the system identifies,
most common ly used. It is also a graphical user then the output will be displayed.
interface (GUI) Package for python. In this pro ject,
tkinter is used for desktop user interface for users. B. Test-case Results:
It plays a vital ro le in creating a graphical user
interface for desktop based application. As our Fig.3 shows how the commands are taken by the
project (Intelligent Bot Application) is a desktop system. So, here in the image, if the user says, ‘open
based application, so tkinter library is used for user YouTube’, then the system recognizes the command
interface and it is a default library that everybody and responds as shown in the image.
can access under python license. Most commands
are available as Tkinter or Tkinter.ttk classes.
B. pyttsx3:
 pyttsx3 is a Python text -to-speech library. Unlike
alternative libraries, it works offline and is
compatible with both Python 2 and Python 3. The
application calls the pyttsx3 file.
C. Speech Recognition:
 There are many speech recognition engines which
are supported by python like APIs, including
Google Speech Engine, Google Cloud Speech API,
Microsoft Bing Vo ice Recognition and IBM .
WhatsApp messages at a certain time, has several Fig. 3. Voice Command Module
other features too.
 Following are some features of py what kit module: Fig.3 represents the terminal where the user can
1. Play a YouTube video. give his input.
2. Perform a Google Search.
 Get information on a particular topic.
 Movement Recognition: The program written in
python will detect motion and store the time
interval of the motion.

978-1-6654-7884-7/22/$31.00 ©2022 IEEE 394

 Proceedings of the International Conference on Sustainable Computing and Data Communication Systems (ICSCDS-2022)
IEEE Xplore Part Number: CFP22AZ5-ART; ISBN: 978-1-6654-7884-7

communicat ion process. Simp ly put, chatbots speed up

services and make the best personal assistant. It can be
used for the benefit of the co mmon people. You can also
book or buy tickets, rooms, and tables instantly , or use
them fo r a variety of reasons, including sending alarms,
setting notes, meeting reminders, and dosing time
alarms.

The proposed methodology helps in defining the

whole procedure of how the project wo rks and it helps to
understand the concepts that are used in pro ject
implementation. As shown in the proposed system, the
entire procedure of the project is done in python by
using different tools like open cv for face recognition,
pyttsx3 for text to speech conversion, speech
Fig. 4. Disered output for the input given by user recognition, mo ment recognition and PyQt5 , t kinter to
create user interface as it helps how the user can interact
Fig.4 shows the desired output for the input given with system and how the system perform and responds
by the user. That is, the input given by the user is YouTube back to the user using geolocation module, that is,
and the YouTube page is displayed on the screen. various frameworks like geopy & geocoder, where the
Similarly, the user can access other service provider details. map feature is integrated that takes addresses from
database and displays these locations on map.

VI. CONCLUSION AND FUTURE WORK

A. Conclusion
At present, technology has changed everything. Everyone is
liv ing a technical life because machines have become a part
of human ’s everyday life act ivit ies. Therefore, this project is
implemented to make the hu man tasks easy that are related
to the desktop activities. With the help of Jarvis bot assistant
which is a personal desktop assistant, the user can perform
multi-tasking at a time.
Fig. 5. Disered output for the input given by user for word In this project, unlike other assistant bots the user can
document use it in secure way undergoing face recognition
authentication and by using NLP with python in backend
process, which works like a personal assistant that perform
tasks like opening word and printing words with user voice
command without any handwork and helps login to desired
applications and acts like an assistant that helps user to
complete simple tasks.
The novelty of this project (Intelligent Bot Applicat ion)
is that, unlike other bot assistant this assistant can activate
the typing mode to write the word documents. Therefore,
the user can print the content through voice commands
without any physical typing requirement as if Cortina in
windows which does not provide this feature to user,
whereas it just opens that application. Moreover, this
Fig. 6. Output for speech recognition for typing mode in personal assistant bot provides face recognition
word document authentication so that the users are secured fro m data
breaches, and intrusion activities. It provides automatic
In Fig.5 and Fig.6 the assistant helps in opening the MS google search, YouTube search, email sending, WhatsApp
word and activates the typing mode for the user if text sending through voice commands, which co mp letely
necessary. Therefore, the user can print the sentences works and responds like hu man by acting like actual
through voice commands without using the keyboard. assistant for the users.
B. Future Scope
Therefore, chatbots not only can be used for chatting
and getting information, but also require artificial In the future, more work will need to be done with chatbots,
intelligence fro m a whole new perspective. The main and new developments of Artificial Intelligence (AI) systems
advantage of chatbots is that they can be serviced at any that can organize natural human-machine interactions (voice,
price. Immed iately responding to user requests with communication, gestures, facial e xpressions, etc.) which is
exactly relevant information speeds up the also required. One of the most studied and popular

978-1-6654-7884-7/22/$31.00 ©2022 IEEE 395

 Proceedings of the International Conference on Sustainable Computing and Data Communication Systems (ICSCDS-2022)
IEEE Xplore Part Number: CFP22AZ5-ART; ISBN: 978-1-6654-7884-7

developement is the direction of interaction based on the Conversational Interface for Weather Information,” IEEE
machine understanding of machines in natural human Transactions on Speech and Audio Processing, vol. 8, no. 1, pp.
85–96, 2000.
language. It also develops human learning to communicate [11] M. Kolss, D. Bernreuther, M. Paulik, S. Stucker, S. Vogel, and
with machines, which learns to communicate with humans, A. Waibel, "Open Domain Speech Recognition & Translation:
exp lores his habits and behaviors, and seeks to become his Lectures and Speeches", in Proceedings of ICASSP, 2006.
personal assistant. [12] D. R. S. Caon, T. Simonnet, P. Sendorek, J. Boudy, and G.
Chollet, "vAssist: The Virtual Interactive Assistant for Daily
Currently the rule-based chat bot for desktops with face Homer-Care", in Proceedings of pHealth, 2011.
recognition authentication that opens the desired application [13] Crevier, D. (1993). AI: The T umultuous Search for Artificial
has been mplemented. As it is a rule-based voice controlled Intelligence. New York, NY: Basic Books, ISBN 0-465-02997-
3.
personal assistant, it has been programmed with some basic [14] R. Gayatri, C. Kalieswari, "Multi-User Chatbot Application", in
rules which will decrease the necessity of user's everyday International Journal of Engineering and Advanced Technology
activities. In future, the project can be extended by adding (IJEAT ) ISSN: 2249 – 8958 (Online), Volume-9 Issue-5, June
additional advanced features like: automatic Google search 2020.
where the users can automatically manage their searches [15] B. A. Shawar, E. Atwell, "Different measurements metrics to
through voice commands by connecting to the google search evaluate a chatbot system", Proceedings of the Workshop on
Bridging the Gap: Academic and Indust rial Research in Dialog
directly, WhatsApp text sending through voice commands T echnologies, 2007.
where the users can send the send the text messages through [16] H. N. Io, C. B. Lee, "Chatbots and Conversational agents: A
voice commands to their desired contacts, automatic Bibliometric analysis", in IEEE International Conference on
YouTube search where the users can directly search the Industrial Engineering and Engineering Management (IEEM) ,
content in the YouTube through voice commands, and 2017.
[17] Rohit Tamrakar, Niraj Wani, "Design and Development of
antonyms and synonyms finding to help in finding the CHAT BOT: A Review",
meanings of words. https://www.researchgate.net/publication/351228837_Design_a
nd_Development_of_CHATBOT_A_Review
REFERENCES [18] Mondal, Anupam, et al. "Chatbot: An automated conversation
[1] Avanish Vijaybahadur Yadav, Sanket Saheb Verma, Deepak system for the educational domain". 2018 International Joint
Dinesh Singh, "Virtual Assistant for blind people," 2021 Symposium on Artificial Intelligence and Natural Language
International journal of advance scientific reasearch and Processing (iSAI-NLP). IEEE, 2018, pp. 1-5.
engineering trends, Volume-6, Issue-5. [19] Ahmad, Nahdatul Akma, et al. "Review of chatbots design
[2] G. Bhatia, H. Tewani, A. Gunda, S. Kamat and A. Shan kar, techniques". International Journal of Computer Applications,
"Pravas Sarthi - A Convenience: MultiLingual Virtual vol. 181, no. 8, pp. 7–10, 2018.
Assistant," 2019 10th International Conference on Computing, [20] Adamopoulou, Eleni, and Lefteris Moussiades. "An overview of
Communication and Networking T echnologies (ICCCNT), chatbot technology". IFIP International Conference on Artificial
2019, pp. 1-5, doi: 10.1109/ICCCNT45670.2019.8944428. Intelligence Applications and Innovations. Springer, Cham, vol.
[3] V. Këpuska and G. Bohouta, "Next-generation of virtual 584, 2020.
personal assistants (Microsoft Cortana, Apple Siri, Amazon
Alexa and Google Home)," 2018 IEEE 8th Annual Computing
and Communication Workshop and Conference (CCW C), 2018,
pp. 99-103, doi: 10.1109/CCWC.2018.8301638.
[4] Munira Ansari, Saalim Shaikh, Mohammed Saad Parbulkar,
Talha Khan, Anupam Singh, 2021, Intelligent Chatbot,
INTERNATIONAL JOURNAL OF ENGINEERING
RESEARCH & TECHNOLOGY (IJERT) NREST – 2021
(Volume 09 – Issue 04).
[5] “ Namita Mhatre, Karan Motani, Maitri Shah, Swati Mali,
"Donna Interactive Chat-bot acting as a Personal Assistant" in
International Journal of Computer Applications (0975 – 8887)
Volume 140 – No.10, April 2016.
[6] M. Dahiya, "A Tool of Conversation: Chatbot" in International
Journal of Computer Sciences and Engineering, Volume-5,
Issue-5.
[7] Carlene Lebeuf, Margaret-Anne Storey, and Alexey Zagalsky,
"Software Bots" in IEEE.
[8] Gk_, "Soul of the Machine: How Chatbots Work.",
https://medium.com/@gk_/how-chat-bots-workdfff656a35e2.
[9] R. Belvin, R. Burns, and C. Hein, "Development of the HRL
route navigation dialogue system", in Proceedings of ACL-
HLT , 2001
[10] V. Zue, S. Seneff, J. R. Glass, J. Polifroni, C. Pao,
T.J.Hazen,and L.Hetherington, “JUPITER: A Telephone Based

978-1-6654-7884-7/22/$31.00 ©2022 IEEE 396