An Interactive Chabot using NLP and Deep Learning

ABSTRACT
In recent times, design and implementation of chatbots have received great attention from
developers and researchers. Chatbots are Artificial Intelligence (AI) based conversational
systems which are able to process human language through various techniques including Natural
Language Processing (NLP) and Neural Network (NN). The main goal of this paper is to
summarize some of the most efficient implementation techniques that have been carried out in
previous years. This paper is not only analyzing critically the previous works on chatbots but
also suggests a proposed methodology in order to develop a state-of-the-art chatbot application
that can be personalized easily according to customer needs. The proposed chatbot can be
implemented using a couple of tools such as DialogFlow, TensorFlow, Android Studio and
Firebase. The proposed Chatbot will be implemented using a couple of tools such as DialogFlow,
TensorFlow, Android Studio, and followed by Machine Learning (ML) and Deep Learning (DL)
techniques including Neural Machine Translation (NMT) and Deep Reinforcement Learning
(RL) models.

1. Introduction
Chatbots – also known as “conversational agents” – are software applications that mimic written
or spoken human speech for the purposes of simulating a conversation or interaction with a real
person. There are two primary ways chatbots are offered to visitors: via web-based applications
or standalone apps. Today, chatbots are used most commonly in the customer service space,
assuming roles traditionally performed by living, breathing human beings such as Tier-1 support
operatives and customer satisfaction reps.

Conversational agents are becoming much more common partly due to the fact that barriers to
entry in creating chatbots (i.e. sophisticated programming knowledge and other highly
specialized technical skills) are becoming increasingly unnecessary.

Here, a method for developing chatbots to serve their users. In general, these chatbots are used
for answering questions in many businesses, providing customer information, providing train
schedules, helping customer reservations, virtual assistants; serve as call centers to serve ten
million customers automatically. A deep learning based conversational artificial intelligence
technique was used as tools for learning conversation between machine and customer. In
addition, the steps required are the technique used in conjunction with the convolution neural
network technique by using Tensorflow training to improve the accuracy of these chatbots. From
the experimental results, using deep learning for chatbots learning, the accuracy is better than the
traditional model.
Finally, the use of technology is expanding widely in everyday life and changing the way of
providing services in many sectors. Consequently, Chatbots can be used in the education sector
as a virtual assistant for students to clarify their doubts and make their life easier.

Basic Terminology for Chatbot-:

1. Chatbot-: A Chatbot is nothing but a software application that helps in conducting a
conversation by using auditory or text-based methodologies. The programs in a Chatbot are
developed to imitate human conversations. The use of Chatbot’s for various purposes include
customer services, request routing, or for information gathering. Some chatbots are extensively
used for word-classification processes by using Deep Learning.

2. Artificial Intelligence-: Artificial Intelligence simulates human intelligence power in various

devices. These AI devices are programmed to reflect like human beings and mimic their actions.
Artificial Intelligence represents the machines that exhibit natural traits that are similar to human
minds such as Problem-solving and Comprehension.

3. Deep Learning-: Deep learning is a branch of machine learning that mimics the neuronal
system in the human brain ( neural network) . Therefore, its ability in the future may be
supernatural. It can increase the amount of processing power, which can simulate the nerve and
neurons (called node) into Artificial Neural Network (ANN) .

4. Tensorflow-: Tensorflow is a software library framework that mainly focuses on machine

learning that uses dataflow graphs and differentiates programming over various number tasks to
build models. It is used by the developers to create large-scale development applications
including the neural networks. It is mainly applied for Classification, Understanding, Prediction,
and Creation.

Types of Chatbot (Reference Google)

Related Work-:
The purpose of this study is to review the literature of business domain conversational agents
with a focus on machine learning. In this regard, several surveys have been performed from a
professional perspective to study conversational agents. The research proposed by Hussain et al
focuses on the classification of chatbot and chatbot design approaches, where the authors
discussed task-oriented and non-task-oriented chatbot categories. These categories have been
taken into account in a debate about how the conversational meaning is treated by chatbots.
Several machine learning techniques, such as Recurrent Neural Network, Sequence to Sequence
neural models and Long Short-Term Memory Networks, were examined by the authors for this
reason. Lokman and Ameedeen presented a scientific analysis of five recent literature chat-bot
systems. Features such as the information domain, response generation, text processing and
machine learning models were considered by the authors. In addition, to evaluate the
performance of the chatbot, they checked the implementation and the assessment approach. A
research on task-oriented and non-task-oriented models was performed by Chen et al, identifying
deep learning techniques and algorithms. The authors addressed research directions that can
exploit research into the dialog method, such as the warm-up stage in domain-specific chatbots, a
deep understanding of language and the real world, and sensitive informationrelated privacy
issues. Nuruzzaman and Hussain suggested a survey to compare eleven applications with
features and technical requirements for the implementation of chatbots. In order to provide
productive and efficient chatbot communications, the authors outlined current limitations. Latest
studies, on the other hand, have reviewed literature based on the business application of
conversational agents. The authors explain the reasons for the increase in the utility of chatbots
and their future in the scope of business in the study presented by Kaghyan etal. In addition, by
contrasting capacities, strengths, and weaknesses, they suggested a discussion about building
platforms. The analysis that most approximates this work was done by Meyer von Wolff et al.

Methodology
The basic definition of chatbot is, it is a computer software program designed to simulate human
conversation via text or audio messages. Today’s AI systems can interact with users, understand
their needs, map their preferences and recommend an appropriate line of action with minimal or
no human intervention. There are lot of popular conversational agents are available today like
Apple’s Siri, Microsoft’s Cortana, Google Assistant, and Amazon’s Alexa.

The basic foundation of chatbots is providing the best response of any query that it receives. The
best response like answering the sender questions, providing sender relevant information, asks
follow-up questions and does the conversation in realistic way.

The following picture shows that how Deep learning based chatbot work internally,
 Source: Microsoft

Since we will be developing a Chatbot with Python using Machine Learning, we need some data
to train our model. We can just create our own dataset to train the model. To create this dataset to
create a chatbot with Python, we need to understand what intents we are going to train. An
“intention” is the user’s intention to interact with a chatbot or the intention behind every message
the chatbot receives from a particular user. The strategy here is to set different intents and create
training samples for those intents and train your chatbot model with these sample training data as
model training data (X) and intents in as model training categories (Y).

Prerequisites:

 Tensorflow :2.3.1

 Nltk : 3.5

 Colorama : 0.4.3

 Numpy : 1.18.5

 scikit_learn : 0.23.2

Datasets:

Now we need to define a few simple intents and a group of messages that match those intents
and also map some responses based on each intent category. I’ll create a JSON file named
“intents.json” including this data as follows:
STEP 1: Data Preparation-:

The First step of this task to create a chatbot with Python and Machine Learning is to prepare the
data to train our chatbot. I’ll start this step by importing the necessary libraries and packages:

Step 2: Now let the model read the Json file and understand the intents.

Step 3 : Now we need to use the label encoder method provided by the Scikit-Learn library in
Python:

➢ Label Encoding : Label Encoding refers to converting the labels into a numeric form
so as to convert them into the machine-readable form. Machine learning algorithms can
then decide in a better way how those labels must be operated. It is an important pre-
processing step for the structured dataset in supervised learning.

Step 4 : Now we need to vectorize the data using the Tokenization method to create a chatbot
with Python and Machine Learning:

➢ Tokenization : Tokenization is the process of dividing text into a set of meaningful

pieces. These pieces are called tokens.

Step 5 : Now the next and most important step in the process of building a chatbot with Python
and Machine Learning is to train a neural network. Now, I will train and create a neural network
to train our chatbot and then save the model.

The proposed system uses the Deep learning at its core. The proposed system uses the
TensorFlow to make a neural network and train it with intent file to generate a response model.
This response model can be used to predict the response from the query of the user.

In this model we developed the chatbots with the following details: when given user input,
heuristics was used by the system to locate the best response from its database of pre-defined
responses. The most appropriate response template may involve simple algorithms like keywords
matching or it may require more complex processing with machine learning or deep learning for
the dialogue selection process which is the essentially a prediction problem, and using heuristics
to identify. Regardless of the used of heuristic, these systems only regurgitate pre-defined
responses and do not generate new output. With a large amount of data available, to build a
retrieval based conversational system can be created more easily as information retrieval
techniques are developing fast. To query the candidate response from the system, a user’s input
utterance was submitted as a query. The matching metrics were performed to give the better
results. We presented the retrieval based conversational system by using the matching algorithms
between the input utterance and the candidate responses. The inner product of two feature
vectors for queries and candidate responses was calculated. Before the inner product was
performed a transformed space of these two inputs are calculated. A mapping from the original
input to the feature vector was the importance steps to be performed. A retrieval technique with
two steps was considered. Firstly, a fast ranking by standard TF-IDF measurement and secondly,
the re-ranking process using conversation-oriented features designed with human expertise. In
this research, we applied the statistical language model called cross-lingual information retrieval
for select the most matching between the query and responds pairs in database. Inside this
method, it learn representations with deep neural networks (DNNs). Inside the representations
are based on shallow representations, which used the one-hot representation of word. To use
deep neural networks they can extract underlying abstract features of data automatically by
exploring multiple layers of non-linear transformation. The combination between convolution
neural networks (C-NNs) and recurrent neural networks (RNNs) was used inside this model. A
series of matching methods can be applied to short-text conversations for retrieval-based
systems. Not all of these methods were originated for conversation problems; however, they give
a good effective for short-text matching tasks and are included as strong baselines for retrieval-
based conversational studies.

To use Deep Learning to train chatbots to talk intelligently, to address issues, and to translate our
purposes accurately from the questions we ask. The model uses sequence-to-sequence (Seq2Seq)
or encoder RNN. The advantage of the Seq2Seq model is that it does not limit the size of input
and output like any other machine-learning model. We are not necessary to know the size of the
input that comes in advance, such as translating machine translation, speech recognition, or
question answering using the chatbots. Dataset used in the training phase of this work is a work
of the log of the message. The message was wrote on social media such as Facebook was used to
train the model. The data in social media log was the main data set. We provide python code to
organize the data into a working format and use Word2Vec to create the word vector. The
Seq2Seq model was trained. We wrote the code on TensorFlow that has the rnn_seq2seq
function available. We, deploys a model that is already trending using the Flask server, linking to
the Facebook Messenger chatbots. The aim of this training was to develop the intelligence
chatbots, which can interact to the user and can talk by using its own language.
Results-:
In my work, I used the convolution neural network technique by using Tensorflow training to
improve the accuracy of these chatbots. From the experimental results, using deep learning for
chatbots learning, the accuracy is better than the traditional model. Ours research’s objective is to
develop a prototype of interactive Chatbot system which has better accuracy and also uses
images. Deep learning technique is used to help chatbots more powerful.

CONCLUSION-:

In conclusion, this paper reviewed the design and implementation of Chatbot techniques. A
number of previous relevant works are outlined properly, along with their strengthens and
weaknesses. Furthermore, a proposed methodology for implementing a state-of-the-art chatbot
has been suggested. In this article, a number of selected articles have been covered in the
literature review, concentrating primarily on Chatbot design techniques in the last decade. A
survey of selected studies that affect Chatbot design has been presented, and the contribution of
each study has been identified.

REFERENCES

• Deep Learning for Chatbots

• Dialogue Intent Classification with Long Short-Term Memory Networks

• Sequence to Sequence Learning with Neural Networks

• A Neural Conversational Model

• Neural Machine Translation by Jointly Learning to Align and Translate

• Effective Approaches to Attention-based Neural Machine Translation

• Neural Approaches to Conversational AI

• Neural Response Generation via GAN with an Approximate Embedding Layer

• Task-oriented Conversational Agent Self-learning Based on Sentiment Analysis

• Deep Reinforcement Learning for Dialogue Generation

• Topic Aware Neural Response Generation

• Response Selection with Topic Clues for Retrieval-based Chatbots

• Bidirectional Recurrent Neural Networks as Generative Models

• Adversarial-Learning-for-Generative-Conversational-Agents

• Few-Shot Generalization Across Dialogue Tasks

• Neural Networks for Text Correction and Completion in Keyboard Decoding