© APR 2018 | IRE Journals | Volume 1 Issue 10 | ISSN: 2456-8880

Sentiment Analysis on Movie Reviews using Recurrent

Neural Network
SUMESH KUAMR NAIR 1, RAVINDRA SONI2
1,2
Department of Computer Science and Engineering, Poornima College of Engineering

Abstract -- In this paper i have done sentiment analysis on I. INTRODUCTION

IMDB dataset using Recurrent Neural network. Sentiment
analysis based on text mining or opinion mining based on
Today extensive datasets are accessible on-line,
different dataset. Sentiment classification is done in three
holding text data or numerical. It has been the major
categories- Positive, Negative and Neutral. Text
classification is done on the dataset and data preprocessing focus for many practitioners and researchers to apply
is done to remove hastags, synonms, acronyms etc. LSTM reasonable approaches and techniques and extract
Recurrent Neural Networks to other algorithms for useful information from those datasets. Wide range of
classifying the sentiment of movie reviews. Recurrent techniques have been proposed and tested to retrieve
neural network provides high accuracy and polarity as information during this time. In addition to text mining
compared to different machine learning classifiers. To and data mining, lately interest for non-topical text
address this task deep learning has become popular analysis have increased drastically and sentiment
method. LSTM (Long short-term memory) model has been
analysis is part of them.
used which is a modified version of RNN (Recurrent
Neural Networks). Recurrent Neural Networks has ability
Sentiment analysis is a process of analyzing the given
handle sequential data very effectively and without
text in order to find out the emotions in it. Sentiment
performing any feature engineering it can learn directly
from low-level features. Instead of exploring LSTMs analysis is about “Text analysis, Information
abilities and capabilities, main focus was to learn how Extraction and Natural Language Processing are kind
embedding can help us to understand user expectations of tasks which aim towards getting the writer’s
from text. Proper pre- processing for data has been feelings expressed in negative or positive comments
implemented. Informal language, contextualization, bad by analyzing sentences or documents” defined by
grammatical structure, misspellings are additional Subhabrata Mukherjee. In simple words, opinion
complicating factors. Reviews are analyzed as binary mining is a process of detecting the sentiment of the
classification task, after processing reviews are classified
writer concerning a particular topic. It is a blend of
as either negative or positive. Features for training and
techniques and strategies about distinguishing and
testing the deep learning model were retrieved by using new
method called ‘word-vector’. Moreover, effect of sentence
detecting subjective information from a text such as
length has also been investigated. Sentiment analysis for opinions and attitudes. Usually, it has been about
short sentences becomes difficult because of lack of opinion polarity to find out whether someone has
contextual information. Multiple hidden layers have been negative, positive or neutral opinion about something.
used in the architecture. Dropout, Normalization and
Parametric Rectified Linear Unit (PReLU) technology has Internet is a huge source of information for every
been used to generalize and improve the accuracy of model. individual. For instance, investors want to be updated
Also, the impact of various hyper-parameter has analysed. with financial news particularly associated with their
Different neural network configurations are evaluated. investment. Organizations are looking for the news
The performance of model is discussed with respect to the about competitors, suppliers and customer’s feedback.
input data and model configuration.
This is a cyclic chain, likewise customers have interest
in reviews of other customers about the products they
Index Terms- Sentiment analysis;PReLU;LSTM;RNN are looking for. Researcher made an excessive amount
of effort to identify the impacts of this technique on
customer insight, trend and financial world. Numerous
applications of sentiment analysis came up in multiple
domains like sentiment analysis of products reviews,

IRE 1700620 ICONIC RESEARCH AND ENGINEERING JOURNALS 242

 © APR 2018 | IRE Journals | Volume 1 Issue 10 | ISSN: 2456-8880

financial news and healthcare. It additionally offers Treat sentiment analysis as classification problem,
them a superior picture of how they stack up against extract features. Train model to determine sentiment.
their competitors.
• Using Rules:
Technically, sentiment analysis is a unique blend of
artificial intelligence and machine learning, allowing Look for presence of specific words in sentence and
organizations to use advanced tools to choose useful define rules based on those words and categorize
and reasonable moves that attract consumers toward sentences.
their services and products. In order to retain
IIn following sections, old approaches for
customers, competitors have to track and monitor the
classification of sentiment analysis and difficulties in
interest of customers. Especially, not towards their
sentiment analysis has explained.
own products and brands only but also towards their
competitors. 2.1 Sentiment Analysis Difficulties
Machine learning have seen rapid change in previous Research demonstrates that the task of sentiment
two years with significant breakthroughs in deep- analysis is more tough than conventional topic based
learning approaches. Deep neural networks enlivened classification of text, regardless of the fact that we
by the human brain architecture and with enough don’t have much classes in SA than classes in topic
processing power these models have been shown based classification [10]. In this task, usually
unbelievable results on many complex problems classification assigned to the text are generally
including Natural Language Processing tasks, even positive or negative. There can be some different
without having excessive domain knowledge. Out binary classes also or multivalued classification. For
there many neural networks are available with their example, neutral, negative or positive, yet those
classic abilities like Deep Belief Networks (DBN) classes are not as much as in topic based classification.
with fast inferencing of the model parameters, Topic based classification is a bit easier than sentiment
Convolutional neural networks (CNN), and Recurrent analysis because this can be achieved with the use of
neutral network (RNN). In this work, I will work with keywords this could be a reason. On the other hand,
LSTM (Long Short- Term Memory). LSTM networks this technique doesn’t perform well with sentiment
are a type of RNN that uses special units in addition to analysis [11].
standard units.
Classification in sentiment analysis is a subjective
II. RELATED RESEARCH method but there could be variations in opinions if
there are number of observers to test. Interpreting the
There has been considerable research done and still
state of mind of a subject may differ person to person
going on sentiment analysis subject. Sentiment
and if someone has only 140 characters or less to
analysis became part of research at the beginning of
express something then its significantly hard to
20th century and in 1990, text subjectivity analysis
determine the mood [12].
was performed by computational linguistic
community [1]. Most of the sentiment analysers used The study of sentiment classification and subjectivity
to work by choosing one or hybrid of following classification is required to perform sentiment analysis
approaches. properly. In subjectivity classification, it is identified
that whether or not provided text data contains
• Using Vocabulary:
opinionated information or factual information.
Worked by choosing the important keywords (usually Similarly, sentiment analysis is a process to classify an
adjectives and verbs) along with modifiers. For opinion into negative or positive. In reality, if we
example, negative words. consider a product review then it requires in depth
analysis of assigned classification because the
• Machine Learning approach: manufacturer of product is interested to know the
details of opinion, for example owner wants to know

IRE 1700620 ICONIC RESEARCH AND ENGINEERING JOURNALS 243

 © APR 2018 | IRE Journals | Volume 1 Issue 10 | ISSN: 2456-8880

what features of a product have been criticised or

praised. Following is the example of review posted by
a user on a pair of Shoes: Sentimental analysis is classified into multiple
sentimental classification techniques. Two of them are
7) I have rated those shoes 4 stars because such popular, first one is machine learning approach and
a cool pair of shoes, but had a few problems. other is lexicon based approach. There is also third
classification technique which is known as hybrid
8) Order a half size down from your regular fit. approach (show in figure 3) and it uses of the above-
mentioned classifications to optimize the solution
9) Uncomfortable in a few places, but overall
[13]. Following is the introduction of mentioned
not too bad.
approaches.

Machine Learning Approach: This approach

10) Arrived 4 days late (DHL's fault), and had a merely depends on text analysis and classification.
small beige stain next to laces, as well as a black scuff Text analysis is mainly used for business decision
on the white sole. making, for which it require text processing. Initially,
it requires some collection of data to train a model,
11) If I was a collector I would have wanted a which later serve and help in prediction of new set of
refund, but I couldn't be bothered to send them back as data without any sort of labels. Model predicts the
I had already wait long enough. unlabelled records by predicting their labelling class.
Classes are classified as positive, negative or neutral.
12) I would recommend those shoes as they are Machine learning approach is further divided into
light weight. following to learning methods.

2. Supervised Learning

In this situation, the main problem is that what exactly 2. Unsupervised Learning
we want to extract from this review. It’s easy
noticeable that there are variety of opinions provided Lexicon Approach: In this approach, there is a need to
in this example, sentence (6) express a positive review define the dictionary or collection of words and
on shoes while sentence (1) and (3) could be positive phrases with their synonyms and antonyms. Most
or negative. The remaining sentences (4) and (5) are common approaches used in lexicon for the collection
inclined toward negative opinion. Number of opinions of words are following two.
in the sentence by user have some targets on which
user expressed views. Sentence (4) and (6) are based 3. The Dictionary Based
on the feature of shoes such as quality, shape, fitness
4. The Corpus Based
whereas sentence (5) is about delivery, nothing related
to product. This review example helps to understand Hybrid Approach: It is most efficient and
the difficulties and challenges in opinion mining or optimized approach among all, it can identify and
sentiment analysis are directly proportional to deep detect the emotions from a text. Support Vector
understanding and require immense data analysis to Machine Algorithm, which works on a technique to
have precisely analysed opinion. There can be other find the best available linear separator between the
factors which increases the difficulty in sentiment classes, is required to achieve the goal of hybrid
analysis for e.g. text expressed with irony, sarcasm or approach.
negation.

Sentimental analysis can also be classified in terms of

levels and ratings. The classification of these
sentiments is based on opinion polarities (positive,
2.2 Sentimental Analysis Classification

IRE 1700620 ICONIC RESEARCH AND ENGINEERING JOURNALS 244

 © APR 2018 | IRE Journals | Volume 1 Issue 10 | ISSN: 2456-8880

negative or neutral) [14]. Figure 3 illustrates the This level is sentiment analysis faces a lot of criticism
hierarchy and tree structure of classification as because of its way of working. As, it is unrealistic due
discussed earlier. to the fact that there may be many possible opinions in
the text. On the other hand, it is useful in situations
2.2.1 Classification Levels where some reviews or final statement about the
product are required. Other use-case of document level
This sub chapter explains about the sentimental
sentiment is scenario news carries some positive or
analysis classification levels. Following are the levels
negative opinions and these opinions reflect in terms
in which sentimental analysis is classified.
of buy or sell signals.
5. Sentence Level
7. User Level
It is used as first filter of analysis. In this level of
This is not a famous or popular level of sentimental
classification, every single sentence is taken under
analysis but researchers have defined some use-cases
consideration to analyse and express the opinion [15].
for this sort of analysis in a situation where user wants
Sentence level works on the assumption that there
to observe user’s network based on the behaviour of
must be single sentiment against one sentence.
the neighbour users [17, 18].
This presumption is not necessary for all the sentences
8. Aspect Level
in the given collection or document. The most
important thing in this classification is to discriminate It is also known as feature level or phrase level
between biased and non- subjective sentences. Non- analysis. It is different from other classification levels
subjective sentences provide no information in in term of method of evaluation. It analyses in such a
decision making. Contrarily to this, subjective way that first it finds the target and then discover its
sentiment provides opinion and detection of those opinion. Other the other hand, other classification
sentences which contains some facts [16]. Sentence levels focus on languages units such as sentences,
level classification provides help to prevent documents and paragraphs. The aim of this
misleading and selecting irrelevant data or sentences. classification is not to find the opinion of entities but
As a result, it is used to increase the efficiency and also their different aspects.
performance of sentence level sentiment classifier. It
is preferred to use when there is need to have more This analysis can be achieved by differentiating polar
than one opinion in one document. It also provides phrases and defining their sentiments from other [19,
support to treat sentences differently for special 20]. In finance, this level of sentiment is used to find
classification. Best scenario of using Sentence Level the relationship between detected polar words and
classification is on conditional and comparative other variables, example of which is firm earnings and
sentences. It is assumed that there is no single strategy stock prices [21].
available to different sentences or whole text of all the
types. In order to improve the accuracy, using All the models built for product analysis and mining
combination of different strategies is preferred. It is of customer opinions about certain product feature is
also preferred to rate opinions in terms of positive or based on aspect level sentiment [22, 23]. In general, all
negative opinion, not in terms of good or bad opinions. the words or phrases in aspect level sentiment directed
to specified topic or an object.
6. Document Level
Figure 4 depicts the process of extracting aspects of
Main objective of this classification is to find out, reviewed products based on the opinions of customers.
either whole document has positive or negative Feature selection process is performed to extract
opinion. This method considers whole document as a feature and based on those feature, trained models
single entity and that is why is not suitable in situation categorize the review.
where evaluation of more than one entity requires.

IRE 1700620 ICONIC RESEARCH AND ENGINEERING JOURNALS 245

 © APR 2018 | IRE Journals | Volume 1 Issue 10 | ISSN: 2456-8880

against submitted record [25]. Decision tree is popular

because it does not require any configuration
expertise. Some of the packages used for
implementation of decision tree in text classification
problems are ID3 and C5.

Linear Classifiers: These techniques are famous due to

simplicity. The objective of this technique is to find
out number of opinions in provided data and find their
polarity by comparing them with list of pre-defined
words. Weights are added against words in such a
manner, a word with most negative opinion has lowest
weight, on the other hand, word with most positive
opinion have highest weight. Most popular type of
Linear classifiers is Support Vector Machine (SVM)
classifiers. [26]

Figure 2 An approach to convert non-grammatical Rule-Based Classifier: It is same to some

words or phrases extent to technique of decision tree, these techniques
are based on rules and feature space [27]. The main
2.2.2 Classification Techniques distinguish in term of that rule-based classifier allows
overlap in the decision tree [26] whereas, decision tree
In sentiment analysis classification techniques are of
classifier uses hierarchical approach. In this classifier,
core importance and most of work is also done on
rules are generated based on different criteria’s, such
based on it. The main objective of these techniques is
as support and confidence [28].
to separate the positive, negative or neutral opinions in
document [24]. This sub-chapter mainly focuses on Probabilistic Classifier: It is also said to be
sub-division of machine learning approach and generative classifiers as it generates a model against
lexicon-based approach, which are discussed each class [29]. It assumes that every class is a part of
previously in chapter 3.2. model. The most widely used probabilistic classifier is
Naive Bayer Classifier, which is simplest to
Machine learning approach is based on text
implement in any programming language due to the
classification, which is used for forecasting and
fact that it involves simple mathematics [30]. It works
business decision making by automating the
on the principle that each model consists of scattered
processing text. This approach is divided into
set of words, frequency of existed words remains same
supervised and unsupervised learning. In supervised
but not the spot. Naive Bayes uses Bayes Theorem,
learning approach, initially models are trained using
which allows the label to find out the set of features.
classifiers of document. These trained models or
documents have some key features, which have topic Unsupervised learning is used in a situation when it is
related words. Supervised learning is further classified difficult to create a class- labelled document, which
into following. makes it more natural and general as compared to
supervised learning. For that purpose, unsupervised
learning is implemented on collected unlabelled
Decision Tree: It is used for prediction and documents. In case of document clustering analysis,
are used for classification as well. If record is given this learning approach is mainly used because it not
with unlabelled or unclassified class label, then only relies on already defined class labelled training
compared with decision tree, which is traced from the documents. It is different from supervised learning in
root to node and find outs the prediction of class

IRE 1700620 ICONIC RESEARCH AND ENGINEERING JOURNALS 246

 © APR 2018 | IRE Journals | Volume 1 Issue 10 | ISSN: 2456-8880

such a way it learns by observation and pre-defined 4. ReLu Function

models are not submitted to solution. [31]
It’s a very simple activation function. Suppose input is
Another unsupervised approach is Lexicon-based, value X and if X is positive the output will be X
which uses dictionary. This dictionary consists of list otherwise 0. ReLu (rectified linear unit) function is:
words and phrases, mainly synonyms and antonyms
with opinions. Most automated and accepted Function (X) = max (0, X)
sentiment word list used for Lexicon-based approach
In Figure 10 there is a straight line and it looks like it’s
are following.
a linear function but ReLu is non- linear in reality. The
Dictionary-Based Approach: It’s works on range of ReLu is [0, infinity]. Computationally ReLu
the basic principle that several small set of opinion is less expensive than other activation functions
words are collected manually together to transform it because it has simple mathematical operation.
in large collection of text [32]. Every time new word
is found, it is added into existing document and this
cycle repeats until no unique remains. The major
disadvantage of this approach is that it totally depends
on large collection of data and it is not possible to enter
almost each opinionated word manually created
document [33].

Corpus-Based Approach: Main use of this

approach is in scenario where there is need to discover
new sentiment word or text from domain of collection 2.5 Recurrent Neural Networks
in list of already known opinion words and to generate
new sentiment lexicon from other [12]. Downside of This Chapter explains the difference between human
this approach is that it will work efficiently only in computation and memorizing power and neural
case when collection of all the English or any language networks. In addition to that, it also explains how
words are already present in pre- defined document Recurrent Neural Networks (RNNs) are better than
[34]. It is further divided into statically and semantic traditional neural networks. It is a human nature that
approach. no one start thinking about the situation or problem
from the very initial, every now and then. Humans try
There is also third type of sentimental analysis to sort out the efficient solution of a problem
approach, it is known as Hybrid Approach. The basic depending on their previous knowledge and
working principle of this approach is to find out understanding. On the other hand, machine neural
anticipation from a text with or without affecting networks lack this power of taking decision and
associated words. In order to obtain effective analysing situations depending on the past
identification, Support Vector Machine Algorithm are information.
also used in this approach. Some of industries like HP,
are using mixture of Machine Learning and Lexicon- In traditional neural networks, it was difficult for
Based approaches together to create hybrid based machine to memorize the background information.
approach. This problem can be elaborated by an example of a
movie, consider a machine needs to determine what
sort of events could happen next at every scene. It
wasnearly impossible for traditional neural networks
to deal with such sort of situations. Contrarily to this,
Recurrent Neural Networks can handle these issues. In
the past few years, RNNs are used widely and almost

IRE 1700620 ICONIC RESEARCH AND ENGINEERING JOURNALS 247

 © APR 2018 | IRE Journals | Volume 1 Issue 10 | ISSN: 2456-8880

eliminate use of traditional neural networks. They can transmitting the information. Collection of neurons
be used to solve vast variety of problems, which with synapses8 which connects them is called a neural
include, language modelling, image captioning, network. There are three different types of layers in a
speech recognition, translation and so on. neural network:

III. THE RNN-LSTM NETWORK PROCESS • Input Layer:

FOR TEXT ANALYSIS  Input fed to network through this layer.
• Hidden Layer:
 This layer processes the input taken from
input layer and there can be multiple hidden
Long Short-Term Memory network is a special kind
layers.
of recurrent neural network was proposed by
• Output Layer:
Hochreiter & Schmidhuber [6] as an extension to
recurrent neural network. LSTM networks are capable  This layer produces the processed data.
of learning long-term dependencies. It’s the default
Figure 6 illustrates the connection between those
behaviour of LSTM network to remember information
layers. Circle represents the neurons and the line
for a long period.
connecting them represents synapses
A recurrent neural network has a chain of repeating
modules of neural network. shows the structure of
typical RNN. Each block in the figure is called a
module.

Figure 7 Connection between layers in neural network

Input layer:
Following notations will used to understand:-
Input layer presents a pattern to neural network. This
layer only deals with input data. Each neuron in input
layer should represent an independent variable that has
• Neural Network Layer some effect on the output of network [36].

Hidden layer:
• Pointwise Operation
Hidden layer is also a combination of neurons and also
• Vector Transfer has activation function. This layer is also known as
middle layer. The main job of hidden layer is to extract
important features from data fed by previous layers or
• Concatenate
layer. There can be multiple hidden layers in network
depends upon the complexity of problem. For
example, if data can be separated linearly then there is
• Copy
no need to use the hidden layer as activation function
Neural Network can be implemented directly on input layer. If a
problem need complex decisions then we can use more
In terms of computer science, a neural network is an than one hidden layer. It’s not sure that increasing the
artificial nervous system for receiving, processing and number of hidden layers will result in high accuracy.

IRE 1700620 ICONIC RESEARCH AND ENGINEERING JOURNALS 248

 © APR 2018 | IRE Journals | Volume 1 Issue 10 | ISSN: 2456-8880

At some extent accuracy becomes constant or falls if

an extra layer has been added. Number of neurons also
effect the accuracy result. If number of neurons are Scikit-learn is another library which provides a range
less than complexity level of problem then there will of unsupervised and supervised learning algorithms
be few neurons in the network to detect the signal from via a consistent interface in Python. But keras library
complicated data. Similarly, if there are excessive is handier than scikit-Learn. It gives freedom to define
amount of neuron used then over-fitting (explained in our own designed machine learning models, rather
section 4.4.2) may occur. than pre-defined ones. We can run Keras on top of
TensorFlow. As Google is putting efforts in making
Output Layer: TensorFlow the fastest, so this way we can get those
benefits. Currently TensorFlow is a scalable (deep
Output layer collects and produces the results in a way learning) engine in the industry.
that it has been designed to produce. Typically output
layer make predictions for classes. Combination of Keras and TensorFlow3 has been used
for sentiment analysis task. Another package Gensim4
Neural network training means calibrating the weights which has been used for word vector handling. Gensim
and calibration is achieved by repeating forward is a Python library which is designed to extract
propagation and backward propagation. semantic topics from documents. Algorithms used in
gensim library are unsupervised and this library is
designed to process unstructured, raw text data.
IV. THE RESULT AND ANALYSIS Gensim is an exceptionally optimized, yet additionally
very specific, library for doing tasks related to text
DATASET AND FRAMEWORK data. It offers a simple, surprisingly efficient AI-
approach to handle raw texts and it is based on SNN5
This chapter contains the information about the (Shallow Neural Network).
framework and dataset, used to train Deep Learning
models and the text representations, and also for IMDB Dataset
experimentation in this thesis.
This dataset contains a collection of 50,000 polar
Framework movie reviews. Labelled as either negative or positive.
Negative reviews hold fewer stars than five stars
For this sentiment analysis task, Keras1 has been used whereas positive ones were rated with more than six
for modelling the DL2 (deep learning) models. Keras stars. This IMDB data6 has been taken from Stanford
is a programming framework for deep learning and its University. Researchers in Stanford University
written in Python programming language. It is a collected IMDB data and performed sentiment
minimalistic library with a focus on fast analysis on that. They achieved 88.89% accuracy.
experimentation and simplifies the process of building Now the dataset is properly divided, 25,000 for
applications based on deep learning. Keras can run on training and 25,000 for testing.
top of Theano or TensorFlow, both of them allows
running computations over GPU’s. Theano is a library Results:
for fast numerical computation. It’s a compiler for
mathematical computation in Python and was This section presents the results obtained from
developed by MILA group at University of Montreal, sentiment analysis. Experiment was conducted on
Canada. TensorFlow was created by Google to replace IMDB dataset.
Theano. These two libraries are quite similar but
Performance on IMDB Dataset:
TensorFlow has tools to support Reinforcement
learning. Reinforcement learning is a type of machine As IMDB datasets exhibit very balanced distributions
learning which allow machines to automatically among the positive and negative reviews in Figure 31.
determine the ideal behaviour in a specific context. Total number of samples are 50,000 from which

IRE 1700620 ICONIC RESEARCH AND ENGINEERING JOURNALS 249

 © APR 2018 | IRE Journals | Volume 1 Issue 10 | ISSN: 2456-8880

25,000 were positive and negative. This dataset was 1 1

labelled and modified by Stanford university [9] 1 0
1 1
28 Loss is calculated on validation and training and it
0 0
tells, how well the model is performing.

0 0

Model 96 IMDB 250 1400

REFERENCES

[1] Gers, F. A., Schmidhuber, J., & Cummins, F.

(2000). Learning to forget: Continual
prediction with LSTM. Neural computation,
12(10), 2451-2471.
[2] Graves, Alex. Supervised sequence labelling
with re- current neural networks. Vol. 385.
No. of positive and negative reviews in IMDB dataset Springer, 2012.
The bars for very short sentences are high which [3] A. Graves, A. Mohamed, and G. Hinton.
Speech recog- nition with deep recurrent
means dataset contains a lot of small sentences. But neural networks, ICASSP, 2013. Hannun, C.
the frequency of medium sized sentences is very Case, J. Casper, B. Catanzaro, G. Diamos, E.
reasonable as in our case a paragraph containing 20 to Elsen, R. Prenger, S. Satheesh, S. Sengupta,
200 words is good which can help to analyse the A. Coates, and A. Ng, Deep Speech:Scaling
history remembering capability of model. In this data, up end-to-end speech recognition, in
arXiv:1412.5567, 2014.
we have also very huge paragraphs containing max
1400 words but they are less in quantity. [4]. Hochreiter, S., & Schmidhuber, J. (1997).
Long short- term memory. Neural
computation, 9(8), 1735-1780.
[5] Andrew L. Maas, Raymond E. Daly, Peter T.
Pham, Dan Huang, Andrew Y. Ng, and
Christopher Potts. (2011). Learning Word
Vectors for Sentiment Analy- sis. The 49th
Annual Meeting of the Association for
Computational Linguistics (ACL 2011)
[6] T. Mikolov, M. Karafiat, L. Burget, J.
Cernocky, S. Khudanpur. Recurrent neural
network based lan- guage model, In:
Proceedings of Interspeech, 2010.
[7] Pascanu, R. and Bengio, Y. (2012). On the
difficulty of training recurrent neural
networks. Technical Re- port
arXiv:1211.5063, Universite de Montreal
[8] Richard Socher, Alex Perelygin, Jean Wu,
Jason Chuang, Christopher D. Manning,
Andrew Y. Ng, and Christopher Potts. 2013b.
Recursive deep mod- els for semantic
compositionality over a sentiment treebank.
In Proceedings of the 2013 Conference on
Empirical Methods in Natural Language
Processing, pages 1631–1642, Stroudsburg,

IRE 1700620 ICONIC RESEARCH AND ENGINEERING JOURNALS 250

 © APR 2018 | IRE Journals | Volume 1 Issue 10 | ISSN: 2456-8880

PA, October. Asso- ciation for Computational

Linguistics.

IRE 1700620 ICONIC RESEARCH AND ENGINEERING JOURNALS 251