Proceedings of the 2nd International Conference on Inventive Communication and Computational Technologies (ICICCT 2018)

IEEE Xplore Compliant - Part Number: CFP18BAC-ART; ISBN:978-1-5386-1974-2

Sentiment Analysis using Neural Networks: A New

Approach
Shiv Dhar Suyog Pednekar Kishan Borad Prof. Ashwini Save
Computer Engineering Computer Engineering Computer Engineering Head of Department
VIVA Institute of VIVA Institute of VIVA Institute of Computer Engineering
Technology Technology Technology VIVA Institute of Technology
Mumbai, India Mumbai, India Mumbai, India Mumbai, India
[email protected] [email protected] [email protected] [email protected]

Abstract—: Sentiment analysis is a trending topic, as everyone

has a unique perception about everything. The systematic
research of these opinions can gain some information which can
prove to be valuable for many companies and industries in
future. A huge number of users are online, and they share their
opinions and comments regularly; this information can be mined
and used efficiently. Using sentiment analysis, various companies
can review their own products and make the necessary changes
in the future. The data is huge and thus it requires efficient
processing to collect this data and analyze it to produce required
result. Sentiment analysis can be performed using various
techniques including Convolutional Neural Network. The
combination of sentiment analysis and Morphological Sentence
Pattern Model sums up to obtain better results. The output Fig. 1: Sentiment analysis process [16]
obtained from MSP model plays a major role in sentiment
analysis and also helps in improving the performance. This information is highly valuable to commercial
organizations; however, the volume of such reviews is growing
Keywords—CNN, Sentiment Analysis, Neural Network, twitter,
MSP, POS tagging.
rapidly, necessitating an automated approach to extracting
meaning from the high number of data. This automated
I. INTRODUCTION approach is provided by sentiment analysis.
Sentiment analysis tries to identify the attitude of an orator, Convolutional Neural Network is a system which is also
writer, or other substance with respect to some topic or the known as class of deep, feed-forward artificial neural networks
overall circumstantial division or emotional response to a file, that has positively been functional for numerous analyses.
communication, or event. It generally uses natural language CNN admits input in the form of single words which requires
processing, computational semantics, text analysis, and extra time and efforts. Morphological Sentence Pattern model
biometrics to systematically recognize, extract, calculate, and identifies the features and expressions of sentences and helps in
study affecting states and subjective data. Sentiment analysis forming shorter patterns. A combination of MSP model and
is widely functional to “voice of the customer” materials like CNN provides improved sentiment analysis.
survey responses and analyses, and also to online and social II. RELATED WORK
media. Sentiment analysis stands at the intersection of natural
language processing and large-scale data mining. Sentiment Analysis is a trending topic and can be very
Sentiment analysis has substantial applications in academia useful for various companies and organizations. In the past,
and also commerce. Sentiment analysis has entitlements in a there were many techniques used for sentiment analysis such
variation of territories, ranging from marketing to customer as the lexicon based approach [2], SVM [10], co-reference
service to scientific medicine. The understanding of human resolution [4] and streaming API [8]. These methods had their
language is a core problem in AI research. At the same time, share of outcome and the result also depended on the data set
with increasingly lowering barriers to the Internet, it is easier used for sentiment analysis. Online shopping is trending these
than ever for end-users to provide feedback on the products days as it’s found to be reliable and secure. People buy
and services they use. products online and post their reviews about them. These are
in the form of tweets or product reviews. It is challenging to
physically read these reviews and assign sentiment to them.

978-1-5386-1974-2/18/$31.00 ©2018 IEEE 1220

 Proceedings of the 2nd International Conference on Inventive Communication and Computational Technologies (ICICCT 2018)
IEEE Xplore Compliant - Part Number: CFP18BAC-ART; ISBN:978-1-5386-1974-2

So, for all these tweets an automated system can be created including (POS) of Speech tagger, the co-reference
which will analysis the review and extract the user percepts. resolution structure, the Named Entity Recognizer (NER)
Duncan and Y. Zhang proposed the neural network approach and many more. In the suggested structure, Stanford Core
for sentiment analysis on twitter [7] to develop neural NLP is used for classification of the input in the forms of
networks which can focus on providing polar views by aspects and expressions.
dividing the opinions in social media and websites having
customer reviews. It collects the reviews from websites and
divides them into positive, negative and neutral reviews. The 2. Part of Speech (POS) Tagging:
system used feed forward neural network. The neural network POS tagging [1] is a significant facet of MSP model
used is on MATLAB, using neural network toolbox. It reduces as it separates the sentence into aspects and expressions.
the input by removing the punctuations, single characters, stop POS tagging is completed using Stanford Core NLP. It is
words like and, to, the etc. and also mentions of other users a blend of tools required for POS tagging. Here, every
using the “@” symbol. word in the sentence is given a label according to its
grammatical context.
III. PROPOSED SYSTEM
Using the process flow diagram, we can explain the
working of this system. In the proposed system, the input
which consists of Twitter tweets is given to the aspect
expression extractor. This extracts pairs of aspect and
expressions from the supplied reviews. The sentiment pattern
extractor builds the entire potential morphological sentence Fig. 3: POS tagging using Stanford core NLP [15]
pattern using the generated pairs. These candidate patterns are
passed as input to the convolutional neural network which has In Stanford Core NLP, PRP denotes Preposition,
been pre-trained on an existing corpus. The output is the VBZ denotes Verb, NN denotes Noun, DT denotes
sentiment polarity of the input review. Determiner and according to the word in the corpus,
several other tags are assigned.

3. Aspect-Expression Pair:
The nouns in the sentence are termed as aspects and
the corresponding adjective in the sentence is termed its
expression. These aspects and expressions are then
collected together and a pair is formed titled as aspect-
expression pair as presented in the figure 4.

Fig. 2: System Flow

A. Morphological Sentence Pattern Model

1. Stanford Core NLP:

Stanford Core NLP [1] is a assortment of human Fig. 4: Aspect Expression Extractor [1]
language tools. It can breakdown a sentence into its
B. Keras
minuscule form. It also helps in naming the words with
their corresponding tags i.e. adjective, verb, noun etc. Keras written in Python, is an open source neural
Stanford Core NLP enfolds several other study tools network library. It is essentially a clean front-end to many

978-1-5386-1974-2/18/$31.00 ©2018 IEEE 1221

 Proceedings of the 2nd International Conference on Inventive Communication and Computational Technologies (ICICCT 2018)
IEEE Xplore Compliant - Part Number: CFP18BAC-ART; ISBN:978-1-5386-1974-2

other lower-level libraries including MXNet, F. System Architecture

Deeplearning4j, TensorFlow, CNTK or Theano. It is Architecture of Sentiment Analysis using CNN with
designed to be a clean interface that allows researchers to MSP model is depicted in figure 5:
describe a high-level neural network model that is then
translated to the above back-ends, which do the heavy
lifting. Keras is used to specify the neural network used in
the suggested structure.

C. Tensorflow
TensorFlow is an open-source software library for
machine learning through a chain of tasks. It is a structure
for building and training neural networks to discover and
decrypt designs and links. TensorFlow was established by
Google, and was later open-sourced. It is still actively
maintained and is a second-generation system,
implementing the insights gleaned from the development
of its predecessors. This is the neural network
implementation that is used in the suggested structure.
D. Convolutional Neural Network
Convolutional neural networks are feed-forward
neural networks that are well-suited for image processing
and also for natural language processing. Originally
inspired by neuron organization in the visual cortex, they
have been very successful at both image recognition and
text analysis tasks. A CNN consists of an input and output
layer, with a certain number of hidden layers in between.
These hidden layers can be one or more of convolutionalal
layers, fully connected layers, normalization layers, or
pooling layers. CNNs are noted for needing relatively little
pre-processing in comparison to other algorithms, inferring
instead the features that would be hand-filtered through to
the algorithm. Convolutional neural network is made up of
neurons [6]. These neurons can have learnable values and
weights. Every neuron receives an input and works in a
flow.
Fig. 5: System Architecture
In this system, convolutional neural network is
trained with curtailed input of twitter tweets. Here, use of
conventional computers can be predictable as it has a fixed G. Stepwise Procedure of proposed system
way to find a solution but using neural network allows it to Step 1: Extraction of tweets from dataset
learn and generate results according to the ever changing
input [17]. Convolutional neural network is a good fit for a) Tweets are mined from the twitter dataset which
this system as it learns from the training set and can even includes 10,00,00 tweets.
produce results which is complex for humans to deduce. b) Random tweets are mined every time extraction is
E. Dataset processed.
In the proposed system, dataset [18] is used. The Step 2: Filtration and POS tagging
original dataset consisted of ~1.57 million tweets; only the
first 1,00,000 were used in the interest of keeping training a) Filtration is done by removing # tags, URLs,
time reasonable. It comprises mainly of tweets from the unwanted stop words like the, in, or etc.
Sentiment140 dataset, plus tweets from other assorted b) This is done using POS tagging, which labels each
sources. Here, random amount of tweets are positive and word in the text.
negative. 80% of tweets i.e. 80000 tweets are used for
training and 20% i.e. 20000 tweets are used for testing.

978-1-5386-1974-2/18/$31.00 ©2018 IEEE 1222

 Proceedings of the 2nd International Conference on Inventive Communication and Computational Technologies (ICICCT 2018)
IEEE Xplore Compliant - Part Number: CFP18BAC-ART; ISBN:978-1-5386-1974-2

Step 3: Extraction of Aspect and Expression 80 74.15 74.66

70 64.69
a) After POS tagging, the entire text is labeled 60
according to their grammatical structure. The Aspects 50
and Expressions are separated from the entire text. 40
b) Once the aspects and expressions are extracted, pair 30
20
of aspects and expressions are formed. It is called as
aspect expression pair. Accuracy without Accuracy without Accuracy with MSP
MSP model pre- MSP model pre- model pre-
c) The original MSP model was used for aspect-based
processing processing processing
sentiment analysis. Here, the model is adapted for
(Twitter Dataset A) (Twitter Dataset B) (Twitter Dataset B)
conventional sentiment analysis. Instead of taking all
the input between the aspect and the expression, only Accuracy %
the nouns and adjectives (potential aspects and
expressions) are used. Fig. 6: Comparison between accuracies before and after MSP model
inclusion
Step 4: Training Neural Network
V. CONCLUSION

a) Neural network is skilled with the given dataset by In the present research, a technique to trim textual input to
forming a layered array. a convolutional neural network based on the morphological
b) The Neural network is trained using 80% of the given sentence pattern (MSP) model is presented. The focus is more
dataset. The remaining 20% tweets are then used for on general sentiment prediction rather than aspect-based
testing. prediction, and adapts the model accordingly. By trimming the
c) The GloVe embedding’s (25-dimensional) for input to the ‘most important’ words, sentiment-wise, the
Twitter are used to transform the words into matrix effectively preprocessed data is passed to the CNN for
form. training. This approach therefore can be considered to
Step 5: Checking the Polarity combine knowledge of the morphological structure of
sentences, with the inferring power of CNNs. In future work,
a) After passing through various layers of neural exploring the effect of various other combinations of part-of-
network, the value of each neuron is varied. At the speech tags and other grammatical structures, on prediction
end of all layers, the network reshapes and provides accuracy can be done.
with a value.
b) Once the entire process is completed, the ACKNOWLEDGMENT
probabilistic value is calculated for the given We would like to express a profound sense of gratitude
sentence. towards Prof. Tatwadarshi P. N., Department of Computer
c) The probability is then checked, if value is 0.5 or Engineering for his constant encouragement and valuable
above, it is denoted as positive or if it is less than 0.5 suggestions. The work that we have been able to present is
it is denoted as negative. possible because of his timely guidance and support.

IV. RESULTS AND DISCUSSIONS REFERENCES

The result of the system, sentiment analysis using CNN [1] Y. Han and K. K. Kim, "Sentiment analysis on social media using
contains the sentiment of the given sentence. The system morphological sentence pattern model," 2017 IEEE 15th International
accepts the input in the form of twitter tweets. These tweets Conference on Software Engineering Research, Management and
Applications (SERA), London, 2017, pp. 79-84.
are passed on to the MSP model, therefore forming aspect and
[2] Z. Nasim, Q. Rajput and S. Haider, "Sentiment analysis of student
expression pairs. The results of the suggested structure include feedback using machine learning and lexicon based approaches," 2017
the tweet from the twitter which is extracted from the dataset. International Conference on Research and Innovation in Information
It also displays the labeled sentence after the POS tagging is Systems (ICRIIS), Langkawi, 2017, pp. 1-6.
completed. The corresponding label is shown along with the [3] Z. Xiangyu, L. Hong and W. Lihong, "A context-based regularization
method for short-text sentiment analysis," 2017 International
entire sentence. Conference on Service Systems and Service Management, Dalian, 2017,
From the studied literature, it was established that the pp. 1-6.
average accuracy obtained earlier was around 74.15% [7]. The [4] M. H. Krishna, K. Rahamathulla and A. Akbar, "A feature based
accuracy obtained once the neural network was trained but approach for sentiment analysis using SVM and coreference
without the use of MSP model is 64.69 % on an average. Once resolution," 2017 International Conference on Inventive Communication
and Computational Technologies (ICICCT), Coimbatore, 2017, pp. 397-
the MSP model is applied and the system is trained, the 399.
accuracy obtained is 74.66%. The comparison can be seen in [5] P. Yadav and D. Pandya, "SentiReview: Sentiment analysis based on
fig. 7. text and emoticons," 2017 International Conference on Innovative
Mechanisms for Industry Applications (ICIMIA), Bangalore, 2017, pp.
467-472.

978-1-5386-1974-2/18/$31.00 ©2018 IEEE 1223

 Proceedings of the 2nd International Conference on Inventive Communication and Computational Technologies (ICICCT 2018)
IEEE Xplore Compliant - Part Number: CFP18BAC-ART; ISBN:978-1-5386-1974-2

[6] Y. Gao, W. Rong, Y. Shen and Z. Xiong, "Convolutional Neural [12] R. Hegde and Seema S., "Aspect based feature extraction and sentiment
Network based sentiment analysis using Adaboost combination," 2016 classification of review data sets using Incremental machine learning
International Joint Conference on Neural Networks (IJCNN), algorithm," 2017 Third International Conference on Advances in
Vancouver, BC, 2016, pp. 1333-1338. Electrical, Electronics, Information, Communication and Bio-
[7] B. Duncan and Y. Zhang, "Neural networks for sentiment analysis on Informatics (AEEICB), Chennai, 2017, pp. 122-125.
Twitter," 2015 IEEE 14th International Conference on Cognitive [13] K. Han, K. Youngsub, Y. Kim, and Jin-Hee Song, "Building Sentiment
Informatics & Cognitive Computing (ICCI*CC), Beijing, 2015, pp. 275- Lexicon for Social Media Analysis using Morphological Sentence
278. Pattern Model." Advanced Science and Technology Letters 136 (2016),
[8] M. Trupthi, S. Pabboju and G. Narasimha, "Sentiment Analysis on pp. 103-106.
Twitter Using Streaming API," 2017 IEEE 7th International Advance [14] S. Jebbara and P. Cimiano, “Aspect-Based Sentiment Analysis Using a
Computing Conference (IACC), Hyderabad, 2017, pp. 915-919. Two-Step Neural Network Architecture,” Communications in Computer
[9] K. Liu, Y. Niu, J. Yang, J. Wang and D. Zhang, "Product Related and Information Science, vol 641, 2016.
Information Sentiment-Content Analysis Based on Convolutional [15] “http://nlpforhackers.io/training-pos-tagger” last accesed on 10/01/2018.
Neural Networks for the Chinese Micro-Blog," 2016 International [16] “https://pythonspot.com/python-sentiment-analysis” last accessed on
Conference on Network and Information Systems for Computers 21/02/2018.
(ICNISC), Wuhan, 2016, pp. 357-361
[17] “https://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11/report.htm
[10] D. V. N. Devi, C. K. Kumar and S. Prasad, "A Feature Based Approach l#Introduction” last accessed on 26/02/2018.
for Sentiment Analysis by Using Support Vector Machine," 2016 IEEE
6th International Conference on Advanced Computing (IACC), [18] “http://thinknook.com/wp-content/uploads/2012/09/Sentiment-Analysis-
Bhimavaram, 2016, pp. 3-8. Dataset.zip” last accessed on 25/03/2018.
[11] E. Anwar, E. Akib & T Ekram, et al. (2015). Localized twitter opinion
mining using sentiment analysis. Decision Analytics.

978-1-5386-1974-2/18/$31.00 ©2018 IEEE 1224