9 III March 2021

https://doi.org/10.22214/ijraset.2021.33390
 International Journal for Research in Applied Science & Engineering Technology (IJRASET)
ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.429
Volume 9 Issue III Mar 2021- Available at www.ijraset.com

Sentimental Analysis of Product Review Data using

Deep Learning
T. Sureshkumar1, S. Asokan2, G. Dharnish3, R. Prakash4, K. Shankar5
1 2, 3, 4, 5
Assistant Professor, UG Students – Final Year, Department of Information Technology, Nandha College of Technology,
Perundurai, Tamilnadu, India

Abstract: A significant undertaking of Natural Language Processing is sentimental analysis. It can be used to identify an
individual’s expression towards something. In this paper, we aim to tackle the problem of sentiment polarity categorization,
which is one of the fundamental problems of sentiment analysis. A very important issue in sentimental analysis is to estimate the
extremity arrangement that decides if a review is positive, negative or unbiased. A general process for sentiment polarity
categorization is proposed with detailed itemized measure portrayals. Information utilized in this study are online item audits
gathered from Amazon.com. Experiments for both sentence-level categorization and review-level categorization are performed
with promising outcomes. Sentimental categorization has gained much traction in recent times.
Keywords: Sentimental analysis, natural language processing, polarity categorization, product reviews, feature vector.

I. INTRODUCTION
Sentiment analysis is often used to derive the emotion / opinion expressed in a text. It has wide applications, including analysis of
product reviews, discovering a brand’s presence online and people’s opinion on a subject online. However, it is hard to implement
sentiment analysis by machine learning because of the nature of human language, such as negation, metaphors, and multiple
sentiments in languages. The goal of this project is to conduct sentiment analysis on Amazon product reviews using Natural
Language Processing (NLP) techniques, Bag of Words (BoW) model, Word2Vec model and Long Short Term Memory (LSTM)
neural network. The dataset consists of 400 thousand reviews of unlocked mobile phones sold on Amazon.com which is publicly
available on Kaggle. Solution to the problem would be useful for a brand to gain a broad sense of user's sentiment towards a product
through online reviews.

A. Deep Learning
Deep learning can be considered as a subset of machine learning. It is a field that is based on learning and improving on its own by
examining computer algorithms. While machine learning uses simpler concepts, deep learning works with artificial neural networks,
which are designed to imitate how humans think and learn. Until recently, neural networks were limited by computing power and
thus were limited in complexity.
However, advancements in Big Data analytics have permitted larger, sophisticated neural networks, allowing computers to observe,
learn, and react to complex situations faster than humans. Deep learning has aided image classification, language translation, speech
recognition. It can be used to solve any pattern recognition problem and without human intervention. Artificial neural networks,
comprising many layers, drive deep learning. Deep Neural Networks (DNNs) are such types of networks where each layer can
perform complex operations such as representation and abstraction that make sense of images, sound, and text. Considered the
fastest-growing field in machine learning, deep learning represents a truly disruptive digital technology, and it is being used by
increasingly more companies to create new business models.

B. Recurrent Neural Network

Recurrent Neural Network (RNN), which is a supervised model, is one such kind of artificial neural network that is primarily
intended to identify patterns in data sequences, such as text, genomes, handwriting, the spoken word, and numerical time’s series
data emanating from sensors, stock markets, and government agencies. Recurrent Neural Networks are often used for problems that
involve sequential data .They are considered as more powerful algorithms because of their ability to remember and store data. They
can understand a sequence much better than any of the other algorithms. Recurrent Neural Networks are created from a class of
networks known as the feedforward networks. They exhibit similar properties to the human brain when it comes to their functioning.

©IJRASET: All Rights are Reserved 1028

 International Journal for Research in Applied Science & Engineering Technology (IJRASET)
ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.429
Volume 9 Issue III Mar 2021- Available at www.ijraset.com

C. Long Short Term Memory (LSTM)

LSTMs are a branch of recurrent neural networks, which have the ability to remember both the past and present data provided to
them. LSTMs are explicitly designed to avoid the long-term dependency problem. Remembering information for long periods of
time is practically their default behavior, not something they struggle to learn. In LSTM, the memory is in the form of a gated cell
that decides whether to store or delete the data based on the importance given to the data. This gated cell can be classified into three
types based on the operation it performs, namely- Input gate, Forget gate and Output gate. These gates are analog in nature and
hence they can perform back propagation at any point in time.

D. Word2Vec
Word2vec is a combination of models used to represent distributed representations of words in a corpus C. Word2Vec (W2V) is an
algorithm that accepts text corpus as an input and outputs a vector representation for each word.
1) Related Work: Xing Fang and Justin Zhan et al., 1. The goal of the authors is to solve the problems in identifying the sentiment
polarity of a statement using reviews of unlocked mobile phones from Amazon. Liu B et al., 2. The authors express that
opinions are usually subjective expressions that describe people’s sentiments, appraisals or feelings towards entities, events and
their properties. Pak A, Paroubek P et al., 3. The authors performed linguistic analysis of the collected corpus and explain
discovered phenomena. Using the corpus, they build a sentiment classifier, which is able to determine positive, negative and
neutral sentiments for a document. Pang B, Lee L et al., 4. Sentiment analysis seeks to identify the viewpoint(s) underlying a
text span. To determine this sentiment polarity, the authors propose a novel machine-learning method that applies text-
categorization techniques to just the subjective portions of the document. Turney PD et al., 5. The classification of a review is
predicted by the average semantic orientation of the phrases in the review that contain adjectives or adverbs. A phrase has a
positive semantic orientation when it has good associations and a negative semantic orientation when it has bad associations. In
this paper, the semantic orientation of a phrase is calculated as the mutual information between the given phrase and word.
Whitelaw C, Garg N, Argamon S et al., 6. The authors used fine-grained semantic distinctions in features used for classification.
Jindal N, Liu B et al., 7. The authors say that the existing research has been focused on classification and summarization of
opinions using natural language processing and data mining techniques. An important issue that has been neglected so far is
opinion spam or trustworthiness of online opinions.
2) Proposed Methodology: The existing Machine learning model fails to provide accurate results when working with large
amounts of datasets such as Text and Image data .This machine learning model may not work well for the reviews that contain
purely implicit statements. Hence in order to solve this issue, we are going for deep learning techniques as they are more
suitable when it comes to working with larger datasets. By using deep learning model, organizations can track the perception of
the brand by the customers, point out the specific details about the attitude, find patterns and trends, Categorize urgency of
mentions according to the relevancy scoring (i.e., which platform, type of user is vital to the brand), Monitor mentions or
reviews of the brand on different platforms (blogs, social media, review sites, forums, etc.)
3) Preprocessing: To prepare the data, reviews with ratings greater than 3 will be labeled and encoded as “1” (positive sentiment)
and reviews with ratings less than 3 will be labeled and encoded as “0” (negative sentiment). Neutral reviews (rating=3) will not
be used. Since it is a huge data set, the labeled data have been split into training set and validation set in 90/10 so that the model
could be trained on more data to improve accuracy. Text preprocessing is needed to convert raw reviews into cleaned review.
Necessary steps include conversion to lowercase, removal of non-characters, removal of stop words, and removal of html tags.
Depending on machine learning algorithms or deep architectures used, cleaned text data are further transformed to suitable
numerical representations, such as 1D feature vector for supervised learning algorithms or 2D tensor for LSTM. Bag of Words
(BOW) is a method to extract features from text documents. These features can be used for training machine learning
algorithms. It creates a vocabulary of all the unique words occurring in all the documents in the training set. In simple terms, it’s
a collection of words to represent a sentence with word count and mostly disregarding the order in which they appear.
Preprocess raw reviews to cleaned reviews, Create BoW using Count Vectorizer / Tfidf vectorizer in sklearn, transform review
text to numerical representations, fit feature vectors to supervised learning algorithm using Multinomial NB and Logistic
Regression in sklearn ,build a pipeline of vectorizer and classifier and improve the model performance by Grid Search in
sklearn.
4) Word2vec: Parse review text to sentences (Word2Vec model takes a list of sentences as inputs), create vocabulary list using
Word2Vec model in genism, Transform each review into numerical representation by computing average feature vectors of
words therein, and Fit the average feature vectors to Random Forest Classifier in sklearn.

©IJRASET: All Rights are Reserved 1029

 International Journal for Research in Applied Science & Engineering Technology (IJRASET)
ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.429
Volume 9 Issue III Mar 2021- Available at www.ijraset.com

5) LSTM: For simple LSTM, prepare text data to 2D tensor in Keras, train a simple LSTM network consisting of an embedding
layer, a LSTM layer and a dense layer, compile and fit the model using log loss function and ADAM optimizer. For LSTM with
Word2Vec Embedding, load pre-trained Word2Vec model, construct embedding layer using embedding matrix as weights, train
a LSTM with Word2Vec embedding, compile and fit the model using log loss function and ADAM optimizer.

II. EXPERIMENTAL RESULTS

The table given below summarizes the accuracy score on validation set of various models trained on 1% and 10% of the training
data respectively. LSTM is proven to be very useful in many text classification problems, whereas Word2Vec embedding provides
additional information on the semantic similarity of words. Thus, the final model of LSTM with Word2Vec embedding is expected
to outperform other models. The results show that the final model has better performance than the benchmark model, but it does not
outperform every models (Tfidf Vectorizer with Logistic Regression has better performance). One possible reason is that only 10%
of training data to is used. It is expected that deep neural network like LSTM could deliver better performance by training on a
larger data set. Second is that no hyper parameter tuning is used for LSTM. To further enhance the performance, we could conduct
random search to find the best parameter set.
Table 1- Experimental results
Trained on 1% of Trained on 10% of
Model
Training data training
Count Vectorizer + Multinomial Naive Bayes 88.35% 91.84%
Tfidf Vectorizer+ Logistic Regression 89.32% 93.10%
Tfidf Vectorizer + Logistic Regression 91.59% 95.63%
(best parameter set by Grid Search)
Word2Vec Embedding + Random Forest Classifier 84.79% 92.26%
Simple LSTM 90.61% 94.14%
Word2Vec Embedding + LSTM 88.35% 94.40%

III. CONCLUSION
Word clouds have been created for positive sentiment reviews and negative sentiment reviews of a selected brand, to get an intuition
of words that frequently appear in different sentiments. For example, reviews with positive sentiment on Apple’s products
frequently contain words such as “good”, “great”, “phone”, whereas reviews with negative sentiment frequently contain words such
as “battery”, “unlocked”, “n’t”. Tfidf Vectorizer has been implemented with Logistic Regression and the results have been
investigated. Top 10 features with smallest coefficients : are 'not', 'return', 'disappointed', 'waste', 'horrible', 'worst', 'poor', 'slow’,
'stopped', 'doesn't. The Top 10 features with largest coefficients are 'great', 'love', 'excellent', 'perfect', 'good', 'easy', 'best', 'far',
'amazing', and 'awesome'. This visualization helps us to justify a bit about how logistic regression learns to classify positive and
negative sentiment in this setting.

REFERENCES
[1] Xing Fang, Justin Zhan –“Sentimental Analysis using Product review data”, 2015.
[2] Liu B – “Sentimental Analysis and Subjectivity”, 2010.
[3] Pak A, Paroubek P –“Twitter as a corpus for Sentimental Analysis and Opinion mining”, Jan 2010.
[4] Pang B, Lee L – “Opinion Mining and Sentimental Analysis”, 2008.
[5] Turney PD –“Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews’, 2002.
[6] Whitelaw C, Garg N, Argamon S- “Using Appraisal Groups for Sentimental Analysis”,2005.
[7] Jindal N, Liu B- “Opinion Spam and Analysis”, 2008.
[8] Data source from kaggle https://www.kaggle.com/PromptCloudHQ/amazon-reviews-unlocked-mobile-phones
[9] “Deep Learning with Word2Vec” from Gensim, https://radimrehurek.com/gensim/models/word2vec.html
[10] ”Deep Learning, NLP, and Representations” http://colah.github.io/posts/2014-07-NLP-RNNs-Representations

©IJRASET: All Rights are Reserved 1030