Application of Convolutional Neural Network in

Sentiment Analysis
Hritika Kadam
MSc Computer Science
SVKM's Mithibai College (Autonomous)
Mumbai, India
[email protected]

Jayasree Ravi
Assistant Professor
Dept. of Computer Science
Mumbai,India

Abstract short-term memory), BERT (Bidirectional

Encoder Representations from Transformers),
The current rate development in the Artificial
RNN (Recurrent Neural Networks) which are
Intelligence models have put Sentiment Analysis
some of the frameworks of NLP.
to a forefront. With many AI models being
developed and integrated into applications of In this paper we discuss the implementation of
various fields, the requirement for the appropriate CNN for the purpose of Sentiment Analysis. This
assessment of the emotions is necessary to implementation model is further divide into two
understand and process the data for the end user. parts-Word Embedding, which is required in data
This paper focuses on a Deep Learning approach, pre-processing, and the CNN model itself.
wherein we understand the applications of
Convolutional Neural Network in Sentiment Convolutional Neural Network
Analysis. Here we try to understand the CNNs are famously used for tasks such as image
implementation of CNN models paired with Word recognition and video recognition, medical image
Embedding techniques to achieve accurate results. analysis and computer vision. The CNN models
The paper discusses the different variations of the are designed and trained so that they are able to
CNN models and their rate of accuracy on an recoginze high level spatial features. The
extensive range of emotions. The paper also architecture consists of three main components –
explains the experimentation of a proposed CNN Convolution Layers, to extract the features of the
model on the “GoEmotions’’ dataset. data and create feature maps for the
Keywords: CNN Models, GoEmotions, characteristics, Pooling Layers, to maintain a low
Sentiment Analysis complexity while reducing the parameters and a
fully connected network, that maps the extracted
I. Introduction features in a final output.
With the introduction of LLM like ChatGPT, CNNs are also used for text classification, but
BARD and Copilot, the interest in research in they have limitation when it comes to recognizing
MLP has peaked highly. The end of goal such the semantic relations between texts. So, for the
models is to understand the request of the user, to task of text classification, while the architecture
understand the tonality of it and provide an remains similar an additional component is added
accurate response which is akin to a human-like to the model – Word Embeddings. Word
response. This entire process can be broken down Embeddings help represent the textual data in a
into different categories, such has the coherence of continuous vector space, making it feasible for the
the response, the sensitivity of the response to the model to recognize the features and map them.
user’s tone, how natural the response sounds.
There are various techniques that are used to
develop such models, such as the LSTM (Long
Word Embeddings paper proposes a model that labels the sentences
rather than the common approach of labelling the
Word embeddings are dense numerical
words.
representations of words in a continuous vector
space, where each word is mapped to a high- Zufan Zhang [3] experimented with data of
dimensional vector of real numbers. These different modalities, implementing attention
embeddings are designed to capture semantic mechanism to capture more nuanced
relationships between words and contextual understanding of the semantics. The authors
information within a given corpus of text. By compare three different frameworks for texts
representing words in a continuous space, encoding- Semantic Embedding. Lexicon
embeddings enable algorithms to understand Embedding and Sentiment Embedding. Further
language more effectively by leveraging integrating three different attention mechanisms in
similarities between word vectors. CNN – attention vector, LSTM attention and
Various techniques exist for generating word attentive pooling. The paper states that the
embeddings, such as Word2Vec, GloVe, and combination of lexicon embeddings and the
transformer-based models like BERT and GPT. attention mechanisms yielded the better
Word2Vec learns embeddings by predicting performance on the general sentiment analysis out
surrounding words given a target word or vice of all three.
versa, while GloVe combines global word co-
The paper [4] introduces several approaches,
occurrence statistics with matrix factorization
integrating Lexicon Embedding with the Kim’s
methods. Transformer-based models like BERT
CNN model [17], along with attention mechanism.
and GPT generate contextualized word
While the study applied the attention model to
embeddings by considering bidirectional context
each word resulting in a more robust
or predicting the next word in a sequence.
understanding, it concludes that the applying the
Sentiment Analysis attention model to multiple words at a time is
possible scope for the future.
Sentiment Analysis is a common opinion mining
technique used for feedbacks and reviews, Shiyang Liao [5] proposes a CNN model trained on
providing a range of three categories – Positive, X (Twitter) dataset to gain understanding of
Negative and Neutral. It extracts the subjective sentiments in real world situations. The authors
information form the text to determine the also use datasets or Movie Reviews and STS Gold
emotion. It’s used across a varied range of to train the model.
domains to gain insights on the user’s views. This
The authors [6] through their study emphasize that
paper attempts to understand its applications in
the sizes of filter matrix as well as the number of
setting with wider range of emotions.
convolution layers have a significant impact on
the accuracy of the model. They conclude that
more convolution layers lead to decrease in
II. Related Research
accuracy, similar with the size of the filter matrix,
The authors of this paper [1] propose a new model, where in the matrix of larger size is not capable of
Character to Sentence Convolutional Neural capturing the details of dataset accurately.
Network (CharSCNN) aiming to resolve the issue
Chen and Zhang [7] use a combination of CNN
revolving around the size of the text. The short
along with SVM. Implementing CNN for feature
texts whose length are at most one sentence long,
analysis and SVM for text classification. The
such as posts on X (Twitter), do not provide
pertained word vectors are achieved using
enough context of the information. The study
Word2Vec algorithm.
focuses on how to accurately perform sentiment
analysis on such short texts using the SST dataset. The authors [8] expand the scope dataset by
utilizing the Hindi Movie Reviews. Along with
Xi Ouyang [2] works on the CNN + Word2Vec
implementing the CNN, they compare the results
framework, the overall goal for his research seems
with other machine learning baseline classification
to be enhancing the understanding of context with
models, such as, K-means, Naïve Bayes, SVM,
respect to the user content on social media. The
deeming that properly trained CNNs perform Sentence Searching (CSS) algorithm to extract
better in sentiment analysis than the baseline complex sentences and ABstract Noun Searching
models of Machine Learning. (ABNS) algorithm to identify the abstract nouns
based on the sentence structure.
The paper [9] discusses the development and
application of DGCN (Dialogue Graph The authors [13], have proposed an enhanced
Convolutional Neural Network) used for emotion version of Alexandre Cunha’s model[18]. Working
recognition in conversations. The study aims to with movie review data, they have seven-layer
recognize several emotions such as, happy, sad, CNN network, which consists of a word
neutral, angry, excited, frustrated, disgust, and embedding layer – to represent the words in
fear, present in the conversational data. The vectors, two 1D-CNN layers, a global max-poling
DGCN model has three components - Sequential layer to reduce the dimension of the data, along
Context Encoder, Speaker-Level Context Encoder, with a fully connected network.
and Emotion Classifier, paired along with Kim’s
Sameeksha Khare [14] focuses on enhancing the
CNN model for extracting features in combination
accuracy of the sentiment analysis, using CNN
with GloVe algorithm to represent the vectors of
and ANN techniques. The study is performed on
the data. The model performs better than other
X(Twitter) data, wherein CNN is utilised to
baseline models on the ERC (Emotion
understand the graphical content in the user
Recognition in Conversation) datasets.
comments and ANN is utilised for the textual
Hannah Kim [10] states research for the sentiment content.
classification on longer texts using a CNN model.
The paper [15] focuses on how to characterize
In the study, the binary and ternary classifications
sentences in a comment using CNN models. The
are considered, where in the author claims that the
authors have used the X(Twitter) dataset, and have
network developed by them has a better accuracy
analysed the performance of the CNN models and
than machine learning classification models such
their sentiment accuracy of the extracted tweets.
as Decision Tree, Random Forest, SVM and Naïve
Bayes. This study further open up the scope for Dr. Bhaludra R Nadh Singh and his team [16]
tasks such as gender classification where the present a study that focuses on the significance of
proposed model might give similar results to this the multimodal data in analysing the emotions of
study. the user. They propose a framework that uses
CNN models to analyse the sentiment across
The paper [11] has taken an multi modal approach
different modalities – text, images and videos,
to detect stress. It uses social media data acquired
aiming to enhance the accuracy in the context of
from X(Twitter) and Facebook. The methodology
social media analytics.
of the paper implements OCR – for image
recognition, NLP, and CNN – for text content
processing. The model steers into being
advantageous for the health-care, helping the end III. Methodology
user in stress detection in a non-invasive manner
GloVe
with quick results.
GloVe (global vectors) is an unsupervised learning
D. Christy Daniel and L. Shyamala [12] propose a
algorithm. First introduced by Stanford, it works
method to counteract the limitation of the CNN
on generating word embeddings that maps a
models for sentiment analysis. As per the study,
matrix based on aggregated global word-word co-
even the though CNN models extract high-level
occurrence from the corpus. This results in a
features, abstract nouns are difficult to identify.
linear substructure of the representation of the
They propose a hybrid Convolutional Neural
words.
Networks - Global Vectors - Complex Sentence
Searching - ABstract Noun Searching (CNN-
GloVe-CSS-ABNS) model. As the emoting words
tend to abstract nouns rather than adjectives or
verbs, the proposed model utilizes Complex
 of a dense layer of 28 units and the sigmoid
activation function.

GoEmotions
GoEmotions: A Dataset of Fine-Grained
Emotions[19], is a corpus consisting of 58,000
comments extracted from popular English
language Reddit threads, with human annotations
to over 27 emotions categories. The emotion
categories are: admiration, amusement, anger,
annoyance, approval, caring, confusion, curiosity,
desire, disappointment, disapproval, disgust,
embarrassment, excitement, fear, gratitude, grief,
Linear Substructure of the words
joy, love, nervousness, optimism, pride,
realization, relief, remorse, sadness, surprise.
The model is trained to learn the word
embeddings that preserve the ratios of co-
occurrence probabilities between words. It’s
defined by the objective function:
J=∑i,j=1Vf(Xij)(wiTw~j+bi+b~j−log(Xij))2
As a result, the dense the vector representations
produced for the vocabulary are capable of
capturing the semantic and syntactic relationships Range of the emotions in the dataset [19]
between the words. For this paper we’ll be
utilizing the 300d data of the pre-trained model
provided in the model.

CNN Model

Kim’s CNN model

Here, we have used a variation of the Kim’s CNN

model [17]. The Kim’s CNN model is widely used
for text classification, which is applied here to
understand her to categorize the semantics of the
sentences. We use the D convolution layer along
side dropout to prevent overfitting of the data. The
pooling layers utilize the global max pooling
function to reduce the dimensionality of the
mapped features. The final classification consists Categorization of the sentiment [19]
Results

After the model trained for a humble 12 rounds of

learning, it got really good at figuring out patterns
in the data, reaching an F1 score of almost 42%.
Then, we did a little experiment with different
cutoff points and found that setting the threshold
at 0.25 gave us the best overall score.
With this threshold in place, we looked at how
well the model did for each emotion separately. It
turned out that some emotions, like amusement,
gratitude, and love, were pretty easy for the model
to predict accurately. But others, like
disappointment, realization, and relief, were a bit
trickier. When we plotted these scores, we could
see these differences clearly.

iv. Conclusion
The paper focused on the use of CNN model for
the categorization of 27 emotions for sentiment
analysis. The experiment provides a F1 score of
41%, which is commendable when compared with
the F1 score of BERT at 42%. While this
experimentation doesn’t provide the optimal
results that we are looking for, it gives a way to
the development of this model further to enhance
its ability to better recognize the sentiments. The
development of the CNN model can be then used
for the enhancement of the combination models of
CNN and NLP algorithms such as LSTM and
BERT.
The figure gives the F1 score as per the emotions.

References
[1] Cicero Nogueira dos Santos, Maira Gatti,
“Deep Convolutional Neural Networks for
Sentiment Analysis of Short Texts”,
COLING (2014).
[2] Xi Ouyang, Pan Zhou, Cheng Hua Li,
Lijun Liu, “Sentiment Analysis Using
Convolutional Neural Network”, IEEE
(2015).
[3] Zufan Zhang, Yang Zou, Chenquan Gan,
“Textual sentiment analysis via three
different attention convo15lutional neural
networks and cross-modality consistent
regression”, Neurocomputing (2017).
[4] Bonggun Shin, Timothy Lee, Jinho D.
Choi, “Lexicon Integrated CNN Models
 with Attention for Sentiment Analysis”, [14] Sameeksha Khare, “Accuracy
arXiv (2017) Enhancement During Sentiment Analysis
in Twitter Using CNN”, IEEE (2022)
[5] Shiyang Liao, Junbo Wang, Ruiyun Yu,
Koichi Sato, Zixue Cheng, “CNN for [15] Poornima A, Nataraj N , Nithya R ,
situations understanding based on Nirmala D , Divya P, “SENTIMENT
sentiment analysis of twitter data”, ANALYSIS OF TWEETS IN TWITTER
Elsevier (2017) USING CNN”, IEEE (2022)
[6] Moch. Ari Nasichuddin, Teguh Bharata [16] Dr. Bhaludra R Nadh Singh, Ms.
Adji, Widyawan3, “Performance SpurthiTippani, Ms.
Improvement Using CNN for Sentiment SravaniReddyaGaddam, Ms. SainniUdaya
Analysis”, IJITEE (2018) Sri, Ms. VuggeUshasree, and Ms.
JampulaVandana, “Intelligent Multimodal
[7] Yuling Chen, Zhi Zhang, “Research on text
Sentiment Analysis Using a CNN-Based
sentiment analysis based on CNNs and
Framework”, IEEE (2024)
SVM”, IEEE (2018)
[17] Yoon Kim, “Convolutional Neural
[8] Sujata Rani, Parteek Kumar, “Deep
Networks for Sentence Classification”,
Learning Based Sentiment Analysis Using
arXiv (2014)
Convolution Neural Network”, Arabian
Journal for Science and Engineering [18] A. A. L. Cunha, M. C. Costa, and M. A. C.
(2018) Pacheco, “Sentiment analysis of youtube
video comments using deep neural
[9] Deepanway Ghosal, Navonil Majumder,
networks,” International Conference on
Soujanya Poria,Niyati Chhaya and
Artificial Intelligence and Soft Computing,
Alexander Gelbukh, “DialogueGCN: A
China, 2019.
Graph Convolutional Neural Network for
Emotion Recogre3w350nition in [19] Dana Alon and Jeongwoo Ko, Software
Conversation”, COLING (2019) Engineers, “GoEmotions: A Dataset for
Fine-Grained Emotion Classification”,
[10] Hannah Kim, Young-Seob Jeong,
Google Research, 2021
“Sentiment Classification Using
Convolutional Neural Networks”, MDPi
(2019)
[11] T Radhika, Ch Sandhya, Punna Mahesh, K
Manohar Reddy, “PRINCIPLE OF
SENTIMENT ANALYSIS USING CNN
AND ONLINE SOCIAL NETWORK
DATA, A TECHNOLOGICAL METHOD
TO STRESS DETECTION”, IJCRT
(2019)
[12] D. Christy Daniel, L. Shyamala, “Hybrid
CNN Classification for Sentiment Analysis
under Deep Learning”, IJITEE (2020)
[13] Kifayat Ullah, Anwar Rashad , Muzammil
Khan ,Yazeed Ghadi , Hanan Aljuaid and
Zubair Nawaz, “A Deep Neural Network-
Based Approach for Sentiment Analysis of
Movie Reviews”, WILEY (2022)