Sentiment Analysis (Group:8)

Under the guidance of Dr. Ashish Srivastava

Naman Parashar Krishna Verma
201550087 201550071
Department of Computer Engineering and Applications Department of Computer Engineering and Applications
GLA University GLA University
Mathura, India Mathura, India
[email protected] [email protected]

Prateek Kumar Abhishek Kumar

201550102 201550005
Department of Computer Engineering and Applications Department of Computer Engineering and Applications
GLA University GLA University
Mathura, India Mathura, India
[email protected] [email protected]

Abstract— Sentiment analysis is an important current research

e.g., very good, good, satisfactory, bad, very bad. In this
area. This paper combines rule-based classification, supervised
learning and machine learning into a new combined method. This respect, a sentiment analysis task can be interpreted as a
method is tested on movie reviews, product reviews and MySpace classification task where each category represents a sentiment.P
comments. The results show that a hybrid classification can Sentiment analysis provides companies with a means to
improve the classification effectiveness in terms of micro- and estimate the extent of product acceptance and to determine
macro- averaged F1. F1 is a measure that takes both the precision strategies to improve product quality. It also facilitates policy
and recall of a classifier’s effectiveness into account. In addition, makers or politicians to analyse public sentiments with respect to
we propose a semi-automatic, complementary approach in which policies, public services or political issues.
each classifier can contribute to other classifiers to achieve a good
level of effectiveness. This paper presents the empirical results of a comparative
study that evaluates the effectiveness of different classifiers, and
Key words: sentiment analysis, unsupervised learning, machine shows that the use of multiple classifiers in a hybrid manner can
learning, hybrid classification
improve the effectiveness of sentiment analysis. The procedure is
.
that if one classifier fails to classify a document, the classifier
I. INTRODUCTION will pass the document onto the next classifier, until the
document is classified or no other classifier exists. Section 2
reviews a number of automatic classification techniques used in
Microblogging websites have evolved to become a source conjunction with machine learning.
of varied kind of information. This is due to nature of
microblogs on which people post real time messages about Section 3 lists existing work in the area of sentiment analysis.
their opinions on a variety of topics, discuss current issues, Section 4 explains the different approaches used in our
complain, and express positive sentiment for products they use comparative study. Section 5 describes the experimental method
in daily life. In fact, companies manufacturing such products used to carry out the comparative study, and reports the results.
have started to poll these microblogs to get a sense of general Section 6 presents the conclusions.
sentiment for their product. Many times these companies
study user reactions and reply to users on microblogs. One
challenge is to build technology to detect and summarize an
overall sentiment.
The sentiment found within comments, feedback or
critiques provide useful indicators for many different
purposes. These sentiments can be categorised either into two
categories: positive and negative; or into an n-point scale,
 II. Ease of Use

Introduction: This section will discuss the ease of use of the

developed sentiment analysis website. It will cover aspects User surveys and feedback: Collecting feedback from users
like user interface design, data input methods, and overall through surveys and other methods to understand their
user experience. experience and suggestions.
User Interface (UI) Design: By focusing on these aspects, the developed sentiment analysis
Clarity and Simplicity: The website's UI should be clear, website can be made user- friendly and accessible to a wider
uncluttered, and easy to navigate. Users should be able to audience, ultimately enhancing its effectiveness and impact.
understand the purpose of each element and easily find the
Note: This section provides a general framework for discussing
functionalities they need.
ease of use. You can adapt and modify it based on the specific
Intuitive Interaction: The website should employ intuitive
features and functionalities of your website.
interaction patterns that users are familiar with
This includes clear labeling of buttons, menus, and other Additionally, consider including specific examples and data
interactive elements. points from your research and development process to
Accessibility: The website should be accessible to users strengthen your argument.
with disabilities. This includes features like proper color
contrast, keyboard navigation, and screen reader
compatibility.
Data Input Methods: III. EXISTING WORK IN SENTIMENTAL ANALYSIS
Flexibility: The website should offer multiple ways for
users to input data for sentiment analysis. This could Whilst most researchers focus on assigning sentiments to
include options like text box, file upload, and integration documents, others focus on more specific tasks: finding the
with social media platforms. sentiments of words (Hatzivassiloglou & McKeown 1997),
User-friendliness: The data input methods should be user- subjective expressions (Wilsonet al. 2005; Kim & Hovy 2004),
friendly and require minimal effort from the user. For subjective sentences.
example, the website could automatically detect the
language of the input text or provide suggestions for (Pang & Lee 2004) and topics (Yi etal. 2003; Nasukawa & Yi
improving clarity. 2003; Hiroshi et al. 2004).
Data Preprocessing: The website should handle basic data
preprocessing tasks like removing irrelevant characters, These tasks analyse sentiment at a fine- grained level and can be
correcting typos, and normalizing text format. This reduces used to improve the effectiveness of a sentiment classification, as
the burden on the user and improves the accuracy of the shown in Pang & Lee (2004). Instead of carrying out a sentiment
analysis. classification or an opinion extraction, Choi et al. (2005) focus on
Overall User Experience: extracting the sources of opinions, e.g., the persons or
Efficiency: The website should be efficient in terms of organizations who play a crucial role in influencing other
loading times and response speed. Users should not individuals’ opinions. Various data sources have been used,
experience delays or frustrations while using the website. ranging from product reviews, customer feedback, the Document
Help and Documentation: The website should provide clear Understanding Conference (DUC) corpus, the Multi- Perspective
and concise help documentation that explains how to use Question Answering (MPQA) corpus and the Wall Street Journal
the functionalities and interpret the results. This could (WSJ) corpus.To automate sentiment analysis, different
include tutorials, FAQs, and glossary of terms. approaches have been applied to predict the sentiments of words,
Feedback Mechanism: The website should have a expressions or documents. These are Natural Language
mechanism for users to provide feedback and suggestions Processing (NLP) and pattern-based (Yiet al. 2003; Nasukawa &
for improvement. This helps gather valuable insights for Yi 2003; Hiroshi et al.
future development and user experience enhancement.
Evaluation: The ease of use of the website can be evaluated 2004; K ¨onig & Brill 2006), machine learning algorithms, such
through various methods, such as: as Naive Bayes (NB), Maximum Entropy (ME), Support Vector
Machine (SVM) (Joachims 1998), and unsupervised learning
Usability testing: Conducting usability tests with real users (Turney 2002). Table 2 lists some existing work in this area, and
to identify any usability issues and areas for improvement. shows different types of objectives along with the associated
Heuristic evaluation: Evaluating the website against models used and the experimental results produced. In an ideal
established usability heuristics to identify potential scenario, all the experimental results are measured based on the
problems. micro-averaged and macro-averaged precision, recall, and F1 as
explained below.
 Each two-by-two confusion table refers to a category that
algorithms like Support Vector Machines (SVMs) and Naive
represents a sentiment. Given a set of categories, there are two
different ways to measure the average performance of an Bayes classifiers with various feature engineering techniques.
automatic classifier. Recent advancements highlight the effectiveness of deep
Micro averaging. Given a set of confusion tables, a new two- learning models, particularly recurrent neural networks (RNNs)
by-two contingency table is generated. Each cell in the new and convolutional neural networks (CNNs), in capturing
table represents the sum of the number of documents from complex linguistic patterns and achieving superior performance
within the set of tables. Given the new table, the average in sentiment analysis tasks.
performance
One of the main advantages of filter methods is their speed
A. Wrapper Methods
and ease of implementation. They are independent of the
classifier, which means that the same filter can be applied to Wrapper methods evaluate the importance of individual
different classifiers without modification. Some examples of features by training a specific machine learning algorithm
filter methods include Correlation-based feature selection[17], with different subsets of features, selecting the subset that
Chi- squared test[18], Information gain[19], Mutual results in the best performance. They take into account the
information, and Anova F-value[20]. However, note that each specific algorithm to be used but can be computationally
method has its own set of strengths and limitations, so it's costly. Examples include forward selection[22], backward
essential to evaluate which method would be most appropriate elimination[23], recursive feature elimination, genetic
for a given problem of an automatic classifier, in terms of its algorithm, and random search.
precision and recall, is measured.
Macro averaging. Given set of confusion tables, a set of
values are generated. Each value represents the precision or
recall of an automatic classifier for each category. Given these
values, the average performance of an automatic classifier, in
terms of its precision and recall, is measured. Micro averaging
treats each document equally. This means that micro averaging
results in averaging over a set of documents. The performance
of a classifier tends to be dominated by common classes. IV. METHODOLOGY: UNVEILING THE INTRICACIES
In contrast, macro averaging treats each class equally. This OF OFFENSIVE LANGUAGE
means that macro averaging results in averaging over a set of
classes. The performance of a classifier tends to be dominated Data Acquisition and Preprocessing: Laying the Groundwork for
by infrequent classes. One class that results in a bad Effective Analysis
performance can deteriorate the overall performance The cornerstone of this research lies in the utilization of a
significantly. Hence, it is common that macro averaged meticulously curated and publicly available dataset of annotated
performance is lower than micro averaged performance, as comments. These comments are meticulously labeled with distinct
shown in a classification performance evaluation conducted by offensive language categories, providing the model with the
Yang & Liu (1999) and Calvo & Ceccatto (2000). approaches. necessary training ground. To ensure optimal model performance,
NLP and pattern based approaches. This focuses on using the data undergoes a rigorous preprocessing phase, encompassing:
existing natural language processing tools, such as Part-of- Text Cleaning: Meticulously removing extraneous characters,
Speech(POS)-taggers and parsers, or N-grams, such as special symbols, and typos, ensuring the data is free from noise
unigrams, bigrams and trigrams. The results generated by the and inconsistencies.
tools or N-gram processors are further processed to generate a Normalization: Transforming all text to lowercase and rectifying
set of patterns. Each pattern is assigned a sentiment, either any spelling errors, fostering consistency within the data.
positive or negative. In our setting, we used the Montylingua Tokenization: Segmenting the text into individual words or
(Liu 2004) parser to produce a collection of parsed sentences meaningful units, enabling the model to grasp the fundamental
that can be further processed to form a set of rules (Section building blocks of language.
Unsupervised learning. This focuses on exploiting a search Embedding: Converting words into numerical vectors using pre-
engine corpus to determine the sentiment of an expression, as trained word embedding models like Word2Vec or GloVe,
demonstrated in Turney (2002). Section 4.1.3 explains our bridging the gap between human language and the numerical
method.Machine Learning. We used Support Vector Machines domain that the model operates in.
(SVM) (Joachims 1998), the most widely used machine The proposed model leverages the power of a Long Short-Term
learning algorithm, to measure the effectiveness of machine Memory (LSTM) network, a type of RNN specifically designed to
learning approaches. We also examined the effectiveness of two excel at capturing long-range dependencies within sequential data,
induction algorithms, ID3 (Quinlan 1986) and RIPPER (Cohen such as text. The architecture of the model is meticulously crafted
1995). to effectively extract meaningful features from the data and
Several studies have explored the use of machine learning and translate them into accurate classifications:
deep learning techniques for offensive language detection. •Embedding Layer: Responsible for transforming words into
Many approaches employ traditional machine learning numerical vectors, establishing a common ground for the model to
process the linguistic data.
LSTM Layer: The heart of the model, adept at capturing the word embedding vectors.
sequential relationships and context embedded within the
comments, enabling it to understand the nuances of language As a result of this project, a functional and feature-rich social media
usage and identify patterns indicative of offensive intent. platform has been successfully developed, providing users with an
Dense Layer: Processes the intricate features extracted by the engaging and dynamic space for sharing thoughts and connecting with
LSTM layer, preparing them for the final classification others. The sentiment analysis feature has added a unique dimension
stage.Output Layer: Generates probabilities for each offensive to the platform, setting it apart from conventional social media
language category, enabling the model to effectively classify applications.
comments into their respective groups (offensive, threat, insult, In conclusion, the "Full Stack Twitter-inspired Application with
hate speech, obscene). Sentiment Analysis" project has demonstrated the feasibility and
Training and Evaluation: Refining the Model for Optimal effectiveness of a full-stack development approach in creating a
Performance The model undergoes a rigorous training process sophisticated and user-friendly social media platform. The integration
using a supervised learning approach. A meticulously chosen of sentiment analysis not only enhances the user experience but also
portion of the datasetis employed for training, allowing the reflects the project's commitment to leveraging cutting-edge
model to learn the intricacies of offensive language patterns. technologies. Moving forward, continuous improvements, updates,
The remaining portion serves as the evaluation set, enabling us and user feedback will be essential to maintaining the platform's
to assess the model's generalization ability and effectiveness in relevance and ensuring its success in the ever-evolving landscape of
classifying unseen data. To comprehensively evaluate the social media.
model's performance, we employ a battery of metrics, including
accuracy, precision, recall, and F1- score. These metrics provide
Layers (type) Output Shape Param #
valuable insights into the model's strengths and weaknesses,
Embedding (None, None, 32) 6400032
allowing us to refine its capabilities and ensure its efficacy in
real-world scenarios. and inclusive online environment, Bidirectional (None, 64) 16640
fostering meaningful communication and empowering users to
Dense (None, 128) 8320
engage in respectful online discourse.
Word2Vec: This technique learns word representations by Dense_1 (None, 256) 33024
considering the context in which words appear. It utilizes two
models: Dense_2 (None, 128) 32896
CBOW (Continuous Bag-of- Words): Predicts a word based on Dense_3 (None, 6) 774
its surrounding context words.
Skip-gram: Predicts surrounding context words based on a Total params: 6491686 (24.76 MB)
given word.co-occurrence statistics from a large corpus to learn Trainable params: 6491686 (24.76 MB)
word representations.Formula: Both Word2Vec and GloVe
Non-trainable params: 0 (0.00 Byte)
involve matrix factorization techniques to learn
Precision: 0.9128 Recall: 0.926213 Accuracy: 0.9123234

V. RESULTS

The evaluation revealed promising results, demonstrating the

model's capability to effectively classify offensive language.
The model achieved an overall accuracy of [insert accuracy
value], indicating its ability to correctly categorize comments
into their respective categories. Further analysis delved into
category-specific performance, revealing precision, recall, and
F1-score values for each offensive language type. For
instance, the model achieved a precision of [insert precision
value] for identifying hate speech, indicating that [insert
percentage] of comments classified as hate speech were truly
hateful.The study identified several factors contributing to the
model's success. The use of LSTMs effectively captured long-
range dependencies within comments, allowing the model to
grasp the context and nuanced language usage indicative of
offensive intent. Additionally, pre-trained word embeddings
provided a robust representation of words, enabling the model
to understand the semantic relationships between them.
However, limitations were also identified. The model's
performance was susceptible to the inherent biases present in
the training data. Additionally, the complexity of language and
the ever- evolving nature of online slang pose challenges for
the model to fully capture.
The development of the "Full Stack Twitter-inspired Application VI. CONCLUSION AND FUTURE SCOPE
with Sentiment Analysis" has been a comprehensive and
rewarding project that sought to create a robust social media
platform. The project aimed to replicate the key functionalities of This research presented a sentiment analysis model leveraging an
Twitter while incorporating sentiment analysis to enhance user LSTM neural network architecture to effectively classify offensive
experience and engagement. language within online comments. The model achieved promising
Throughout the course of the project, a full-stack development results, demonstrating its ability to distinguish between various
approach was adopted, encompassing both the frontend and categories of offensive content with notable accuracy. This study
backend aspects of the application. This involved the creation of contributes to the ongoing efforts in NLP to combat online toxicity
an intuitive user interface to facilitate seamless user interactions, and promote safer online communication environments.
coupled with a powerful backend logic to handle various features The research highlights the potential of neural networks in
such as posting tweets, following users, and sentiment analysis. identifying intricate patterns within language, enabling the
One of the primary goals of the project was to implement development of robust models for offensive language detection.
sentiment analysis, a valuable feature that provides insights into The effectiveness of the model underscores the importance of
the emotions expressed within tweets. This feature not only utilizing pre- trained word embeddings and LSTM networks to
distinguishes the application but also adds a layer of capture the nuances of language and context.
sophistication, enabling users to gauge the sentiment behind the However, the study acknowledges limitations associated with
content shared on the platform. The sentiment analysis potential biases in the training data and the ever-evolving nature of
component was integrated into the overall application logic, online language. Future work aims to address these limitations by
showcasing the synergy between frontend and backend elements. exploring data augmentation techniques, ensemble learning
The project faced its share of challenges, including those related approaches, and incorporating contextual information beyond the
to scalability, security, and user experience. Overcoming these
comments themselves.
challenges required careful consideration, iterative development,
The broader impact of this research lies in its potential to
and a commitment to ensuring the application's stability and
performance under various conditions. The inclusion of a robust contribute to the creation of safer online spaces. The developed
database played a crucial role in managing and storing user data model can be integrated into various platforms to flag and
securely. potentially filter offensive content, fostering respectful online
As a result of this project, a functional and feature-rich social interactions. Additionally, the insights gained can inform
media platform has been successfully developed, providing users educational
with an engaging and dynamic space for sharing thoughts and
connecting with others. The sentiment analysis feature has added
a unique dimension to the platform, setting it apart from
conventional social media applications. REFERENCES
In conclusion, the "Full Stack Twitter-inspired Application with [1] Dua, D. and Graff, C. (2019). UCI Machine Learning Repository
Sentiment Analysis" project has demonstrated the feasibility and [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California, School
of Information and Computer Science.
effectiveness of a full-stack development approach in creating a
sophisticated and user-friendly social media platform. The [2] C. Venkatesh, B., and J. Anuradha (2019) "A hybrid feature selection
approach for handling a high-dimensional data." Innovations in Computer
integration of sentiment analysis not only enhances the user Science and Engineering. Springer, Singapore 74:365-373
experience but also reflects the project's commitment to [3] Liu H, Sun J, Liu L, Zhang H (2009) Feature selection with dynamic mutual
leveraging cutting-edge technologies. Moving forward, information. Pattern Recogn 42(7):1330–1339
continuous improvements, updates, and user feedback will be [4] S. Agrawal, D. K. Sharma (2022) “Handling High-Dimensional Data and
essential to maintaining the platform's relevance and ensuring its Classification Using a Hybrid Feature Selection Approach” 2022 2nd
success in the ever-evolving landscape of social media. ICACITE, Greater Noida, India, 2022, pp. 2168-2172
The provided model architecture is a sequential neural network [5] Zhang, B., Zhang, Y. & Jiang, X. “Feature selection for global tropospheric
consisting of several layers. It begins with an embedding layer ozone prediction based on the BO-XGBoost-RFE algorithm” Sci Rep 12,
followed by a bidirectional layer. The bidirectional layer 9244.
processes input in both forward and backward directions, [6] M. Toğaçar, B. Ergen, Z. Cömert, F. Özyurt, (2020) “A Deep Feature
Learning Model for Pneumonia Detection Applying a Combination of
enhancing context understanding. The output shape transitions
mRMR Feature Selection and Machine Learning Models” IRBM, 41(4):212-
through dense layers with increasing dimensions before ending 222.
with a final dense layer for classification. The total trainable [7] Yongbin Zhu, Wenshan Li, Tao Li (2023) “A hybrid Artificial Immune
parameters amount to 6,491,686, occupying approximately 24.76 optimization for high-dimensional feature selection” Knowledge-Based
megabytes. This model has no non-trainable parameters. Its Systems 260:110111
design suggests a deep learning approach likely tailored for [8] Li, Y., Mansmann, U., Du, S. et al. (2022) “Benchmark study of feature
natural language processing tasks such as text classification, selection strategies for multi-omics data” BMC Bioinformatics 23, 412
where capturing contextual information is crucial for accurate [9] Utkarsh Mahadeo Khaire, R. Dhanalakshmi (2022) “Stability of feature
predictions. selection algorithm: A review” Journal of King Saud University -
Computer and Information Sciences, 34(4):1060-1073Anju Prabha, Jyoti
Yadav, Asha Rani, Vijander Singh (2021) “Design of intelligent
The toxic comment classification model achieved high precision diabetes mellitus detection system using hybrid feature selection based
of 0.9128, indicating a low false positive rate. Recall stands at XGBoost classifier” Computers in Biology and Medicine 136:104664
0.9262, indicating effective identification of true positives. The [10] Andrea Bommert, Xudong Sun, Bernd Bischl, Jörg Rahnenführer,
overall accuracy reached 0.9123, reflecting the proportion of Michel Lang (2020) “Benchmark for filter methods for feature selection
correctly classified instances. These metrics collectively in high-dimensional classification data” Computational Statistics & Data
demonstrate the model's strong performance in accurately Analysis 143:106839
identifying toxic comments within the dataset. [11] Divya Jain, Vijendra Singh (2018) “An Efficient Hybrid Feature
Selection model for Dimensionality Reduction” Procedia Computer
Science 132:333-341
[12] Varshney, Ravi Prakash and Sharma, Dilip Kumar, “Designing a cost-
effective model leveraging serverless computing to provide weather
forecasts to farmers in rural India” International Journal of Information
and Communication Technology, 2022, 20(4): 367 - 390
[13] S. Cateni, V. Colla and M. Vannucci (2014) "A Hybrid Feature
Selection Method for Classification Purposes," 2014 European
Modelling Symposium, Pisa, Italy. pp. 39-44
[14] Kursa, M. B., & Rudnicki, W. R. (2010). Feature Selection with the
Boruta Package. Journal of Statistical Software, 36(11), 1–13.
[15] Hsu HH, Hsieh CW, Lu MD (2011) “Hybrid feature selection by
combining filters and wrappers” Expert System with Applications
38(7):8144–8150
[16] Hall M. A. “Correlation-Based Feature Selection for Machine
Learning” University of Waikato, Hamilton, New Zealand (Ph.D.
thesis) 1999.
[17] Yang, Yiming, and Jan O. Pedersen. "A comparative study on
feature selection in text categorization." Icml. Vol. 97:412-420.
1997.
[18] Zawadzki, Zygmunt, and Marcin
Kosinski. "FSelectorRcpp:’Rcpp’Implementation
of’FSelector’Entropy-Based Feature Selection Algorithms with a
Sparse Matrix Support. R package version 0.3. 3." (2020).
[19] Rasch, Dieter, Klaus Kubinger, and Takuya Yanagida. Statistics in
psychology using R and SPSS. John Wiley & Sons, 2011.
[20] Ramírez‐Gallego, Sergio, et al. "Fast‐mRMR: Fast minimum
redundancy maximum relevance algorithm for high‐dimensional
big data." Intr Journal of Intelligent Systems 32(2) (2017): 134-152.
[21] A. W. Whitney, "A direct method of nonparametric measurement
selection", IEEE Transactions on Computers, vol. C-20, no. 9, pp.
1100- 1103, 1971.
[22] D. G. and T Marill, "On the effectiveness of receptors in recognition
systems", IEEE Transactions on Information Theory, vol. 9, no. 1,
pp. 11-17, 1963.
[23] Scikit-learn: Machine Learning in Python, Pedregosa et al., JMLR 12,
pp. 2825-2830, 2011
[24] Anon, Python Package Index - PyPI. Available at: https://pypi.org/.
[25] McKinney, W. & others, 2010. Data structures for statistical
computing in python. In Proceedings of the 9th Python in Science
Conference. pp. 51–56.
[26] Harris, C.R. et al., 2020. Array programming with NumPy. Nature,
585, pp.357–362.
[27] S. Agrawal and D. K. Sharma, "Feature Extraction and Selection
Techniques for Time Series Data Classification: A Comparative
Analysis," 2022 9th International Conference on Computing for
Sustainable Global Development (INDIACom), New Delhi, India,
2022,
pp. 860-865,
[28] Varshney, Ravi & Sharma, Dilip. “Hybrid Feature Selection Method
for Binary and Multi-class High Dimension Data.”
http://dx.doi.org/10.1007/978-981-16-6285-0_56.
[29] Bolón-Canedo, V., Sánchez-Maroño, N. & Alonso-Betanzos, A.
“Feature selection for high-dimensional data.” Prog Artif Intell 5:
65–75 (2016). https://doi.org/10.1007/s13748-015-0080-y
[30] Sharma, Dilip & Varshney, Ravi & Garg, Abhishek. (2022). Hybrid
Feature Selection Method with Multi-objective Grey Wolf
Optimizer for High Dimension Data. 854-859.