2024 2nd International Conference on Computer, Communication and Control (IC4)

Significance of Sentiment Analysis Approaches

using Machine Learning (ML) Techniques
2024 2nd International Conference on Computer, Communication and Control (IC4) | 979-8-3503-8793-3/24/$31.00 ©2024 IEEE | DOI: 10.1109/IC457434.2024.10486310

Gurpreet Singh Jaspreet Singh Bebesh Tripathy

Department of CSE Department of CSE Department of CSE
Chandigarh University Chandigarh University Chandigarh University
Mohali, India Mohali, India Mohali, India
[email protected] [email protected] [email protected]

Abstract—Innovation and research in academia and business and identify growing trends. Politicians usually give speeches
continue to focus on sentiment analysis. There are many different
methods to communicate feelings. Occasionally, it manifests itself
in a variety of ways, including thoughts, advice, and viewpoints
toward other people. Regardless of the circumstance, a person’s
rapid reaction can be an expression of their feelings. Opinion
mining, or sentiment analysis, is the practice of discovering
and classifying people’ feelings or opinions re-grading any
products, services, or events, whether they are good, negative,
or neutral in nature. In practical uses, machine learning (ML)
approaches have been used to address this challenge. Machine
learning (ML) refers to a wide spectrum of techniques for helping
computers acquire their possess procedures without demanding
to be explicitly instructed by one or more human-developed
computational methods. It is used to solve obstacles for which
the expenditure of developing computational methods by people
with programming skills could become prohibitive. The principal Fig. 1. Document By Country-wise: Sentiment Analysis Process
objective has become to evaluate and read the application’s use
of ML approaches for sentiment analysis of critiques obtained or they participate in debates that are recorded for public
from consumers. This study has important ramifications for opinions. Sentiment Analysis can help to get public opinion if
companies that offer services in terms of creating consumer
its positive or negative. On platforms like YouTube,
selection methods for businesses that satisfy their specifications.
Additionally, there is a significant impact on future study areas thousands to millions of videos and podcasts are uploaded
for investigators. everyday [3]. Sentiment Analysis can help in knowing how
Keywords—Sentiment Analysis, Sentiment Analysis users are responding to these videos. Sentiment analysis is a
Approaches, Machine Learning, Techniques for Sentiment crucial application of machine learning with significant
Analysis, Process of Sentiment Analysis. importance and widespread use across various domains.
Machine learning for sentiment analysis, often referred to as
I. INTRODUCTION opinion mining, is a powerful and widely used approach for
Sentiment analysis of audio and video text has grown in automatically determining the sentiment or emotional tone in
popularity. It can be applied to a variety of tasks, such as text data. machine learning project is to collect labeled data.
market research, analysis of customer reviews, and monitoring In the context of sentiment analysis, this involves gathering
of social media. Audio and video transcription, feature text data and labeling it with sentiment categories. ext data
extraction, sentiment classification, and analysis are some of needs to be preprocessed before it can be used for training
the processes in the process of sentiment analysis of audio and machine learning models. This involves tasks like tokenization,
video text [1]. Sentiment analysis on audio and video text has removing stop words, and stemming or lemmatization to
grown in importance as a tool for businesses and organization reduce words to their root forms. To train machine learning
to better understand the opinions and preferences of their models, text data needs to be transformed into numerical
audience due to the popularity of audio and video content on a features that the models can understand [4]. Sentiment analysis
variety of platforms, including social media, podcasts, and datasets can often be imbalanced, with one sentiment category
video-sharing websites [2]. Transcribing audio and video significantly more prevalent than others. This problem can be
content into text is known as audio and video transcription. It solved by employing strategies like weighted loss functions,
can help news broadcasts and other form of media to take undersampling, or oversampling. Continuous observation and
public opinion

979-8-3503-8793-3/24/$31.00 ©2024 IEEE

Authorized licensed use limited to: BMS Institute of Technology. Downloaded on June 13,2024 at 06:55:11 UTC from IEEE Xplore. Restrictions apply.
upkeep are necessary for sentiment analysis machine learning (MFCC) for audio sentiment analysis and visual
models. This involves frequently retraining the model with processing techniques for video sentiment analysis.
fresh data so that it can adjust to changing sentiment and • Multimodal Sentiment Analysis Combining information
language patterns [5]. Deep learning and natural language from both audio and video sources can lead to better
processing advances are driving further developments in sentiment analysis results. Multimodal models process
machine learning for sentiment analysis. It offers useful data from different modalities and integrate them for a
insights from text data and has a wide range of applications, more comprehensive analysis [11].
from business and marketing to social media monitoring, Contextual Analysis Sentiment analysis in audio and
customer service, and more. video should consider the surrounding context to
understand the intended emotions accurately. Contextual
II. LITERATURE SURVEY analysis can involve examining facial expressions, tone
The capacity of machine learning (ML) algorithms to handle of voice, and environmental cues to infer sentiment more
big datasets and recognize intricate patterns in text and by effectively [12].
document per year by source is a key feature of sentiment • Emotion Recognition Emotion recognition is a subset of
analysis, as Figure illustrates. Below are some of the key sentiment analysis that focuses on identifying specific
significances emotions conveyed in audio and video content. This
can involve training models to detect emotions like
happiness, anger, sadness, and surprise based on vocal
cues and facial expressions [13].
Transfer Learning Transfer learning techniques, where
models pretrained on large text datasets are fine-tuned
for audio and video sentiment analysis, can help leverage
the knowledge gained from text sentiment analysis tasks.
Human-AI Hybrid Systems In cases where sentiment
analysis accuracy is crucial, human annotators can assist
in reviewing and refining the sentiment predictions made
by AI models, leading to higher-quality results.
• Adaptive Models: Models that can adapt and learn over
Fig. 2. Audio-Based Sentiment Analysis Process time based on user feedback can provide more accurate
sentiment analysis as they gain insights from real-world
of sentiment analysis approaches using ML techniques given
interactions [14].
in Table 1:
• Contextual Embeddings: Utilizing contextual embeddings
III. APPROACHES AND CHALLENGES from models like GPT-3 or BERT can help capture
Sentiment analysis and evaluation of audio and video text the nuances of sentiment in audio and video text by
present unique challenges compared to traditional textual considering the surrounding words and phrases. User-
sentiment analysis [9]. There are some approaches and Generated Content Analysis: Social media platforms and
challenges for sentiment analysis and evaluation of audio and user-generated content provide a wealth of audio and
video text: video data. Analyzing sentiments in these platforms can
offer insights into public opinions and trends.
A. Approaches
Speech-to-Text Conversion: Audio and video data need B. Challenges
to be converted into text format using speech-to-text (STT) 1) Noisy Data: Audio and video data often contain back-
technology before sentiment analysis can be performed. ground noise, overlapping speech, and varying audio
Several automatic transcription services, such as Google quality. These factors can negatively impact the accuracy
Cloud Speech-to-Text and IBM Watson Speech to Text, can of sentiment analysis models.
be utilized for this purpose. 2) Lack of Textual Clues: Body language, tone of voice,
• Text-Based Sentiment Analysis Once the audio or video and facial expressions are examples of non-verbal
is transcribed into text, traditional text-based sentiment signals that are used in audio and video to communicate
analysis techniques can be applied [10]. These include meaning. It might be difficult to extract these cues and
machine learning algorithms, rule-based techniques, and apply them to sentiment analysis [15].
deep learning models such as transformer-based models 3) Data Preprocessing: Preprocessing audio and video
like BERT or recurrent neural networks (RNNs). data requires specialized techniques for noise reduction,
Audio/Video Sentiment Analysis Instead of converting speech enhancement, and audio segmentation before
to text and then using text-based models, specialized sentiment analysis can be performed effectively.
models that directly analyse audio or video content can
be used. These models can include audio processing
techniques like Mel-frequency cepstral coefficients

Authorized licensed use limited to: BMS Institute of Technology. Downloaded on June 13,2024 at 06:55:11 UTC from IEEE Xplore. Restrictions apply.
 TABLE I
SENTIMENT ANALYSIS APPROACHES USING ML TECHNIQUES: LITERATURE SURVEY

Authors Approached Used Summary Surveys

Maghilan S, et.al (2017) The authors emphasize the importance The authors introduce a new method for sentiment analy-
of considering individual differences in sis of spoken language, focusing on individual differences
language patterns and expressions when in language patterns and expressions [6]. The method,
analyzing mood using audio samples, acoustic features, and machine
learning, improves accuracy and has potential
applications.
Ziheng Zhang, et.al (2018) The author proposes a deep learning The study presents a deep learning framework for
framework for recognizing emotions recognizing emotions from audio and visual data,
from audio and visual data. combining convolutional neural networks for visual
processing and long-term short-term memory networks
for speech analysis, demonstrating improved
performance [7].
Harika Abburi, et.al (2017) The authors of this article approach
multimodal sentiment analysis by The article discusses multimodal sentiment analysis using
adopting the concept of choice choice domains, focusing on specific regions of interest.
domains. However, due to limited data, this concept has not been
applied to deep neural network classifiers. The authors
suggest combining text and audio modalities with video
for improved performance [8].
Li, et.al (2017) The author proposes a method to im-
prove sentiment classification by The study proposes a method for improving sentiment
combining multiple SVM classifiers classification by combining multiple SVM classifiers with
with different sets of functions. different functions. Experimental results show higher
accuracy, precision, recall, and F1 score in mood
classification, highlighting the impact of different feature
sets and combination rules.
Bo Pang, et.al (2002) The authors divide text into positive,
negative, and neutral attitudes using a The authors propose that machine learning techniques
naive Bayesian classifier. can enhance sentiment classification accuracy, potentially
improving product and service quality. They define
sentiment classification as determining a speaker’s or
author’s attitude towards a topic, addressing challenges
like linguistic ambiguity and large training data. They
introduce a naive Bayesian classifier for sentiment
classification, achieving an accuracy of 82.4 in a dataset
of manually tagged movie reviews [9].
Yuanbin Wu, et.al (2009) The authors propose a machine
learning-based approach to train a The authors propose a machine learning-based approach
phrase dependency parser that can to improve opinion mining by incorporating phrase de-
accurately extract the relationships pendency analysis. This method accurately extracts
between product features and opinions. relationships between product features and opinions, im-
proving the accuracy and applicability of opinion mining
techniques in product reviews.

4) Limited Labeled Data: Compared to textual data, labeled 9) Dynamic Sentiment: Sentiments can change rapidly
audio and video data for sentiment analysis are often within a conversation or video, making it necessary for
scarcer. Training effective models with limited labeled models to capture dynamic shifts in sentiment over time
data can be difficult [16]. [17] [18].
5) Computational Complexity: Some multimodal sentiment 10) Cultural Sensitivity: Cultural differences can
analysis approaches require substantial computational significantly impact the interpretation of sentiments in
resources due to the processing of audio, video, and audio and video content. Developing models that are
textual data. culturally sensitive and adaptable is essential.
6) Cultural and Language Variability: Audio and video
content can be in different languages and exhibit Addressing these challenges and refining approaches for
cultural nuances that may not be adequately captured by sentiment analysis and evaluation of audio and video text in-
sentiment analysis models trained on a specific dataset. volves a combination of advanced technology development,
7) Sarcasm and Irony: Detecting sarcasm and irony can be ethical considerations, and interdisciplinary research efforts
more challenging in audio and video compared to text, [18]. To overcome these challenges and improve audio and
where linguistic cues are more apparent [17]. video sentiment analysis, ongoing research focuses on
8) Real-time Analysis: Real-time sentiment analysis for developing specialized models, utilizing multimodal
live audio or video streams introduces additional approaches, and improving data collection and annotation
challenges due to time constraints and the need for techniques. The evolving nature of sentiment analysis in audio
efficient processing. and video text calls for continuous research and innovation to
overcome these

Authorized licensed use limited to: BMS Institute of Technology. Downloaded on June 13,2024 at 06:55:11 UTC from IEEE Xplore. Restrictions apply.
challenges and develop robust methodologies that can provide
valuable insights from multimedia content.
IV. TECHNIQUES FOR SENTIMENT ANALYSIS
Opinion mining, or sentiment analysis, is the process of
identifying the sentiment or emotional tone that is
communicated in textual data. Sentiment analysis may be
done in a variety of ways, from rule-based approaches to
machine learning-based approaches. Some popular sentiment
analysis methodologies are listed below:
Fig. 4. Audio-Based Sentiment Analysis Process
A. Text-Based Sentiment Analysis Techniques
Text-based sentiment analysis approaches refer to ways of
examining textual material in order to identify the sentiment or • Emotion Recognition Models: Train models (e.g.,
emotional tone that is being conveyed [19]. These methods are Convolutional Neural Networks, Recurrent Neural
essential for comprehending attitudes, feelings, and opinions Networks) to classify emotions based on audio features.
in textual material and text-based sentiment analysis methods:
C. Video-Based Sentiment Analysis Techniques
• Lexicon-Based Approaches: Utilize sentiment lexicons or
dictionaries to assign sentiment scores to words in the • Facial Expression Analysis: Facial Expression Analysis:
text. Aggregate scores to determine overall sentiment. To identify emotions, use computer vision techniques to
• Machine Learning Models: Train supervised machine
examine lip, eye, and facial movements.
learning models (e.g., Naive Bayes, Support Vector Ma- • Gesture and Body Language Analysis: Detect and analyze
chines, Random Forests) on labeled textual data to gestures, postures, and body movements to infer emotions
classify sentiment. and sentiments from video content.

V. PROCESS OF SENTIMENT ANALYSIS

• Data Collection and Preprocessing:
– Collect a diverse and representative dataset of audio
and video content.
– Convert audio content to text using speech-to-text
(STT) technology.
– Segment the content into smaller units, such as
sentences, paragraphs, or scenes.
• Textual Sentiment Analysis:
– Tokenize the text into words or sub word units.
Fig. 3. Text-Based Sentiment Analysis Process – Remove stop words, punctuation, and special char-
acters.
• Deep Learning Models: For sequence-to-sequence – Lemmatize or stem words to reduce variations.
sequential classification, use deep learning architectures – Apply sentiment lexicons, dictionaries, or pretrained
such as Recurrent Neural Networks (RNNs), Long Short- models to assign sentiment scores to individual
Term Memory (LSTM) networks, or Transformer-based words or phrases.
models (e.g., BERT) [20]. – Calculate aggregate sentiment scores for each
• Aspect-Based Sentiment Analysis: Analyze sentiment to- segment using methods like summing, averaging, or
wards specific aspects or entities within the audio or video weighted aggregation.
text, enabling a more detailed analysis.
• Audio/Video Feature Extraction:
B. Audio-Based Sentiment Analysis Techniques – Extract audio features such as Mel-frequency
Audio-based sentiment analysis is a growing field, and cepstral coefficients (MFCCs), pitch, energy, and
its application is expanding to various domains, including spectral features.
healthcare, call centers, entertainment, and human-computer – Extract video features such as facial expressions,
interaction. Advancements in speech recognition technology head movements, gestures, and body language.
and machine learning have contributed to the development Emotion Recognition (Optional):
of more accurate and robust audio-based sentiment analysis – Use audio features to recognize emotions conveyed
systems [21]. through tone, pitch, and speech patterns.
• Acoustic Features: Extract audio features such as pitch, – Employ computer vision techniques to recognize
intensity, spectral characteristics, and rhythm to capture emotions from facial expressions and body language
emotional cues in speech. in video content.

Authorized licensed use limited to: BMS Institute of Technology. Downloaded on June 13,2024 at 06:55:11 UTC from IEEE Xplore. Restrictions apply.
 • Bias and Fairness Analysis: REFERENCES
– Evaluate and mitigate potential biases in sentiment [1] A. Ghandour and B. J. Woodford, “Regulating internet of things: The
analysis models, particularly related to cultural, gen- case of the united arab emirates,” 2021.
der, or linguistic biases. [2] G. Singh and J. Singh, “A fog computing based agriculture-iot frame-
work for detection of alert conditions and effective crop protection,”
• Application and Deployment: in 2023 5th International Conference on Smart Systems and Inventive
Technology (ICSSIT). IEEE, 2023, pp. 537–543.
– Apply sentiment analysis results to real-world appli- [3] K. Al-Amleh, “A study into the adoption of internet of things–iot
cations such as social media monitoring, customer technologies within contractors in dubai, united arab emirates,” Ph.D.
feedback analysis, content recommendation, or brand dissertation, The British University in Dubai (BUiD), 2020.
perception analysis. [4] M. Awawdeh, A. Bashir, T. Faisal, K. Alhammadi, M. Almansori,
and S. Almazrouei, “Embedded ventilation air conditioning system for
– Continuously update models to adapt to evolving protection purposes with iot control,” in 2019 Advances in Science
language and cultural trends. and Engineering Technology International Conferences (ASET). IEEE,
2019, pp. 1–6.
• Ethical Considerations: [5] H. Younies, T. Na et al., “Effect of cybercrime laws on protecting
– Ensure data privacy and obtain necessary citizens and businesses in the united arab emirates (uae),” Journal of
Financial Crime, vol. 27, no. 4, pp. 1089–1105, 2020.
permissions for analyzing user-generated content. [6] G. Singh and J. Singh, “A cost effective iot-assisted framework coupled
– Address potential ethical issues related to sentiment with fog computing for smart agriculture,” in 2023 IEEE 8th Interna-
analysis, bias, and user consent. tional Conference for Convergence in Technology (I2CT). IEEE, 2023,
pp. 1–8.
Iterative algorithms combining language, audio processing, [7] F. Hu, Security and privacy in Internet of things (IoTs): Models,
computer vision, and machine learning are used in the Algorithms, and Implementations. CRC Press, 2016.
sentiment analysis and assessment of audio and video text. [8] C. Chandramma, P. Prakash, P. Nandankar, H. Roopa, I. Kathir, and
P. Singh, “Automation of camel race by controlling dc motor speed
Customization of each step based on the specific using blynk application through iot,” in AIP Conference Proceedings,
characteristics of the data and the intended application is vol. 2690, no. 1. AIP Publishing, 2023.
crucial for achieving accurate and meaningful sentiment [9] P. A. Hafeez, G. Singh, J. Singh, C. Prabha, and A. Verma, “Iot
in agriculture and healthcare: Applications and challenges,” in 2022
analysis results. It’s important to note that the process can be 3rd International Conference on Smart Electronics and Communication
adapted and customized based on the specific goals, (ICOSEC). IEEE, 2022, pp. 446–450.
constraints, and challenges of the sentiment analysis task for [10] J. Singh, G. Singh, G. Aggarwal et al., “Inclusion of aerial computing
in internet of things: Prospects and applications,” in 2022 Third Interna-
audio and video text [22]. Additionally, the integration of tional Conference on Intelligent Computing Instrumentation and Control
different modalities (text, audio, video) and the utilization of Technologies (ICICICT). IEEE, 2022, pp. 1664–1669.
advanced techniques (deep learning, multimodal models) can [11] O. B. Olushola, “Factors affecting iot adoption,” IOSR J. Comput.
Eng.(IOSR-JCE), vol. 21, pp. 19–24, 2019.
further enhance the accuracy and depth of sentiment analysis. [12] N. Banwala, G. Singh, J. Singh, V. S. Goswami, and A. Bagnia,
“Recommendation system and its techniques in machine learning: A
VI. CONCLUSION survey,” in Soft Computing: Theories and Applications: Proceedings of
SoCTA 2022. Springer, 2023, pp. 197–205.
An especially effective approach for organisations trying [13] P. Sharma, G. Singh, and N. Kaur, “Data valuation using shapley value
determine sentiments, feelings, as well as emotion around in machine learning,” in AIP Conference Proceedings, vol. 2916, no. 1.
their company’s identity is sentiment analysis. The majority of AIP Publishing, 2023.
[14] I. Dagar, G. Singh, P. K. Singh, and S. Maheshwari, “A review on
sentiment research missions have so far been carried out nearly evolution, application and future of cloud computing,” in AIP Conference
exclusively by enterprises and companies using information Proceedings, vol. 2916, no. 1. AIP Publishing, 2023.
from social media, survey results, and content published by [15] R. Khanna, G. Singh, and P. Singh, “Xe the new variant of covid:
Analysis and discussion,” in AIP Conference Proceedings, vol. 2916,
users’ hubs. Businesses like these may gain additional insight no. 1. AIP Publishing, 2023.
into customer conduct and eventually provide more accurate [16] G. Singh and S. K. Singh, “Analysis of data visualization techniques
goods, services, and sensations to their target audience by useful for machine learning and visual reality,” in 2023 2nd International
Conference on Edge Computing and Applications (ICECAA). IEEE,
polling and analyzing the opinions of consumers. To 2023, pp. 1077–1082.
appropriately recognize the significance of communication via [17] S. Prabowo, M. Abdurohman, and H. H. Nuha, “Internet of things se-
the internet in society along with the information they reveal curity and privacy policy: Indonesia landscape,” in Information Systems
for Intelligent Systems: Proceedings of ISBM 2022. Springer, 2023,
about those who live behind their screen, sentiment analysis pp. 201–210.
will keep continuing to delve deeper, moving beyond the [18] R. Alharbi and H. Almagwashi, “The privacy requirments for wearable
surface of likes, comments, and shares. Humans frequently iot devices in healthcare domain,” in 2019 7th International Conference
on Future Internet of Things and Cloud Workshops (FiCloudW). IEEE,
use complex language to convey their opinions. Additionally, 2019, pp. 18–25.
even while they convey their beliefs, their vocabulary itself is [19] H. Hacid, F. Outay, H.-y. Paik, A. Alloum, M. Petrocchi, M. R.
deceptive. Additionally, negation, topic alterations, and Bouadjenek, A. Beheshti, X. Liu, and A. Maaradji, Service-Oriented
fluctuations are frequently seen in sub-sentiment as well as Computing–ICSOC 2020 Workshops: AIOps, CFTIC, STRAPS, AI-PA,
AI-IOTS, and Satellite Events, Dubai, United Arab Emirates, December
intra contextual analysis. A review of the most recent 14–17, 2020, Proceedings. Springer Nature, 2021, vol. 12632.
algorithms used to categorize communications into two [20] A. I. Aljumah, M. T. Nuseir, and G. A. El Refae, “Internet of things
categories messages or junk is given. Discussion followed (iot): A way to expedite production and service performance empirical,
evidence from textile industry of united arab emirates (uae),” in 2022 9th
regarding the efforts made by several researchers to apply International Conference on Internet of Things: Systems, Management
machine learning classifiers to address receiving this message and Security (IOTSMS). IEEE, 2022, pp. 1–8.
issue.

Authorized licensed use limited to: BMS Institute of Technology. Downloaded on June 13,2024 at 06:55:11 UTC from IEEE Xplore. Restrictions apply.
[21] W. Shao and L. Li, “Analysis of the development route of iot in china,”
Perking: China Science and Technology Information, vol. 24, pp. 330–
331, 2009.
[22] B. Li and J. Yu, “Research and application on the smart home based on
component technologies and internet of things,” Procedia Engineering,
vol. 15, pp. 2087–2092, 2011.

Authorized licensed use limited to: BMS Institute of Technology. Downloaded on June 13,2024 at 06:55:11 UTC from IEEE Xplore. Restrictions apply.