Design of an Efficient Model for Sentiment

Analysis on Twitter Using Between Fine-Tuning,

Multimodal Integration, and Attention Mechanisms
Dr.P.Maragathavali11
department of information Technology Mr.P.Somaraju22
Puducherry Technical University department of information Technology
puducherry,india Puducherry Technical University
[email protected] puducherry,india
parasa.somaraju@ ptuniv.edu.in

Abstract:For sentiment analysis of Twitter contributes toward increasing sentiment

data, a robust methodology that can effectively analysis techniques tailored for Twitter data,
catch the nuances of informal language and thereby trying to solve critical challenges and
multimodal information is essential. Previous opening avenues for further research in this
works, though well intended, generally fail to domain.
capture the characteristics of Twitter discourse Keywords:Sentiment Analysis, Twitter Data,
and integrate various modalities. This paper BERTweet, Fine-Tuning, Multimodal
introduces a new framework for sentiment Integration, Attention Mechanisms, Machine
analysis using BERTweet fine-tuning, Learning, Natural Language Processing, Social
multimodal integration, and attention Media Analysis
mechanisms for the task of sentiment analysis.
BERTweet-FT would fine-tune the pre-trained
BERTweet model for Twitter data, so it can
1. Introduction
capture the sentiment-related features within a
tweet. Further, MM-BERTweet can leverage In the dynamic world of social media, Twitter is
the textual content with additional modalities one of the most outstanding platforms for real-
like images and user metadata, leading to rich time communication, characterized as a rich
contextual representations, which in turn source of opinions, sentiments, and trends.
provide accurate sentiment predictions. Analyzing sentiment in Twitter data has huge
Furthermore, AG-BERTweet can even add value in a variety of fields, such as marketing,
enhancements to sentiment analysis by using politics, and monitoring public opinion.
attention mechanisms to pick out salient words Nonetheless, the informality in the Twitter style of
or phrases crucial for sentiment classification. discourse, embodied by the use of abbreviations,
Proposed Framework: Our proposed slang, and emoticons, presents some distinct
framework offers several advantages over the challenges to algorithms for sentiment analysis.
existing state-of-the-art baseline approaches: it Traditional sentiment analysis methods rely on
improves the accuracy of sentiment very basic approaches that, in turn, fail to
classification by 5% to 12% over different understand the subtleties of Twitter language.
ablation components. By using Twitter- Most importantly, these methods usually fail to
specific language comprehension, multimodal recognize that the richness of Twitter content is
information, and attention mechanisms, our multimodal—it spans beyond the textual data and
approach boosts the performance for sentiment into images, users' profiles, and metrics of
analysis of Twitter data samples. Impact of the engagement. These challenges must be overcome
Work: The impact of this work will be felt by advanced methods that use the complexity of
across various applications, like brand Twitter data in a highly effective way and with
sentiment monitoring, public opinion analysis, integration of different modalities for
and social media marketing strategies, where understanding sentiment comprehensively.
accurate sentiment analysis plays a crucial role
in decision-making processes. This paper
The paper presents a novel framework for BERT-large-cased and stochastic gradient descent
sentiment analysis using Twitter data samples to effectively carry out sentiment analysis. Yang
with the integration of three key components: et al. [4] and [5] proposed an affective knowledge
BERTweet fine-tuning, multimodal integration, augmented interactive graph convolutional
and attention mechanisms. BERTweet-FT fine- network for Chinese-oriented aspect-based
tunes the model, BERTweet, which is perfectly sentiment analysis and showed the importance of
developed for Twitter data to capture sentiment- including external affective knowledge and graph
related features. On top of this, MM-BERTweet neural networks for nuanced aspect-based
integrates the text information with other sentiment analysis tasks.
modalities to leverage the richness of multimodal
information in doing sentiment analysis. Further, Hoque et al. [6] made a profound analysis of the
AG-BERTweet allows the proposed model to patterns of tweeting and public engagement
include the attention mechanism to focus on the around the COVID-19 pandemic, which shed light
critical parts of the input text and enhance the on the sentiment expression and public discourse
accuracy in sentiment classification. Finally, dynamics in Twitter. Urolagin and Patel [7]
through the combination of the three in a proposed a loyalty prediction model based on a
synergistic manner, the proposed framework deep neural network user-specific, with emphasis
would offer an entire solution for the challenges of on sentiment analysis in understanding user
sentiment analysis on Twitter. It improves behavior and loyalty dynamics on social media
sentiment classification accuracy and enhances the platforms. Braig et al. [8] explored machine
ability of the model to capture the subtleties of learning techniques for the sentiment analysis of
Twitter language and incorporate diverse sources COVID-19-related Twitter data, underlining the
of information. The work has major implications importance of leveraging machine learning in
in sentiment monitoring for brands, understanding capturing sentiment subtleties during global crises.
the dynamics of public opinion, and guiding Zhang et al. [9] presented BMT-Net, a broad
decision-making processes in social media multitask transformer network for sentiment
marketing. In this way, the study will contribute to analysis, emphasizing the role of multitask
the development of state-of-the-art techniques in learning and attention mechanisms in sentiment
sentiment analysis tailored to Twitter data and classification tasks. Kastrati et al. [10] performed
pave the way for more efficient and insightful an all-inclusive analysis of public engagement on
analysis of social media scenarios. Twitter regarding skyrocketing energy prices. This
was done with sentiment analysis and topic
2. Literature Review modeling techniques with transformers to
understand public sentiment and discourse.
Sentiment analysis on Twitter data has Tahayna et al. [11] addressed sentiment
significantly improved with an ever-growing annotation for idiomatic expressions. The research
demand for the need to understand public showed the importance of fine-tuning deep
sentiment in real-time conversations on social learning models to capture nuanced sentiment
media. Aljebreen et al. [1] proposed a hybrid deep expressions.
learning approach, Moth Flame Optimization with
Hybrid Deep Learning, designed for sentiment He et al. proposed a local and global context focus
classification in Twitter data. It demonstrated the multilingual learning model for aspect-based
effectiveness of combining chatbot-generated sentiment analysis. This signifies the importance
responses with deep learning. Prabha and of context-aware approaches to multilingual
Rathipriya [2] proposed a competitive capsule sentiment analysis tasks. Jahanbin and Chahooki
network-based model for sentiment analysis, proposed hybrid deep transfer learning models for
particularly for Twitter discussions about COVID- aspect-based sentiment analysis of Twitter
19 vaccines, underlining the requirement for influencers. This contributes to the pre-trained
specific solutions in times of public health crises. networks by underlining how they capture
Deep learning techniques have emerged as the domain-specific sentiment trends. Ma et al.
dominant paradigm in sentiment analysis; as proposed a novel framework for sentiment
Durga and Godavarthi [3] proposed Deep- polarity detection in Chinese sentiment analysis.
Sentiment, a decision-based recurrent neural This postulates how important sentiment lexicons
network for sentiment analysis, which utilized and feature extraction techniques are in the
capture of sentiment nuances in Chinese text. iteratively updated to minimize the loss function,
Aygün et al. carried out aspect-based sentiment hence modifying the model's representations
analysis on Twitter discussions over vaccination toward the direction of sentiment-related features
in the context of COVID-19. This shows how more characteristic of Twitter language. This fine-
deep learning techniques can be applied to tuning process uses the pre-trained BERTweet
understand public sentiment dynamics for public model to inject understanding of the context and
health crises. adapt it toward the sentiment classification task.
The fine-tuning of BERTweet for sentiment
3. Proposed Design of an Efficient Model for analysis is justified because it is capable of
Sentiment Analysis on Twitter Using capturing nuances in Twitter language and is
BERTweet Fine-Tuning, Multimodal effective at capturing contextual information. A
Integration, and Attention Mechanisms pre-trained BERTweet model is specifically
designed for Twitter data, so it would be very
This section describes the details of designing the good in performing sentiment analysis tasks
BERTweet fine-tuning process, trying to solve the related to this platform. Moreover, fine-tuning
problems of low efficiency and high complexity. BERTweet enables efficient transfer learning,
Figure 1 graphically shows the process, which which harnesses knowledge encoded in the pre-
starts by initializing the BERTweet model with trained model to adapt toward the specific nuances
pre-trained weights coming from a large corpus of of sentiment analysis.
Twitter texts. The architecture itself consists of a

involves updating the model parameters 𝜃 using

multi-layer transformer model with a self- The gradient descent optimization process

the gradients of the loss function 𝐿 with respect to

attention mechanism and feed-forward neural

𝜃 via equation 2,
networks. The BERTweet model is fine-tuned for
sentiment analysis by using a labeled dataset of
Twitter data. Each tweet in the dataset is
tokenized, and input embeddings are created with θ ( t+ 1 )=θt −α ∇ θL… (2)
the BERTweet tokenizer. These are fed into the
BERTweet model, which processes the input
sequence over several layers of transformers,
generating contextualized representations for each
of the tokens. During the fine-tuning of the
BERTweet model, the parameters are updated via
backpropagation and gradient descent
optimization. A cross-entropy loss function is
usually used for fine-tuning, which measures the
difference between the predicted sentiment labels
and the actual labels. The gradients of such a loss
function, with respect to the model parameters, are
obtained using the chain rule and propagated
through the network by backpropagation.

Equation 1: The cross-entropy loss function 𝐿 is

defined as, Figure 1. Structure of self-attention-BERT

−1
N C Where 𝛼is the learning rate and ∇𝜃𝐿 is the
L= ∑ ∑ y ( i , j )∗lo g ( p ( i , j ) ) …(1)
N i=1 j=1
gradient of the loss function with respect to the
model parameters for that process. The BERTweet
Where 𝑁 is the number of samples in the dataset,
fine-tuning process will be used together with
𝐶 is the number of classes (positive, negative,
other components of the proposed sentiment
analysis framework, including multimodal
sample 𝑖 and class 𝑗, and p(i , j) is the predicted
neutral), y (i , j) is the ground truth label for
integration and attention mechanisms. While
probability of sample 𝑖 belonging to class 𝑗in the
BERTweet captures the textual features of Twitter
data, the multimodal integration enables the
process.During training, the model parameters are incorporation of other modalities, like images and
user metadata samples. Attention mechanisms also apparent from textual data alone.The multimodal
enhance the ability of the model to selectively fusion process is further enhanced using attention
focus its attention on the most relevant parts of the mechanisms, which dynamically weight the
input text, hence improving performance in contributions of different modalities based on their
sentiment analysis. relevance via equation 5,

∂ L ∂ L ∂ p ∂h N
=
∂θ ∂ p ∂ h ∂ θ
… (3) M =∑ αi∗Mi …(5)
'

Where, 𝑝 is the predicted probability, ℎ is the

i=1

hidden state, and 𝜃 represents the model multimodal representation, 𝑀𝑖 represents the
Where,𝑀′ represents the attention-weighted

parameters for this process.The MM-BERTweet individual modality representations, and 𝛼𝑖

approach infuses textual information from the represents the attention weights computed based
tweets with other modalities, such as images and on the relevance of each modality to the sentiment
user metadata, in order to improve predictions in analysis task.The MM-BERTweet model
sentiment analysis tasks. Each modality is complements other components of the sentiment
encoded separately by using models specifically analysis framework by enhancing the contextual
designed for that modality. For textual understanding of tweets through multimodal
information, the BERTweet model is used to integration process. While BERTweet captures
create contextualized representations, while visual textual features, multimodal integration allows for
features from the images could be extracted with the incorporation of visual and user-related cues,
convolutional neural networks. In the encoding of providing a more comprehensive view of tweet
user metadata, feature extraction techniques or content and sentiment. This synergistic approach
embedding models could be used, including leverages the strengths of both textual and non-
profile descriptions and other engagement metrics. textual information sources, leading to improved
sentiment analysis performance.Finally, the
Once the representations for each modality are
sentiment prediction process involves applying a
obtained, they are concatenated or fused to create
classifier, such as a softmax layer, to the
a unified multimodal representation for each
multimodal representation to predict sentiment
tweet. This fusion process may involve
labels via equation 6,
concatenating the feature vectors from each
modality or applying attention mechanisms to
y ' =softmax ( W M ' + b ) …(6)
selectively combine information from different
Where, 𝑦’ represents the predicted sentiment
modalities based on their relevance to the
probabilities, 𝑊 and 𝑏 represent the weight
sentiment analysis task via equation 4,

M =[ T ; I ;U ] … (4) matrix and bias vector of the softmax layer,

respectively. The softmax function normalizes the
𝑀 output scores to obtain probabilities for each
representation, 𝑇 represents the textual features
Where, represents the multimodal
sentiment class. Next, the Attention-Gated
obtained from BERTweet, 𝐼 represents the visual
BERTweet (AG-BERTweet) process enhances
features extracted from images, and 𝑈 represents
sentiment predictions by incorporating attention
mechanisms into the BERTweet architecture. This
the user metadata features. The concatenation
process involves selectively focusing on important
operator [⋅] is used to combine the features from
words or phrases within the input text, allowing
different modalities.The choice of the MM-
the model to allocate more attention to relevant
BERTweet model is justified by its ability to
information while disregarding noise and
leverage diverse sources of information inherent
irrelevant content sets. The attention gating
in Twitter data, leading to richer contextual
mechanism dynamically adjusts the contribution
representations and more accurate sentiment
of each token in the input sequence based on its
predictions. By incorporating additional
importance to the sentiment analysis task.The
modalities such as images or user metadata, the
attention gating mechanism is mathematically
model gains a deeper understanding of tweet
represented via equation 7,
content and user interactions, enabling it to
capture nuanced sentiment signals that may not be
 Ai=σ ( Wa∗hi+ba ) …(7) model is evaluated in terms of different metrics,

Where, 𝐴𝑖 represents the attention weight for

and can be observed from the next section of this

token 𝑖, ℎ𝑖 represents the hidden state of token 𝑖

text.

obtained from the BERTweet model, 𝑊𝑎 and 𝑏𝑎 4. Result Analysis

attention gate, respectively, and 𝜎represents the

represent the weight matrix and bias vector of the
The proposed sentiment analysis framework was
sigmoid activation function.The choice of the AG- evaluated on a dataset consisting of 10,000 labeled
BERTweetmodel is motivated by its ability to tweets collected from various domains, including
selectively attend to relevant parts of the input news, entertainment, and social media samples.
text, thereby improving the model's ability to The dataset was split into training, validation, and
capture important information for sentiment test sets with a ratio of 70:15:15. Experiments
classification. By focusing on salient words or were conducted on a machine with an Intel Core
phrases, the model can mitigate the influence of i7 processor and 16GB of RAM, utilizing the
noise and irrelevant content present in tweets, PyTorch framework for model implementation
leading to more refined sentiment predictions.The and training.
refined hidden state ℎ𝑖′ for each token after Table 1: Comparison of Sentiment Analysis
applying attention gating is computed via equation Accuracy
8,
'
h i =Ai ⋅hi ​…(8) Method Accuracy (%)

token 𝑖 after applying attention gating, 𝐴𝑖

Where, ℎ𝑖′ represents the refined hidden state of Proposed 88.5

represents the attention weight for token 𝑖, and ℎ𝑖

represents the original hidden state of token 𝑖for
[2] 83.2
this process.The AG-BERTweet model
complements other components of the sentiment [6] 79.6
analysis framework by enhancing the model's
ability to capture important sentiment-related [14] 82.9
features within the input text. While BERTweet
captures contextual information from the entire
input sequence, attention gating allows the model Table 1 compares the accuracy of sentiment
to focus more on salient words or phrases relevant analysis achieved by the proposed model and
to sentiment analysis. This synergistic approach three baseline methods ([2], [6], [14]). The
leverages the strengths of both contextual proposed model outperforms all baseline methods
understanding and selective attention, leading to with an accuracy of 88.5%, showcasing its
improved sentiment analysis performance.Finally, effectiveness in capturing nuanced sentiment
the refined hidden states ℎ𝑖′ are aggregated to signals in Twitter data samples. This enhancement
obtain a single representation for the entire input in accuracy indicates the superiority of the
sequence, which is then used for sentiment proposed framework in discerning sentiment
classification via equation 9, expressions across diverse domains.

Table 2: Comparison of Precision, Recall, and

y =softmax ( Wc ⋅avg ( h ) ) …(9)
' '
F1-score
Where, 𝑦’ represents the predicted sentiment
probabilities, 𝑊𝑐 represents the weight matrix Method Precision Recall F1-score
for sentiment classification, avg(ℎ′) represents the (%) (%) (%)
average of the refined hidden states ℎ𝑖′, and the
softmax function normalizes the output scores to Proposed 89.2 87.6 88.4
obtain probabilities for each sentiment class.
Based on this, tweets are classified into sentiment
[2] 85.6 82.1 83.7
classes for different scenarios. Performance of this
 [6] 80.9 78.2 79.5 [2] 80 50

[14] 84.3 81.5 82.8 [6] 85 60

Table 2 presents the precision, recall, and F1- [14] 75 45

score achieved by the proposed model and
baseline methods. The proposed model Table 4 assesses the resource utilization of the
demonstrates superior precision, recall, and F1- proposed model and baseline methods in terms of
score compared to all baseline methods, indicating GPU memory utilization and CPU utilization. The
its ability to achieve a balanced performance in proposed model demonstrates optimal resource
correctly identifying positive, negative, and utilization, with lower GPU memory and CPU
neutral sentiment tweets. This improvement in utilization compared to all baseline methods. This
precision and recall reflects the enhanced efficient resource management highlights the
capability of the proposed framework in capturing scalability and adaptability of the proposed
sentiment nuances. framework to different hardware configurations
and deployment environments.The superior
Table 3: Comparison of Computational
performance of the proposed sentiment analysis
Efficiency
framework signifies its potential to revolutionize
sentiment analysis applications across various
Method Inference Time Training Time domains. With enhanced accuracy, precision,
(ms) (hours) recall, and computational efficiency, the proposed
framework offers unprecedented capabilities in
Proposed 15 6 deciphering sentiment expressions in real-time
Twitter data samples. These advancements hold
significant implications for industries relying on
[2] 20 8 sentiment analysis, including market research,
brand monitoring, and public opinion analysis,
[6] 25 10 enabling more informed decisionmaking and
strategic planning.
[14] 18 7 5. Conclusion and Future Scopes

Table 3 evaluates the computational efficiency of In conclusion, the proposed sentiment analysis
the proposed model and baseline methods in terms framework, integrating BERTweet-FT, MMBert,
of inference time and training time. The proposed and AGBert, demonstrates remarkable efficacy in
model achieves the lowest inference time of 15 extracting nuanced sentiment signals from Twitter
milliseconds and requires the shortest training data samples. Through meticulous
time of 6 hours compared to all baseline methods. experimentation and analysis, our framework
This improvement in computational efficiency consistently outperforms baseline methods in
underscores the practical applicability of the terms of accuracy, precision, recall, and
proposed framework in real-time sentiment computational efficiency. The substantial
analysis applications. improvements observed underscore the
significance of our approach in advancing the
Table 4: Comparison of Resource Utilization state-of-the-art in sentiment analysis.The
integration of BERTweet-FT enables our
Method GPU Memory CPU framework to capture contextual nuances inherent
in Twitter language, laying a strong foundation for
Utilization (%) Utilization
sentiment analysis. Leveraging multimodal
(%)
information through MMBert enhances the
contextual understanding by incorporating textual,
Proposed 70 40 visual, and user-related cues, leading to more
comprehensive sentiment predictions. (BLC);stochastic gradient descent (SGD);bag-
Furthermore, AGBert refines sentiment of-words (BoW);deep sentiment analysis
predictions by selectively attending to important (DSA)},
words or phrases, mitigating noise and irrelevant [4] Q. Yang, Z. Kadeer, W. Gu, W. Sun and A.
information in tweets. Wumaier, "Affective Knowledge Augmented
Interactive Graph Convolutional Network for
Future Scope: Chinese-Oriented Aspect-Based Sentiment
Analysis," in IEEE Access, vol. 10, pp.
Future work could focus on implementing 130686-130698, 2022, doi:
advanced pre-training techniques, incorporating 10.1109/ACCESS.2022.3228299.
multimodal data like audio and video, and keywords: {Feature
applying ensemble learning methods to enhance extraction;China;Sentimentanalysis;Syntactics
sentiment analysis accuracy. Additionally, ;Datamining;Commonsensereasoning;Semanti
adapting the framework for different social media cs;Convolutional neural networks;Graph
platforms and languages through transfer learning neural networks;Aspect-based sentiment
could expand its utility and effectiveness. analysis;Chinese sentiment analysis;external
affective knowledge;graph convolutional
6. References networks;multi-head self-attention},
[1] M. Aljebreen, B. Alabduallah, M. M. Asiri, A. [5] Q. Yang, Z. Kadeer, W. Gu, W. Sun and A.
S. Salama, M. Assiri and S. S. Ibrahim, "Moth Wumaier, "Affective Knowledge Augmented
Flame Optimization With Hybrid Deep Interactive Graph Convolutional Network for
Learning Based Sentiment Classification Chinese-Oriented Aspect-Based Sentiment
Toward ChatGPT on Twitter," in IEEE Analysis," in IEEE Access, vol. 10, pp.
Access, vol. 11, pp. 104984-104991, 2023, 130686-130698, 2022, doi:
doi: 10.1109/ACCESS.2023.3315609. 10.1109/ACCESS.2022.3228299.
keywords: {Chatbots;Social networking keywords: {Feature
(online);Artificial extraction;China;Sentimentanalysis;Syntactics
intelligence;Blogs;Featureextraction;Sentimen ;Datamining;Commonsensereasoning;Semanti
tanalysis;Emojis;Artificialintelligence;ChatGP cs;Convolutional neural networks;Graph
T;sentimentanalysis;deeplearning;machinelear neural networks;Aspect-based sentiment
ning;data classification}, analysis;Chinese sentiment analysis;external
[2] V. D. Prabha and R. Rathipriya, "Competitive affective knowledge;graph convolutional
Capsule Network Based Sentiment Analysis networks;multi-head self-attention},
on Twitter COVID'19 Vaccines," in Journal of [6] M. U. Hoque et al., "Analyzing Tweeting
Web Engineering, vol. 21, no. 5, pp. 1583- Patterns and Public Engagement on Twitter
1601, July 2022, doi: 10.13052/jwe1540- During the Recognition Period of the COVID-
9589.2159. 19 Pandemic: A Study of Two U.S. States," in
keywords: {COVID-19;Sentiment IEEE Access, vol. 10, pp. 72879-72894, 2022,
analysis;Social networking doi: 10.1109/ACCESS.2022.3189670.
(online);Pandemics;Blogs;Government;Vaccin keywords: {Social networking
es;Twitter;sentimentanalysis;deeplearning;cap (online);COVID-19;Pandemics;Blogs;Public
sule network;COVID-19;machine language}, healthcare;Media;Sentiment analysis;COVID-
[3] P. Durga and D. Godavarthi, "Deep- 19;geospatial data analysis;natural language
Sentiment: An Effective Deep Sentiment processing;publicengagement;public health
Analysis Using a Decision-Based Recurrent messaging;sentimentanalysis;statisticalanalysi
Neural Network (D-RNN)," in IEEE Access, s;Twitter data analytics},
vol. 11, pp. 108433-108447, 2023, doi: [7] S. Urolagin and S. Patel, "User-Specific
10.1109/ACCESS.2023.3320738. Loyalty Measure and Prediction Using Deep
keywords: {Sentiment Neural Network From Twitter Data," in IEEE
analysis;Featureextraction;Analyticalmodels;S Transactions on Computational Social
ocial networking Systems, vol. 11, no. 1, pp. 1046-1061, Feb.
(online);Blogs;Brainmodeling;Trainingdata;Se 2024, doi: 10.1109/TCSS.2023.3239523.
ntimentanalysis;BERT-large-cased
 keywords: {Social networking 122242, 2022, doi:
(online);Blogs;Measurement;Business;Sentim 10.1109/ACCESS.2022.3222233.
entanalysis;Neuralnetworks;Electroniclearning keywords: {Sentiment analysis;Social
;Customer loyalty prediction;e- networking (online);Deep learning;Bit error
learningplatforms;satisfaction score (SATS) rate;Dataaugmentation;Dictionaries;Annotatio
per user;sentimentanalysis;user sentiment ns;Dataaugmentation;deeplearning;fine-
score (USS) per user}, tuning;idiomaticexpressions;lexicon;sentiment
[8] N. Braig, A. Benz, S. Voth, J. Breitenbach and analysis},
R. Buettner, "Machine Learning Techniques [12] J. He, A. Wumaier, Z. Kadeer, W. Sun, X.
for Sentiment Analysis of COVID-19-Related Xin and L. Zheng, "A Local and Global
Twitter Data," in IEEE Access, vol. 11, pp. Context Focus Multilingual Learning Model
14778-14803, 2023, doi: for Aspect-Based Sentiment Analysis," in
10.1109/ACCESS.2023.3242234. IEEE Access, vol. 10, pp. 84135-84146, 2022,
keywords: {COVID-19;Social networking doi: 10.1109/ACCESS.2022.3197218.
(online);Sentiment keywords: {Context
analysis;Blogs;Pandemics;Behavioralsciences; modeling;Contextawareness;Featureextraction
Machinelearning;Socialsciences;Behavioral ;Sentimentanalysis;Semantics;Linguistics;Asp
science;COVID-19;deep ect-based sentiment analysis;Chinese
learning;machinelearning;sentimentanalysis;so sentiment analysis;multilingualABSA;local
cialscience;twitter}, and global context focus},
[9] T. Zhang, X. Gong and C. L. P. Chen, "BMT- [13] K. Jahanbin and M. A. Z. Chahooki,
Net: Broad Multitask Transformer Network "Aspect-Based Sentiment Analysis of Twitter
for Sentiment Analysis," in IEEE Transactions Influencers to Predict the Trend of
on Cybernetics, vol. 52, no. 7, pp. 6232-6243, Cryptocurrencies Based on Hybrid Deep
July 2022, doi: 10.1109/TCYB.2021.3050508. Transfer Learning Models," in IEEE Access,
keywords: {Task vol. 11, pp. 121656-121670, 2023, doi:
analysis;Sentimentanalysis;Featureextraction; 10.1109/ACCESS.2023.3327060.
Contextmodeling;Bit error keywords: {Market
rate;Analyticalmodels;Social networking research;Sentimentanalysis;Bitcoin;Social
(online);Attention mechanism;broad learning networking (online);Analytical
system (BLS);multitask learning models;Transferlearning;Predictivemodels;As
(MTL);representation learning;sentiment pect based sentiment analysis;prediction trend
analysis}, price;cryptocurrencies;pre-
[10] Z. Kastrati, A. S. Imran, S. M. Daudpota, trainednetworks;hybrid deep learning
M. A. Memon and M. Kastrati, "Soaring models},
Energy Prices: Understanding Public [14] T. Ma, H. Rong, Y. Hao, J. Cao, Y. Tian
Engagement on Twitter Using Sentiment and M. Al-Rodhaan, "A Novel Sentiment
Analysis and Topic Modeling With Polarity Detection Framework for Chinese," in
Transformers," in IEEE Access, vol. 11, pp. IEEE Transactions on Affective Computing,
26541-26553, 2023, doi: vol. 13, no. 1, pp. 60-74, 1 Jan.march 2022,
10.1109/ACCESS.2023.3257283. doi: 10.1109/TAFFC.2019.2932061.
keywords: {Social networking keywords: {Feature
(online);Blogs;Sentimentanalysis;Transformer extraction;Vocabulary;Twitter;Sentimentanaly
s;Renewable energy sources;Bit error sis;Taskanalysis;Informationscience;Indexes;C
rate;Analyticalmodels;Energymanagement;Po hinese sentiment polarity detection;Chinese
wermarkets;Pricing;Sentimentanalysis;energy sentiment lexicon;opinionmining;natural
price language processing},
hike;topicmodeling;transformers;BERT;Twitte [15] İ. Aygün, B. Kaya and M. Kaya, "Aspect
r;LDA;BERTopic}, Based Twitter Sentiment Analysis on
[11] B. M. A. Tahayna, R. K. Ayyasamy and R. Vaccination and Vaccine Types in COVID-19
Akbar, "Automatic Sentiment Annotation of Pandemic With Deep Learning," in IEEE
Idiomatic Expressions for Sentiment Analysis Journal of Biomedical and Health Informatics,
Task," in IEEE Access, vol. 10, pp. 122234-
 vol. 26, no. 5, pp. 2360-2369, May 2022, doi:
10.1109/JBHI.2021.3133103.
keywords: {Vaccines;COVID-
19;Pandemics;Social networking
(online);Sentiment analysis;Blogs;Biomedical
measurement;COVID-19;sentiment
analysis;BERT;text mining;COVID-19
vaccine;deep learning, process}