Volume 9, Issue 2, February 2024 International Journal of Innovative Science and Research Technology

ISSN No:-2456-2165

Sentiment Analysis in Financial Markets

Ashwini K. Bhavsar; Tejaswini H. Gavhane
Computer Engineering Department
Pimpri Chinchwad College of Engineering & Research Pune, Maharashtra

Abstract:- Moving ahead in this era of data, there is a lot financial domain is a leverage and helps gain profits and
of information, which if used in the right way, can be understand the market well. Hence nowadays, Algorithms
used in the financial domain as well, to determine the are used to analyse the public opinions and determine if the
market. This prediction can lead to large profits and news in the flow is going to impact the market positively or
help in understanding the complex financial markets. negatively. Text mining is used to extract features and
Sentiment analysis is a kind of data mining technique, perform sentiment analysis in order to obtain meaningful
which can be used to process and understand the textual conclusions from otherwise random and unstructured data.
content to derive meaningful insights. In this paper, for The unstructured data in the public domain is made useful
the purpose of sentiment analysis, natural language by using the data mining techniques. There is a really large
processing will be used, which is the area of machine amount of data, that cannot be easily processed or used to
learning in the rise. The techniques will be applied here form conclusions for which Text mining and Natural
onto a large dataset from Twitter and hence, analyse the Language Processing are used. Sentiment Analysis is a
public opinions about the financial markets. structured investigation of public opinion and emotions
towards topics, events, individuals and companies (Liu,
Keywords:- Predicting finances, Natural Language 2012). It has been known that decisions made by humans is
Processing(NLP),Financial Markets, Analysis of Sentiments, impacted by the emotions and thoughts about topics, hence
Mining Text. the best way to predict the decisions in advance is to analyse
those emotions, which affect important sectors such as
I. INTRODUCTION financial sectors. Also, it has been noted that the news in the
flow can impact to a great extent the future decisions,
Firstly, to answer why sentiment analysis is necessary especially in the context of financial trades, if a company is
in the domain of finance, it is important to understand that going to be at loss or profit is hugely affected by the news in
the financial condition is controlled by the market, and a the public and altogether emotions of the people about that
loss in the market eventually leads to the loss of the public company. Apart from the opinions of individuals about a
and investors. Analysing the market, gives one the topic, the news broadcasted by the news channels on twitter
advantage to invest safely and reduce the chances of also constitutes a great information. Several algorithms for
monetary loss to the largest extent. Such analysis helps the the purpose of sentiment analysis have been proposed, even
investors in investing smartly and prevent a collapse of the in the financial domain to predict the future market activities
financial situation. Therefore, one of the most emerging (Hagenau, Liebmann and Neumann, 2013) and to draw a
sectors of the country is quantitative research or HFT (high correspondence between public emotions and market
frequency trading).In the early 1950s, the research in the performances (Wisniewski and Lambe, 2013). One of the
field of Quantitative Finance was started with time series major challenges faced by the researchers in the sentiment
analysis, where a lot of past data with Open, High, Low, analysis for financial data is the phrases and terms specific
Close prices were stored, further machine learning and to the domain and its true apprehension. To deal with that,
statistical techniques such as mean reversion, ARIMA are domain specific lexicons were introduced (Yekrangi and
applied on to the dataset to predict the future prices of the Abdolvand, 2020). This lexicon-based method is a hybrid of
stocks. To observe a better and more precise forecast, corpus based (Kamps et al., 2004) and dictionary based
machine learning algorithms like RL (Reinforcement natural language processing method in order to derive useful
Learning), Gated Recurrent Network (GRU) and LSTM and valid information related to financial domain. Domain
(Long Short-Term Memory) are used nowadays. These specific lexicons are used and defined for the sentiment
techniques are popular in machine learning and are known analysis. In the further of this paper, we will first review the
to predict the future in a more accurate way than other literature, followed by the working of natural language
models or statistical methods. In these researches, the data processing, encompassing sentiment analysis through text
is numerical and only deals with the technical data, but it mining and other technical details. Further, future
was discovered later that besides the historical trends, the discussions and research on this topic will end with the
public opinions and news, which is present in the form of paper's conclusion.
textual data also shapes and changes the market affectively.
The news and discussions around a particular stock and the
general public opinion also impact the financial markets to a
great extent. Besides the technological advantage of having
access to information as early as possible, the ability to
process the information and take advantage of it in the

IJISRT24FEB087 www.ijisrt.com 83
Volume 9, Issue 2, February 2024 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
II. LITERATURE REVIEW 2011) found a strong association. Social media data is a
great source of unstructured data for forecasting market
Sentiment Analysis was first introduced in the paper trends and attitude shifts because of its volume and
Nasukawa, T. And Yi, J. ,2003, which mainly proposed it as immediacy.Sentiment analysis has also been used in forums
a technique to deal with the semantic-pragmatic gap. and online communities, in addition to Twitter. Discussions
Sentiment analysis applied to financial news articles has in online communities might yield useful sentiment
been the subject of numerous research. (Ding et al., 2014) indicators that shed light on investor opinion and possible
found that emotions in financial news have a big effect on market moves, according to research by (Audrino, Sigrist
stock prices. These results highlight how sentiment analysis and Ballinari, 2020). Though sentiment analysis has
might improve market movement prediction modelling. potential, there are obstacles in the financial sector.
Furthermore, algorithmic trading algorithms have used Sentiment analysis is challenging in the financial domain
sentiment analysis in financial news(Bollen, Mao and Zeng, because financial terminology is context-specific and
2011)to investigate whether sentiment on Twitter might be ambiguous. Moreover, erroneous signals and market noise
used efficiently to forecast changes and trends in the might result in poor investment choices. Context-aware
financial market, demonstrating the viability of trading sentiment analysis algorithms are one of the complex
techniques based on sentiment. The emergence of social methodologies needed to address these difficulties.
media platforms has given financial sentiment analysis a Researchers are investigating techniques like sentiment
new angle. An analysis of the link between stock market lexicons tailored to financial contexts to improve accuracy
results and Twitter sentiment by (Zhang, Fuehres and Gloor, and exclude sentiment that isn't relevant.

Table 1: Objective/Methodology
Author Objective/Methodology Finding /Observation
11. Proposed sentiment analysis Sentiment analysis is now widely used in the natural language processing
domain in order to analyse texts in various contexts.
3. Twitter Sentimental analysis They explored how the opinions formed on social media platforms are a major
step in financial trading and affect the stock market.
9,10 Used SVM for sentiment analysis It gave an accuracy of approximately 71.837 % , which is quite an achievement
for a simple classifier-based algorithm.
1. Used Naïve Bayes for sentiment Improved the classification accuracy to 90.3 %
analysis

III. METHODOLOGY or public opinion, identifying the sentiments, selecting

important features, which is also called feature selection,
Sentiment analysis is a classification task; it has classifying those sentiments into different categories and
several methods and ways in which it can be performed. The then further polarising those classifications, as illustrated in
process can include various steps such as Collecting the data Figure 1.

Fig. 1: Sentiment Analysis Steps and Processes(User generated figure)

IJISRT24FEB087 www.ijisrt.com 84
Volume 9, Issue 2, February 2024 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
Sentence-level classification, document-level the two sorts of approaches that are Lexicon-based. Further
classification, and aspect-level classification are the three statistical or semantic approaches can be used for corpus-
main categories in sentiment analysis. Sentences are a form based approaches.Firstly, we need to perform preprocessing
of documents, just smaller in size, hence there is little on the textual data, procedures such as removing stop words,
distinction between sentence-level and document-level removing words with too high frequency or too low
categorization. There are majorly two major techniques in frequency, and performing stemming, which is defined as
sentiment analysis, namely, the method using lexicon extracting the meaningful word out of another form ofthe
andanother approach using machine learning. Both same word, such as if the word is running, it is stored as run,
supervised and unsupervised machine learning techniques to get the actual meaning of the word and understand the
are possible, with supervised learning offering many context of use.
categories.Techniques based on dictionary and corpus are

Fig. 2: Sentiment Classification Techniques(User generated figure

Following the use of these pre-processing approaches, based approach is applied. The popular machine learning
we proceed to feature selection, utilising characteristics like: algorithms leverage linguistic characteristics. The technique
 Term frequency: This characteristic is word n-grams, based on lexicon makes use of a sentiment lexicon, that is a
and the count of those words normally utilises term set of well-known sentiment expressions that have been
frequency to assign relative value to the features. precompiled and are specific to a certain situation. There are
 Part of Speech (POS): Locate the adjectives, as they are several models and methods available in the machine
a crucial signifier of viewpoints. learning approach to apply supervised learning such as:
 Opinion words and phrases: Identifying terms that are  Naïve Bayes classifier: This classifier, which evaluates a
particular to the text's context and that may be utilised to class's probability using the document's word
assess whether an opinion is favourable or unfavourable. distribution., is the most basic and often used one.
 Negations: When a statement contains a negation, such Because this model uses Bag of Words (BOW) feature
as "not good" being equal to "bad," the opinion extraction, it does not take word position into account.
orientation might be altered. This is not accurate enough due to the negligence of the
context in which the word is being used.(A. Jabbar
The document must first be converted into word Alkubaisi, Kamaruddin and Husni, 2018). This method
embeddings for feature selection to be applied. Various resulted into a high accuracy of 90.38 % , which enables
techniques for word embedding include Global Vectors for a higher extent of correct prediction in the financial
Word Representation (GloVe), Word2Vec, and Bag of sector. A hybrid algorithm of naïve bayes was also used,
Words (BOW); any one of these methods can be used; the for further improvement.
discussion for the right method of embedding is out of the
scope of this topic. After the features are extracted, either
the machine-learning approach is applied, or the lexicon-

IJISRT24FEB087 www.ijisrt.com 85
Volume 9, Issue 2, February 2024 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
Table 2: Model Accuracy parameters*

*From A. Jabbar Alkubaisi, Kamaruddin and Husni, 2018.

 Support Vector Machines: The SVMs work on the principle of separation between the two classes and the distance between
them when plotted onto a plane. Because text is sparse, text data are well-suited for SVM classification. Therefore, they can be
easily separated into linearly separable categories.(Mullen and Collier, n.d.)This method when applied on financial prediction
on Nigerian bank data, gave an accuracy of 71.837% (Onwuegbuche, Wafula and Mung’atu, 2019)
 Decision Tree: This method usually consists of dividing the data space with the leaf nodes using a condition based on which
the data is divided.
 Neural Networks: For classifying categories, neural networks are used in general and hence can be utilised for the task of
sentiment analysis as well, although the outcomes are not very different from what the support vector machines can
predict.(Sohangir et al., 2018). Using Convolution Neural Networks, gave approximately 86 % accuracy , which was
comparatively better than LSTMs , which turned out to be ineffective.
 Pre-trained models such as BERT, GPT, etc.

Fig. 3: Correlation matrix of predicted sentiment and market*

*from Georgios Fatouros et al. (2023)

Using models such as GPT gives a high correlation, are complex and involve a variety of factors, which include
which typically suggests that there is a high correlation socioeconomic conditions and other factors. analysing the
present between the market values and the sentiments sentiments present in social media about certain topics can
predicted by these models. However, it must be noted that help determine the general public opinion but sometimes
the high correlation does not necessarily imply predictive can be biased or favoured.
power over future market prices. Financial market systems

IJISRT24FEB087 www.ijisrt.com 86
Volume 9, Issue 2, February 2024 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
IV. RESULTS

From the above discussion, it is clear that various models have been tested and following results have been observed-

Table 3: Classic ML model performance *(Karanikola et al., 2023)

Table 4: Deep Learning Model Performance Comparisons *(Karanikola et al., 2023)

V. FUTURE RESEARCH prolonged processing time when dealing with complex data.
Further, language models can be developed that leverage
As the research is advancing in this domain, more and their complete potential in finance as well.Hence the future
more language models are being utilised to sense the market research work can be explored in the domains of language
and hence language models such as GPT(Georgios Fatouros models , efficient memory and processing time management
et al., 2023) are being used, but instead of domain-specific as well as other advancements in the natural language
news headlines, it will be better if unrelated news is also fed processing methods.
into the modelas sometimes various other factors also play a
significant role in deciding the fate of stocks and VI. CONCLUSION
trades.Apart from this, the data duration that we can analyse
right now is comparatively small for a volatile domain such In conclusion, this paper explores all the possible ways
as finance; hence, in future studies, a longer duration of data of implementing natural language processing in analysing
can validate the findings and predictions.It is crucial to the sentiments in predicting the financial market
expand our research horizon, as with the passing of time, we movements. It first explores the sentiment anlysis , followed
are getting introduced to newer and more capable language by analysing thesignificance of applying sentiment analysis
models such as OpenAI’s ChatGPT, which is trained using to the financial domain, followed by the basic NLP
real-world data and relies less on synthetically generated techniques required to do so. Analysing sentiments of the
datasets. As the research is also advancing in the domain of market helps make smart decisions and is also attractive for
LLMs, future research can explore options by comparing investors as it prevents huge market losses and financial
existing and emerging models.A major issue that needs to be collapses. The literature review emphasisesthe fact that the
addressed in the future is the response time, which is large previous research in this domain has been done mainly in
when dealing with large amounts of data. Models often face the quantitative data and is now slowly progressing to
fluctuations in system loads and a memory shortage or a analysing public opinion, as it has been realised that the

IJISRT24FEB087 www.ijisrt.com 87
Volume 9, Issue 2, February 2024 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
unstructured data present on social media can be exploited https://www.cs.uic.edu/~liub/FBS/SentimentAnalysis-
to get useful insights about the market. and-OpinionMining.pdf.
Themethodologyfocuses on the various steps that are [10]. Mullen, T. and Collier, N. (n.d.). Sentiment analysis
required in sentiment analysis: preprocessing techniques, using support vector machines with diverse
feature selection methods and model exploration. After information sources. [online] Available at:
cleaning the unstructured data and transforming it into https://aclanthology.org/W04-3253.pdf.
desired word embeddings, various models are to be [11]. Onwuegbuche, F.C., Wafula, J.M. and Mung’atu, J.K.
explored, such as support vector machines and language (2019). Support Vector Machine for Sentiment
models such as BERT, GPT, RoBERTa, etc.Further, the Analysis of Nigerian Banks Financial Tweets. Journal
future scope of research in the domain has been discussed, of Data Analysis and Information Processing, 07(04),
describing how memory issues with the existing models pp.153–173.
persist and how analysing a larger amount of data can lead doi:https://doi.org/10.4236/jdaip.2019.74010.
to better analysis and prediction as the finance industry is [12]. Sentiment Analysis Capturing Favorability Using
indeed a complex one and is affected by constantly various Natural Language Processing. Proceedings of the 2nd
topics. International Conference on Knowledge Capture,
Florida, 23-25 October 2003, 70-77. - References -
REFERENCES Scientific Research Publishing, n.d.)
[13]. Sohangir, S., Wang, D., Pomeranets, A. and
[1]. A.JabbarAlkubaisi, G.A., Kamaruddin, S.S. and Husni, Khoshgoftaar, T.M. (2018). Big Data: Deep Learning
H. (2018). Stock Market Classification Model Using for financial sentiment analysis. Journal of Big Data,
Sentiment Analysis on Twitter Based on Hybrid Naive 5(1). doi:https://doi.org/10.1186/s40537-017-0111-6.
Bayes Classifiers. Computer and Information Science, [14]. Wisniewski, T.P. and Lambe, B. (2013). The role of
11(1), p.52. doi:https://doi.org/10.5539/cis.v11n1p52. media in the credit crunch: The case of the banking
[2]. Audrino, F., Sigrist, F. and Ballinari, D. (2020). The sector. Journal of Economic Behavior& Organization,
impact of sentiment and attention measures on stock 85, pp.163–175.
market volatility. International Journal of Forecasting, doi:https://doi.org/10.1016/j.jebo.2011.10.012.
[online] 36(2), pp.334–357. [15]. www.scirp.org. (n.d.). Nasukawa, T. and Yi, J. (2003)
doi:https://doi.org/10.1016/j.ijforecast.2019.05.010. Sentiment Analysis Capturing Favorability Using
[3]. Bollen, J., Mao, H. and Zeng, X. (2011). Twitter mood Natural Language Processing. Proceedings of the 2nd
predicts the stock market. Journal of Computational International Conference on Knowledge Capture,
Science, 2(1), pp.1–8. Florida, 23-25 October 2003, 70-77. - References -
doi:https://doi.org/10.1016/j.jocs.2010.12.007. Scientific Research Publishing. [online] Available at:
[4]. Ding, X., Zhang, Y., Liu, T. and Duan, J. (2014). https://www.scirp.org/reference/referencespapers?refer
Using Structured Events to Predict Stock Price enceid=1830388 [Accessed 10 Jan. 2024].
Movement: An Empirical Investigation. [online] [16]. Yekrangi, M. and Abdolvand, N. (2020). Financial
Association for Computational Linguistics, pp.1415– markets sentiment analysis: developing a specialized
1425. Available at: https://aclanthology.org/D14- Lexicon. Journal of Intelligent Information Systems,
1148.pdf. 57(1), pp.127–146. doi:https://doi.org/10.1007/s10844-
[5]. Georgios Fatouros, Soldatos, J., KalliopiKouroumali, 020-00630-9.
Makridis, G. and Kyriazis, D. (2023). Transforming [17]. Zhang, X., Fuehres, H. and Gloor, P.A. (2011).
Sentiment Analysis in the Financial Domain with Predicting Stock Market Indicators Through Twitter ‘I
ChatGPT. arXiv (Cornell University). hope it is not as bad as I fear’. Procedia - Social and
doi:https://doi.org/10.1016/j.mlwa.2023.100508. Behavioral Sciences, 26, pp.55–62.
[6]. Hagenau, M., Liebmann, M. and Neumann, D. (2013). doi:https://doi.org/10.1016/j.sbspro.2011.10.562.
Automated news reading: Stock price prediction based
on financial news using context-capturing features.
Decision Support Systems, 55(3), pp.685–697.
doi:https://doi.org/10.1016/j.dss.2013.02.006.
[7]. Kamps, J., Marx, M., Mokken, R.J. and de Rijke, M.
(2004). Using WordNet to Measure Semantic
Orientations of Adjectives. [online] ACLWeb.
Available at: https://aclanthology.org/L04-1473/
[Accessed 10 Jan. 2024].
[8]. Karanikola, A., Davrazos, G., Liapis, C.M. and
Kotsiantis, S. (2023). Financial sentiment analysis:
Classic methods vs. deep learning models. Intelligent
Decision Technologies, [online] 17(4), pp.893–915.
doi:https://doi.org/10.3233/idt-230478.
[9]. Liu, B. (2012). Sentiment Analysis and Opinion
Mining. [online] Available at:

IJISRT24FEB087 www.ijisrt.com 88