ENHANCED DEEP LEARNING-BASED STOCK PREDICTION PLATFORM

USING BILSTM FOR ACCURATE MARKET ANALYSIS

Shivashankar S
Associate Professor
Department of Information Technology
Sri Krishna College of Engineering &
Technology, Coimbatore
[email protected]

Mohamed Hasheem N Naveen Krishna A S Praveen K

Student Student Student
Department of Information Technology Department of Information Technology Department of Information Technology
Sri Krishna College of Engineering & Sri Krishna College of Engineering & Sri Krishna College of Engineering &
Technology, Coimbatore Technology, Coimbatore Technology, Coimbatore
[email protected] [email protected] [email protected]

ABSTRACT 1. INTRODUCTION

Since deep learning algorithms can identify Over the past few decades, the ability to
complex patterns in market data, there has been an accurately predict stock prices has been a top
increasing interest in using them for financial priority for analysts, financiers, and financial
prediction in recent years. In this study, the stock institutions. Since trillions of dollars' worth of
price movements of multiple markets were predicted transactions take place every day in today's financial
using a Bidirectional Long Short-Term Memory markets, even a small increase in prediction
(BiLSTM) neural network. A forecast model that accuracy can have a huge financial impact. Because
can spot patterns and trends in stock prices was financial time-series data is complex, dynamic, and
developed using data from Yahoo Finance. The nonlinear, stock price prediction is challenging in
experiment demonstrated the model's effectiveness and of itself. A number of factors, including market
in stock price forecasting by performing well on the sentiment, company performance, geopolitical
test set. The findings have implications for risk events, and economic conditions, influence stock
assessment and financial decision-making and prices. The interdependence of the variables is not
emphasize the significance of using deep learning taken into consideration by conventional prediction
techniques for stock price prediction.This methods, particularly since market trends frequently
experiment successfully predicted stock price deviate from theoretical forecasts.The analysis and
patterns with good accuracy by modeling complex prediction of financial data has been completely
temporal relationships using a Bidirectional Long transformed in recent years by advancements in
Short-Term Memory (BiLSTM) neural network. machine learning (ML) and deep learning (DL). The
Enhancing the model's resistance to various market techniques that have performed the best with
conditions and using sentiment analysis to more sequential data, such as stock prices, are Recurrent
accurately forecast future events are two more areas Neural Networks (RNNs) and their variations, Long
for future research. Overall, this study improves Short-Term Memory (LSTM) and Bidirectional
stock price forecasting techniques, facilitating well- Long Short-Term Memory (BiLSTM). They are
informed financial choices. very good at seeing trends over time and using
historical data to forecast future movement. Second,
Keywords: Deep learning; BiLSTM, stock price by adding sentiment from news, models can better
prediction; financial decision-making; risk understand market sentiment and how it influences
management. changes in stock prices. Our platform attempts to
provide a higher predictive ability to enable better
informed financial decisions by utilizing sentiment data that is received in sequences, like stock prices
and historical stock data. over time. The ability of RNNs to "remember"
previous inputs sets them apart from other models
and enables them to recognize patterns that depend
on earlier data points. However, there are some
1.1 STOCK PREDICTION
issues with standard RNNs that make it hard for
them to learn from long-term patterns, like the
Because precise predictions can yield enormous
vanishing gradient problem. Because of this flaw,
profits for traders and investors, stock price
they are unable to predict the long-term, erratic, and
forecasting is very popular in the financial markets.
frequently intricate changes in stock prices.
However, a variety of factors, including global
events, company performance, economic indicators, 1.4 BILSTM
and market sentiment, cause the stock market to
fluctuate continuously. Because of this, it is
By processing data both forward and
challenging to predict prices with precision. Because
backward, Bidirectional Long Short-Term Memory
they can't keep up with the ever-changing dynamics
(BiLSTM) networks overcome the drawbacks of
of stock prices, traditional methods like
conventional RNNs. The model is particularly well-
autoregression and linear regression often produce
suited for applications like stock price forecasting
inaccurate forecasts. As a result, more and more
because of its bidirectional nature, which aids in
people are thinking about advanced deep learning
taking dependencies from previous and future time
and machine learning techniques. The methods are
steps into account. BiLSTM networks can more
perfect for analyzing financial data over time
effectively capture complex temporal patterns in
because they can identify complex patterns and
stock price data by utilizing bidirectional processing
adapt to new information.
and the best features of LSTMs. The memory door,
input door, and output door are the three doors that
make up an LSTM cell. Keep in mind that the output
1.2 DEEP LEARNING gate's output is not the LSTM cell's final output. HT
and CT are the LSTM cell's final outputs.
A subset of machine learning called deep
learning has revolutionized a number of sectors, The three yellow boxes in the image above that are
including finance, by offering fresh approaches to labeled "sigma" are these three doors. The amount
challenging issues. Deep learning models are of information that should pass through is indicated
particularly helpful for tasks like stock price by the sigmoid layer's output, which ranges from 0
forecasting because they can automatically learn and to 1. We use sentiment analysis of financial news to
extract features from large datasets, setting them increase prediction accuracy even more. Share
apart from traditional approaches. The ability of prices will be impacted by the rich sentiment hints
deep learning algorithms, particularly those based about the market outlook that are often revealed in
on neural networks, to recognize nonlinear patterns news reports. For example, if the company has good
and temporal dependencies in data is essential for news, its stock price will rise, and if it has bad news,
simulating the behavior of the stock market. Deep it will fall. We use VADER and TextBlob to obtain
learning algorithms have become a popular tool for sentiment scores, which we then feed into the
financial prediction because of their ability to handle BiLSTM model using historical stock data.
enormous volumes of data and adapt to changing
market conditions.

1.3 RECURRENT NEURAL NETWORK (RNN)

A type of deep learning called recurrent

neural networks (RNNs) was created to work with
 disadvantages of various strategies, and the
contribution of sentiment analysis and Bidirectional
Long Short-Term Memory (BiLSTM) networks to
modern financial forecasting are all covered in this
literature review. For many years, predicting stock
prices has been a significant problem in data science
Figure 1: LSTM Structure Model and finance. Researchers have been able to achieve
significantly higher accuracy and performance in
BiLSTM is made up of forward LSTM and financial forecasting thanks to deep learning models,
backward LSTM for stock trend prediction. The specifically Recurrent Neural Networks (RNNs) and
input time series is fed into the LSTM model in the their later evolution, Long Short-Term Memory
original order in the forward LSTM layer. The input (LSTM) and Bidirectional LSTM
time series is fed into the LSTM model in reverse (BiLSTM).Furthermore, it has been demonstrated
order in the backward LSTM layer. By connecting that combining social media and news sentiment
the two LSTMS to the same output layer, this analysis is a successful strategy for improving model
structure is able to extract the time series' performance. Regarding sentiment analysis, hybrid
bidirectional relationship. As a result, theoretical models, and deep learning architectures for stock
prediction performance ought to outperform one- price prediction, this overview offers the best
way LSTM, and the particular BiLSTM expression relevant literature in this field. Siyuan Wang (2024)
is displayed below. highlighted that traditional statistical models, such
as Generalized Auto-Regressive Conditional
Heteroskedasticity (GARCH) and Auto-Regressive
Integrated Moving Average (ARIMA), are unable to
identify complex temporal structures and non-linear
relationships in stock price variation.

Wang's research employed a Bidirectional LSTM

(BiLSTM) model to predict the stock prices of
Apple Inc., demonstrating the model's ability to
where σ is the activation function and Ht is process historical data in both directions. By
the hidden layer input. By updating the forward simultaneously documenting short-term and long-
structure and the reverse structure takes the final term dependencies, this approach significantly
input. increased precision. With a Mean Squared Error
(MSE) of 24.37 and a Root Mean Squared Error
(RMSE) of 4.93, the study outperformed
conventional models. These findings support the
claim that deep learning techniques—more
especially, BiLSTM—outperform conventional
statistical techniques [1]. For improved stock price
prediction accuracy, Xu & Purkayastha (2024)
presented an Attention-BiLSTM model with
Empirical Mode Decomposition (EMD) and
investor sentiment analysis.To extract underlying
Figure 2: Working of BiLSTM Model trends, they divided the stock price data into
different time periods and supplemented it with
2. LITERATURE REVIEW
market sentiment gleaned from financial news.
From traditional statistical methods to
By paying attention, the model was able to improve
advanced machine learning and deep learning
predictive accuracy by taking into account only the
algorithms, the field of stock price forecasting has
most significant time steps. This study demonstrated
experienced significant change over time. The
that the hybrid model outperformed the traditional
evolution of techniques, the advantages and
LSTM and BiLSTM models, especially when VADER for sentiment extraction and news
market trends were erratic. The system's ability to sentiment usage. Because it gives price-based
adaptively shift its focus through the use of attention models a psychological component, the study
layers greatly enhanced its capacity to predict underlined the importance of taking external market
sudden changes in the market. For improved stock sentiment into account when forecasting financial
market prediction, this study suggests combining outcomes.
temporal learning with sentiment sources outside of
the domain [2]. Chen and Zhao (2022) talked about Zhang et al. (2024) proposed a hybrid model that
using deep learning models to forecast stock prices uses Gradient Boosting Decision Trees (GBDT) and
using sentiment analysis. The study demonstrated BiLSTM for multi-factor stock prediction. The
that adding sentiment from financial news improved hybrid model used technical indicators, sentiment
the model's ability to forecast market trends.The information, and macroeconomic indicators to get
study forecasted stock returns using sentiment an overall view of stock movement. In times of
analysis techniques in conjunction with LSTM economic shock, hybrid models outperformed
models. BiLSTM single models. The paper concludes that by
leveraging the advantages of tree-based and
After adding sentiment data and price histories, they sequence-based learning, ensemble models may be
were able to reduce the error rate by 15%. The able to produce more accurate predictions [5].
results of this study support the idea that qualitative
information from sources other than the model— 3. RELATED WORK
such as market sentiment—can improve the
performance of quantitative financial models. Applications of machine learning and deep
According to the authors, combining multiple learning to stock price forecasting have been
sentiment analysis techniques (such as VADER and extensively studied; researchers are looking into a
TextBlob) results in predictions that are more number of models and techniques to improve
precise and context-specific [3]. By adding LSTM forecast accuracy. With a focus on Bidirectional
and attention mechanisms, Zhao & Wang (2022) Long Short-Term Memory (BiLSTM) networks and
presented an improved model for learning sentiment analysis in financial prediction, the
significant temporal features in finance. In order to section that follows highlights some of the key
optimize the system's ability to capture market research and methodologies that are relevant to this
shocks and emerging trends, the model was created project. Because of their ability to handle sequence
to dynamically assign weights to valuable inputs. data, deep learning algorithms—more especially,
They demonstrated that, when compared to standard Recurrent Neural Networks (RNNs) and their
BiLSTM networks, the Attention-BiLSTM model variations—have gained popularity for stock price
reduced prediction errors by 15%. This was prediction. Wang et al. (2019) demonstrated the
particularly true during periods of high volatility, effectiveness of LSTM-based neural networks in
when the majority of traditional models perform predicting stock market patterns, underscoring their
poorly. capacity to identify long-term dependencies in time-
series data. The vanishing gradient problem that
The researchers hypothesized that adding plagues traditional RNNs is resolved by LSTMs,
multi-source data, such as sentiment and technical increasing their efficacy for financial forecasting.
data, would make the model even stronger [4]. The However, the ability of regular LSTMs to learn
impact of stock price fluctuations subsequent to dependencies across future time steps is limited, and
financial news coverage was the focus of Patel & they only update information in one direction
Rajan (2019). They developed a deep learning (forward). In order to enhance the performance of
model that combined time-series models and text- deep learning models in the task of stock price
based sentiment analysis. Their findings showed that forecasting, attention mechanisms have also been
when stock prices rise, positive sentiment usually investigated. These mechanisms improve models'
follows, and when stock prices fall, negative ability to learn crucial temporal features by enabling
sentiment usually follows. Researchers increased the them to selectively focus on the most significant
prediction model's accuracy by 10% by using areas in the input. By focusing on the most
significant temporal features, Zhao and Wang information. We can spot trends and patterns in the
(2022) used attention mechanisms in neural stock movement by using this historical data. We
networks to significantly increase the accuracy of also use the NewsAPI to scrape social media posts
stock price prediction.Their study demonstrated how and financial news stories to get a sense of market
attention mechanisms can improve the model's sentiment. After that, we analyze this text data to
ability to identify and exploit trends in changes in determine the investors' emotions, which provides
stock prices. us with additional data for our model. To improve
the dataset's quality, we preprocess and clean it. To
4. METHODOLOGY avoid bias, we handle missing values, normalize
numerical variables, and encode categorical
The main topic of this paper is how to variables. In order to improve the model's efficacy,
forecast stock price time series using the BiLSTM we also eliminate any features that are superfluous.
model. We can use this potent deep learning We keep open, high, low, and volume as auxiliary
technique to more accurately predict stock prices features, but our primary focus is on closing prices.
with the right data preparation, model building, and We normalize data using MinMaxScaler to scale
training optimizations. However, please keep in everything from 0 to 1 in order to level the playing
mind that there is always some degree of uncertainty field for all stock prices. Because it is on a larger
in any prediction because stock prices are impacted scale, this keeps any feature from overpowering the
by a wide range of complex factors and the market model. We use TextBlob and VADER to generate
itself can change significantly. For readers interested sentiment scores for sentiment analysis of news and
in stock price prediction, it provides some references Twitter data, which we then add as extra features to
and motivation. We anticipate more opportunities show how stock prices may be impacted by market
for stock price forecasting in the future as deep sentiment.
learning technology continues to advance.Every
stage, from data collection to model evaluation, is C. FEATURE EXTRACTION
crucial to ensuring that the model can handle
intricate, non-linear financial data and incorporates We incorporate additional features from the raw data
market sentiment for improved prediction to improve the model's predictive ability. These new
performance. features enhance the model's comprehension of the
data and its capacity to identify patterns in stock
A. DATA COLLECTION price fluctuations. We compute and add
fundamental technical indicators, such as the
We gather information from various sources for this Moving Averages (MA10, MA30) and Relative
project in order to determine the factors influencing Strength Index (RSI), to the dataset. The model can
stock prices. Our primary source of historical stock thus observe momentum and changes in stock
price information is Yahoo Finance, which includes prices. We also incorporate sentiment scores from
trading volumes as well as opening, high, low, and financial news and social media as extra features.
closing prices. We can use this data to train our D. TRAINING AND TESTING
model to identify patterns in the direction of stock
prices. We use the NewsAPI to retrieve financial To train the BiLSTM model and evaluate
news articles and social media posts in order to its performance, the preprocessed, feature-rich
assess market sentiment. After that, we analyze this dataset is separated into training and test sets. The
text to measure investor sentiment and incorporate it three main layers are used to build the BiLSTM
as an extra input into our model. model. Bidirectional LSTM layers utilize
B. DATA PRE-PROCESSING dependencies from both past and future time steps
by processing the input data both forward and
We gather data for this project from a backward. Dropout Layers, which randomly disable
variety of sources in order to investigate the factors a percentage of neurons during training, are used to
that affect stock prices. Yahoo Finance provides the prevent overfitting. To predict the stock price, the
majority of the data, which includes historical stock final output layer uses a dense layer. The Adam
prices along with open, high, low, close, and volume optimizer, which modifies the learning rate during
training to improve convergence, is used to train on
the training dataset. The loss function, which
computes the difference between expected and
actual stock prices, is Mean Squared Error (MSE).
After training, the model's performance is evaluated
using the testing dataset.
D. MODEL EVALUATION

A range of performance metrics are used to evaluate

the accuracy and dependability of the BiLSTM
model.

The average squared differences between actual and Figure 3: Closing Price (AAPL)
predicted stock prices are measured by the Mean
Squared Error, or MSE. A low MSE improves the
model.

The root The mean squared error, or RMSE, is

simply the MSE squared. In the same unit as the
stock prices, it provides us with a sense of how
accurate the forecasts are. It is widely used in
finance and is simpler to understand than MSE.

The average of the absolute discrepancies between

predicted and actual stock prices is known as the
mean absolute error, or MAE. It's merely a simple
metric to determine how inaccurate the projections Figure 4: Comparison of 10-Day and 30-Day
are. Moving Averages

Sentiment Impact Analysis: Using model results

with and without sentiment data, we investigate how
sentiment analysis affects model performance. It
demonstrates the necessity of market sentiment for
stock price prediction.

5. RESULT ANALYSIS

We use a system that combines sentiment

analysis and a Bidirectional Long Short-Term
Memory (BiLSTM) network to predict stock prices.
In this section, we compare actual and predicted
stock prices, track important metrics, and see how
adding sentiment data affects the model's
forecasting accuracy. We plan to compare the new Figure 5: Predicted vs Actual Prices
stock prediction method with traditional forecasting
methods. The primary focus is on whether and how
well the model detects complex patterns over time,
as well as how much sentiment information aids in 6. CONCLUSION
prediction.
In this study, we developed and
implemented an improved deep learning-based
stock prediction platform that integrates sentiment
analysis and Bidirectional Long Short-Term 381, pp. 01017, 2023, doi:
Memory (BiLSTM) networks to improve the 10.1051/matecconf/202338101017.
accuracy of stock price forecasting. This study
sought to address the limitations of statistical models [7] "Design of Multilayered Bidirectional LSTM
by applying deep learning techniques that capture Network for Stock Price Prediction Using DIF
complex temporal dependencies and market Features," by D. R. Babu and B. Sathyanarayana, in
sentiment. Unlike traditional unidirectional models, International Journal of Engineering Sciences &
BiLSTM processes input sequences both forward Research Technology, vol. 12, no. 12, pp. 1-13,
and backward, allowing it to capture the entire 2023.
temporal context of stock price movements. This
bidirectional nature reduces the error rate and [8] "Contrasting the Efficiency of Stock Price
enhances the capacity to recognize stock price Prediction Models Using Various Types of LSTM
turning points for real-time decision-making. Our Models Aided with Sentiment Analysis," by V.
platform combines sentiment signals from financial Sangwan, V. K. Singh, and B. C. V, in arXiv
news with historical stock prices to generate more preprint arXiv:2307.07868, 2023.
accurate forecasts.
[9] "Stock Market Prediction Using News Sentiment
7. REFERENCES Analysis," by M. H. Alostad and M. Davuluri, in
2020 IEEE International Conference on Big Data
[1] "Predicting Stock Prices with FinBERT-LSTM: (Big Data), pp. 5370-5372, 2020.[10] "An LSTM-
Integrating News Sentiment Analysis," by W. Gu, GRU Based Hybrid Framework for Secured Stock
Y. Zhong, S. Li, C. Wei, L. Dong, Z. Wang, and C. Price Prediction," by G. R. Patra and M. N.
Yan, in arXiv preprint arXiv:2407.16150, 2024. Mohanty, in Journal of Statistics and Management
Systems, vol. 25, no. 6, pp. 1491-1499,2022.
[2] "A hybrid model integrating deep learning with
investor sentiment analysis for stock price [10] "The performance of LSTM and BiLSTM in
prediction," by N. Jing, Z. Wu, and H. Wang, in forecasting time series," by S. Siami-Namini, N.
Expert Systems with Applications, vol. 178, pp. Tavakoli, and A. S. Namin in 2019 IEEE
115019, 2021, doi: 10.1016/j.eswa.2021.115019. International Conference on Big Data (Big Data),
2019:IEEE,pp.3285-3292.
[3] "Sentiment analysis for event-based stock price
predictions using bidirectional long short term
memory," by O. Setyawan and H. F. Pardede, in
Journal of Information System, Informatics and
Computing, vol. 6, no. 1, pp. 50-58, 2022, doi:
10.52362/jisicom.v6i1.772.

[4] "Stock Market Prediction Based on BiLSTM,"

by S. Wang, in Transactions on Computer Science
and Intelligent Systems Research, vol. 5, pp. 1030-
1034, 2024, doi: 10.62051/1gf0ac18.

[5] "Enhancing Stock Price Prediction through

Attention-BiLSTM and Investor Sentiment
Analysis," by K. Xu and B. Purkayastha, in
Academic Journal of Sociology and Management,
vol. 2, no. 6, pp. 14-18, 2024, doi:
10.5281/zenodo.14065931.

[6] "A hybrid sentiment based stock price prediction

model using machine learning," by A. Mehmood
and M. K. Ali, in MATEC Web of Conferences, vol.