Stock Price Prediction Using RNN and LSTM

Sijie Tang, Xiaofei Luo, Hamdan Ahmed

1. Introduction

In the dynamic world of financial markets, accurately predicting stock prices is a goal
that captures the interest and ambition of investors and analysts alike. The challenge
lies in deciphering the intricate patterns and trends hidden within market data to make
well-informed decisions that could significantly influence financial outcomes. Among the
various methodologies employed in this pursuit, Long Short-Term Memory (LSTM)
networks and Recurrent Neural Networks (RNNs) stand out for their effectiveness in
dealing with sequential data.

LSTM networks are an advanced type of RNN, specifically designed to address the
limitations of traditional RNNs by better handling long-term dependencies. These
networks are equipped with special units that can maintain information in memory for
long periods, making them exceptionally adept at recognizing and remembering
patterns over extended sequences. This capability is particularly beneficial in the
context of stock price prediction, where understanding long-term trends and the
influence of historical data is crucial.

RNNs, with their unique architecture that allows information to persist, have
revolutionized the way we approach problems involving sequential data. Unlike models
that treat every data point independently, RNNs consider the sequence as a whole,
learning and remembering the context from previously encountered data. This feature
makes RNNs particularly valuable for predicting the future values of time series data,
such as stock prices, by capturing the complex relationships within the data over time.

This exploration aims to shed light on the advantages of utilizing LSTM and RNN
models for stock price forecasting. We will examine the distinctive strengths of each
model to understand better situations in which one might be more advantageous than
the other with hands-on experiments. (All source code can be found here :
https://colab.research.google.com/drive/16Ux9o3_tbw8brKRRZL-YdLi9yuQaqtR_#scroll
To=_AJEFoDS1Qu). Through this analysis, we aspire to contribute valuable insights to
the ongoing discussion on effective predictive strategies in the volatile and
unpredictable realm of financial markets, thereby aiding in the refinement of
decision-making processes.

2. Data Preprocessing and Feature Engineering

For our analysis of stock market trends, we utilized the Alpha Vantage free API, a
comprehensive tool providing real-time and historical stock data. Alpha Vantage is
renowned for its wide array of financial data, including stock prices, trading volumes,
and technical indicators, making it an invaluable resource for investors and researchers
alike. By leveraging a Python script, we efficiently downloaded the necessary stock
data, converting it into a .CSV file format for easy manipulation and analysis. This
streamlined approach enabled us to gather and organize large datasets, setting the
foundation for our predictive modeling with LSTM and RNN, and facilitating a deeper
exploration into stock price behaviors.

a. Missing Value Imputation:

In the initial exploration of the collected datasets, a critical concern emerged
regarding the presence of null values. Recognizing the potential disruption these
null values could cause to the genuine pattern of price movement, a strategic
approach for handling them was imperative. To address this issue efficiently, a
widely accepted method was employed – the replacement of null values with the
mean of the previous 30 days' prices. This approach not only mitigated the
impact of missing data but also ensured the preservation of the underlying
temporal dynamics essential for accurate modeling.

b. Normalization / Feature Scaling:

In pursuit of enhancing the model's robustness and convergence during training,
the MinMaxScaler was applied to normalize and scale the features within the
dataset. Normalization is a crucial step, particularly when dealing with diverse
features that may have varying scales. The MinMaxScaler method was chosen
for its ability to transform the data into a standardized range, ensuring that each
feature contributes proportionally to the overall learning process without being
disproportionately influenced by its original scale.
 c. Creating Training Data and Testing Data:
To facilitate effective model training and evaluation, the raw data underwent a
meticulous process of splitting, taking into account the temporal or ratio-based
considerations. This step ensured the creation of distinct training and testing
datasets, preserving the temporal order of the data to simulate real-world
conditions accurately.

d. Creating x_train and y_train from Training Data:

With the segmented training data in place, the subsequent step involved creating
the input (x_train) and output (y_train) sequences for the model. Specifically
designed for time series analysis, the x_train dataset comprised 30 time steps,
capturing the historical patterns leading up to the prediction point. Meanwhile, the
y_train dataset encapsulated the single output corresponding to the subsequent
time step, forming a coherent pairing essential for training the model to
understand sequential dependencies.

e. Reshape:
Recognizing the input requirements of RNNs, LSTMs, a critical transformation
was implemented to convert the training data into a 3-D array format. This
reshaping process aligns with the architectural demands of these recurrent
neural networks, where the input structure involves a three-dimensional array,
facilitating the extraction of temporal dependencies within the dataset. This
reshaped format ensures compatibility with the intricacies of these advanced
neural network architectures, paving the way for effective time series analysis
and prediction.

3. Models: Vanilla RNN, LSTM

Vanilla Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM)
networks are pivotal players in the domain of sequential data processing, each offering
distinctive advantages in the context of stock price prediction. The Vanilla RNN, while
conceptually powerful, faces challenges in effectively capturing long-term dependencies
due to the vanishing gradient problem. In contrast, LSTM networks are designed to
mitigate this issue by incorporating memory cells, allowing them to selectively retain or
discard information over extended sequences. The LSTM's ability to maintain a more
robust memory makes it well-suited for modeling complex relationships in time series
data, such as stock prices, where historical context plays a crucial role. As we explore
these models, we aim to unravel their strengths and limitations, providing insights into
the nuances that influence their performance in predicting the dynamic and often
unpredictable nature of financial markets.

4. Training, Prediction, and Performance Analysis

1) Train the Model:

a) Utilize the .sequential() function to construct the neural network's
architecture, defining the model's structure. This function acts as the
pipeline for adding layers to the model.

b) Employ the .compile() function to compile the model, specifying the loss
function, optimization method, and evaluation metric. The loss function
assesses the model's performance, while the optimizer works to minimize
this loss during training.

c) Use the .fit() function to execute the actual model training, leveraging the
compiled architecture and training data.

Here, we set 25 for epoch since it is the turning point that validation error
 started to increase instead of decreasing as epoch size became larger
when we tried different epoch size.

2) Predict using Trained Model:

a) Once the model is trained, deploy it to make predictions on new or unseen
data, utilizing the knowledge gained during the training phase.

3) Prediction Performance Analysis: Evaluate the performance of the trained

model using regression-related metrics:

● R^2 (R squared):
A measure of how well the model's predictions match the actual values.

● Mean Squared Error (MSE):

The average of the squared errors between predicted and actual values.

● Root Mean Squared Error (RMSE):

The square root of MSE, giving more weight to larger errors.

● Mean Absolute Error (MAE):

The average of the absolute errors, considering only the magnitude of
errors.
These metrics provide a comprehensive analysis of the model's accuracy and reliability
in predicting continuous variables, particularly relevant in scenarios like stock price
prediction. Additionally, the evaluation includes Mean Absolute Percentage Error
(MAPE), which focuses on the percentage deviation, providing insights into the model's
effectiveness in handling both positive and negative errors. Finally, the R squared value
serves as a measure of the model's risk when predicting financial asset prices, offering
valuable information about the model's overall performance.

5. Results and Conclusion

The application of Recurrent Neural Networks (RNN) and Long Short-Term Memory
(LSTM) networks for stock price prediction yielded noteworthy insights into their
respective performances. The evaluation metrics provide a comprehensive overview of
the models' accuracy and reliability in capturing the intricate dynamics of financial
markets.

Model R Squared Mean Absolute Error Mean Squared Error Root Mean Squared Error
RNN 0.9255 1.6413 4.4485 2.1091
LSTM 0.9012 1.9412 5.9011 2.4292

The R squared values, indicative of the goodness of fit, showcase a high level of
accuracy for both RNN and LSTM models. With R squared values of 0.9255 and 0.9012
respectively, these models effectively capture the variability in the stock price data,
demonstrating their ability to explain a significant portion of the observed fluctuations.
Examining the mean absolute error (MAE), mean squared error (MSE), and root mean
squared error (RMSE), we observe consistently low values for both RNN and LSTM
models. Lower values in these metrics suggest that the models exhibit minimal
prediction errors, reinforcing their efficacy in forecasting stock prices.

The fact that based on metrics above RNN performs better than LSTM here is
interesting, since LSTMs are known to be the preferred model between the two for
sequential data like stock prices. There could be two main reasons behind that : dataset
characteristics and hyperparameter tuning. The characteristics of the stock price
dataset might align better with the capabilities of a basic RNN compared to an LSTM. In
this particular stock price dataset which dates back to 25 years ago, current stock
closing price is primarily influenced by the relatively immediate past rather than
historical events that happened long ago as a result of COVID-19 pandemic outbreak.
Which means this dataset is of short-term dependency. However, LSTM only performs
better than RNN for capturing long-term dependencies. Another factor could play a role
is hyperparameter values like learning rate, batch size, number of layers, and number of
units. In the above experiment, we used the same set of hyperparameters for both
models. And that choice of hyperparameters may be more optimal for RNN rather than
LSTM. We also tried to tune the hyperparameters with both models but have not found
the hyperparameters set that would be optimal for LSTM yet by the time this report is
written.
In conclusion, both RNN and LSTM models prove to be formidable tools for stock price
prediction. While RNN excels with a slightly higher R squared value, LSTM maintains
competitive performance across all metrics. The choice between these models may
depend on specific use cases or preferences. Overall, the achieved results underscore
the potential of deep learning techniques in capturing the complexities of financial
markets, paving the way for informed decision-making in stock trading and investment
scenarios.

6. Future work

While the current study provides valuable insights into the application of Recurrent
Neural Networks (RNN) and Long Short-Term Memory (LSTM) networks for stock price
prediction, there are several avenues for future research and enhancements:

1) Model Fine-Tuning and Hyperparameter Optimization:

Explore further refinement of model architectures and hyperparameters to
achieve even higher accuracy. This may involve an exhaustive search for optimal
configurations or the utilization of advanced techniques such as Bayesian
optimization.

2) Ensemble Models:
Investigate the potential benefits of ensemble modeling by combining predictions
from multiple models. Ensemble methods, such as stacking or blending, could
harness the strengths of both RNN and LSTM to create a more robust and
accurate forecasting system.

3) Real-time Prediction:
Adapt the models for real-time prediction to cater to the dynamic nature of
financial markets. This would involve optimizing the model architecture and
training process to handle streaming data effectively.

By addressing these areas in future research, it is possible to enhance the effectiveness

and applicability of deep learning models in the domain of stock price prediction and
contribute to the broader field of financial forecasting.

References

1. Fischer, T.; Krauss, C. Deep learning with long short-term memory networks for
financial market predictions. Eur. J. Oper. Res.2018,270, 654–669.
2. LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature 2015,521, 436–444.
3. Nabipour, M.; Nayyeri, P.; Jabani, H.; Mosavi, A.; Salwana, E. Deep learning for
stock market prediction. Entropy 2020,22, 840.
4. Yong, B.X.; Rahim, M.R.A.; Abdullah, A.S. A stock market trading system using
deep neural network. In Proceedings of theAsian Simulation Conference,
Melaka, Malaysia, 27–29 August 2017; pp. 356–364.