STOCK PRICE PREDICTION USING LSTM,RNN

AND CNN-SLIDING WINDOW MODEL

Sreelekshmy Selvin, Vinayakumar R, Gopalakrishnan E.A, Vijay Krishna Menon, Soman K.P
Centre for Computational Engineering and Networking (CEN),
Amrita School of Engineering,Coimbatore
Amrita Vishwa Vidyapeetham, Amrita University,India
Email:[email protected]

Abstract—Stock market or equity market have a pro- • Fundamental Analysis

found impact in today’s economy. A rise or fall in the • Technical Analysis
share price has an important role in determining the in- • Time Series Forecasting
vestor’s gain. The existing forecasting methods make use
of both linear (AR,MA,ARIMA) and non-linear algorithms Fundamental analysis is a type of investment analysis where
(ARCH,GARCH,Neural Networks),but they focus on predicting the share value of a company is estimated by analyzing its
the stock index movement or price forecasting for a single sales,earnings,profits and other economic factors. This method
company using the daily closing price. The proposed method
is a model independent approach. Here we are not fitting the is most suited for long term forecasting. Technical analysis
data to a specific model, rather we are identifying the latent uses the historical price of stocks for identifying the future
dynamics existing in the data using deep learning architectures. price. Moving average is a commonly used algorithm for
In this work we use three different deep learning architectures for technical analysis. It can be considered as the unweighted
the price prediction of NSE listed companies and compares their mean of past n data points. This method is suitable for short
performance. We are applying a sliding window approach for
predicting future values on a short term basis.The performance term predictions. The third method is the analysis of time
of the models were quantified using percentage error. series data. It involves basically two classes of algorithms,
Index Terms—Time series, Stock market, RNN,LSTM,CNN they are
• Linear Models
I. I NTRODUCTION • Non Linear Models
Forecasting can be defined as the prediction of some future The different linear models are AR,ARMA,ARIMA and its
event or events by analyzing the historical data. It spans many variations [2] [3] [4]. These models uses some predefined
areas including business and industry, economics, environmen- equations to fit a mathematical model to a univariate time
tal science and finance. Forecasting problems can be classified series. The main disadvantage of these models is that, they
as do not account for the latent dynamics existing in the data.
• Short term forecasting (prediction for few Since they consider only univariate time series, the inter
seconds,minutes,days, weeks or months) dependencies among the various stocks are not identified by
• Medium term forecasting (prediction for 1 to 2 years) these models.Also the model identified for one series will not
• Long term forecasting (prediction beyond 2years) fit for the other. Due to these reasons, it is not possible to
Many of the forecasting problems involve the analysis of identify the patterns or dynamics present in the data as a
time . A time series data can be defined as a chronological whole.
sequence of observations for a selected variable. In our case Non-linear models involve methods like ARCH,GARCH,[3]
the variable is stock price.It can either be univariate or TAR,Deep learning algorithms [5].In [6] an analysis on the
multivariate. Univariate data includes information about only inter dependency between stock price and stock volume for
one particular stock whereas multivariate data includes stock 29 selected companies listed in NIFTY 50 has been done.
prices of more than one company for various instances of The poposed work focuses on the application of deep learning
time. Analysis of time series data helps in identifying patterns, algorithms for stock price prediction [7] [8]. Deep neural net-
trends and periods or cycles existing in the data. In the case works can be considered as non-linear function approximators
of stock market, an early knowledge of the bullish or bearish which are capable of mapping non-linear functions . Based on
mode helps in investing money wisely.Also the analysis of the type of application, various types of deep neural network
patterns helps in identifying the best performing companies architectures are used. These include multi layer perceptrons
for a specified period. This makes time series analysis and (MLP),Recursive Neural Networks (RNN),Long Short Term
forecasting an important area of research. Memory(LSTM), CNN(Convolutional Neural Network) etc
The existing methods for stock price forecasting can be [9].They have been applied in various areas like image pro-
classified as follows[1] cessing, natural language processing, time series analysis etc.

978-1-5090-6367-3/17/$31.00 ©2017 IEEE 1643

Deep learning algorithms are capable of identifying hidden data overlap. The data set contains minute wise data of NSE
patterns and underlying dynamics in the data through a self listed companies. Here we are trying to obtain a generalized
learning process. model for the purpose of prediction which can use minute
In the case of stock market, the data generated is enormous wise data as input. This kind of modeling have applications
and is highly non-linear. To model such kind of dynamical in algorithmic trading where high frequency trading occurs.
data we need models that can analyze the hidden patterns and The paper is structured as follows section[II] explains the
underlying dynamics. Deep learning algorithms are capable proposed methodology.Results and discussions can be found
of identifying and exploiting the interactions and patterns in Section[III] and Section[IV] includes the conclusion.
existing in a data through a self learning process. Unlike other
II. M ETHODOLOGY
algorithms, deep learning models can effectively model these
type of data and can give a good prediction by analyzing the The data set consists of minute wise stock price for 1721
interactions and hidden patterns within the data. In [5], we NSE listed companies for the period of July 2014 to June
can see the application of various deep learning models for 2015. It includes informations like day stamp, time stamp,
multivariate time series analysis. The first attempt to model transaction id, stock price and volume of stock sold in each
a financial time series using a neural network model was minute. For this work we have selected two different sectors,
introduced in [10]. This work made an attempt to model a IT sector and Pharma sector. Two companies from IT sector
neural network model for decoding the nonlinear regularities and one company from Pharma sector were taken for the
in asset price movements for IBM. However the scope of study. These companies were identified by the help of NIFTY-
the work was limited, but it helped in establishing evidences IT index and NIFTY-Pharma index. The data for these three
against EMH [11]. companies were extracted from the available data and was
Researches in the area of financial time series analysis using subjected to preprocessing to obtain the stock price.
NN models used different input variables for predicting the The work is based on a sliding window approach for a
stock return. In some works, data from a single time series short term future prediction. The window size was fixed to
were used as input [10], [8]. Certain works considered the be 100 minutes with an overlap of 90 minute’s information
inclusion of heterogeneous market information and macroe- and prediction was made for 10 minutes in future. The best
conomic variables. In [12], a combination of financial time window length was identified by calculating the error for
series analysis and NLP have been introduced. In [13] and [7], various window sizes. The train data consists of stock price of
deep learning architectures have been used for the modeling Infosys for the period July-01-2014 to October-14-2014 and
of multivariate financial time series. In [14], a NN model test data consists of stock price for Infosys, TCS and CIPLA
using technical analysis variables have been implemented for for the period of October-16-2014 to November-28-2014.
the prediction of Shanghai stock market. The work compared
the performance of two learning algorithms and two weight
initialization methods. The results sh6wn that efficiency of
back propagation can be increased by conjugate gradient Fig. 1: Proposed Model Block Diagram
learning with multiple linear regression weight initialization.
In 1996, [15] used back propagation and RNN models for The data varies with in a range of 2000 to 4000 for Infosys
the prediction of stock index for five different stock markets. and TCS and for Cipla it is 400 to 700. To unify the data
In [16], application of time delay, recurrent and probabilistic range, it was subjected to normalization and was mapped to a
neural network models were introduced for daily stock pre- range of 0 to 1. This normalized data was given to the network
diction. In [17], application of machine learning algorithms for training. All the models were trained for 1000 epochs by
like PSO and LS-SVM have been used for the prediction of varying the layer size for fine tuning. If the loss (mean squared
S&P 500 stock market. Implementation of genetic algorithms error) for the current epoch is less than the value obtained in
along with neural network models were introduced in [18]. The previous epoch, the weight matrices for that epoch is stored.
work combined the application of both genetic algorithm and After the training process each of these models were tested
artificial neural network for forecasting. In this work weights and the model with least RMSE (Root Mean Squared Error)
for NN was obtained from the genetic algorithm.However, the is taken as the final model for prediction.
prediction accuracy for this model was low. Application of We have used three different deep learning architectures,
wavelet transform for prediction was introduced in [19]. The RNN, LSTM and CNN for this work. RNN is a class of neural
work used wavelet transform to describe short term features in network where connections between the computational units
stock trends. With the introduction of LSTM [20], the analysis form a directed circle. Unlike feed forward networks, RNN
of time dependent data become more efficient. These type of can use their internal memory to process arbitrary sequence
networks have the capability of holding past information. They of inputs. Each of the computing unit in an RNN has a time
have been used in stock price prediction by [8], [7]. varying real valued activation and modifiable weight. RNNs
The proposed method focuses on predicting stock price are created by applying the same set of weights recursively
for NSE (National Stock Exchange) listed companies.The over a graph-like structure. Many of the RNNs use (1) to define
approach we have adopted is a sliding window approach with the values of their hidden units.

1644
 identifying long term dependencies and uses them for future
ht = f (ht−1 , xt ; θ) (1) prediction. However CNN architectures mainly focuses on the
given input sequence and does not use any previous history or
In the case of RNN, the learned model always has the information during the learning process.The motivation behind
same input size, because it is specified in terms of transition testing the models with data from other companies is to check
from one state to another. Also the architecture uses the same for interdependencies among the companies and to understand
transition function with the same parameters at every time the market dynamics.
step. LSTM is a special kind of RNN, introduced in 1997 The train data was normalized. Test data was also subjected
by Hochreiter and Schmidhuber [20] . In the case of LSTM to the same normalization. After obtaining the predicted out-
architecture, the usual hidden layers are replaced with LSTM put, denormalization was applied and percentage error was
cells. The cells are composed of various gates that can control calculated using the available true labels.The error percentage
the input flow. An LSTM cell consists of input gate, cell state, was calculated using (7)
forget gate, and output gate. It also consists of sigmoid layer, h i
tanh layer and point wise multiplication operation.The various i i
abs Xreal − Xpredicted
gates and their functions are as follows ep = i
× 100 (7)
Xreal
• Input gate : Input gate consists of the input.
i
• Cell State : Runs through the entire network and has the where ep is the error percentage,Xreal is the ith real value
i th
ability to add or remove information with the help of and Xpredicted is the i predicted value. Error percentage
gates. gives the magnitude of error present in the output.
• Forget gate layer: Decides the fraction of the information
to be allowed.
III. RESULTS AND DISCUSSION
• Output gate : It consists of the output generated by the
LSTM. The experiment was done for three different deep learning
• Sigmoid layer generates numbers between zero and one, models. The maximum value of error percentage obtained for
describing how much of each component should be let each model is given in Table[I]. From the table it is clear
through. that CNN is giving more accurate results than the other two
• Tanh layer generates a new vector, which will be added models. This is due to the reason that CNN does not depend
to the state. on any previous information for prediction. It uses only the
current window for prediction. This enables the model to
The cell state is updated based on the outputs form the
understand the dynamical changes and patterns occurring in
gates. Mathematically we can represent it using the following
the current window. However in the case of RNN and LSTM,
equations.
it uses information from previous lags to predict the future
instances. Since stock market is a highly dynamical system,
ft = σ(Wf .[ht−1 , xt ] + bf ) (2)
the patterns and dynamics existing with in the system will not
it = σ(Wi .[ht−1 , xt ] + bi ) (3) always be the same. This cause learning problems to LSTM
and RNN architecture and hence the models fails to capture
ct = tanh(Wc .[ht−1 , xt ] + bc ) (4) the dynamical changes accurately.
ot = σ(Wo [ht−1 , xt ] + bo ) (5)
TABLE I: ERROR PERCENTAGE
ht = ot ∗ tanh(ct ) (6) COMPANY RNN LSTM CNN
Infosys 3.90 4.18 2.36
where xt : input vector, ht : output vector, ct : cell state TCS 7.65 7.82 8.96
vector, ft : forget gate vector, it : input gate vector, ot : output Cipla 3.83 3.94 3.63
gate vector and W,b are the parameter matrix and vector.
Convolutional neural networks or CNNs, are a specialized For comparison we have used ARIMA, which is a linear
kind of neural network for processing data that has a known, model used for forecasting.The error percentage obtained for
grid-like topology. This include time-series data, which can the three companies are as follows
be thought of as a 1D and image data, which can be thought
of as a 2D grid of pixels.The network employs a math- TABLE II: ERROR PERCENATGE - ARIMA
ematical operation called convolution and hence known as COMPANY Error Percentage
convolutional neural network. It is a specialized kind of linear Infosys 31.91
operation. Convolutional networks use convolution instead of TCS 21.16
Cipla 36.53
general matrix multiplication in at least one of their layers.
The motivation behind using these three models is to identify
From Table[I] and Table[II] it is clear deep learning models
whether there is any long term dependency existing in the
are outperforming ARIMA.
given data. This can be identified from the performance of
the models. RNN and LSTM architectures are capable of

1645
Fig. 2: Plot for Real value vs Predicted value for INFOSYS Fig. 6: Plot for Real value vs Predicted value for TCS using
using RNN LSTM

Fig. 3: Plot for Real value vs Predicted value for INFOSYS Fig. 7: Plot for Real value vs Predicted value for TCS using
using LSTM CNN

when compared to the other two networks.

In case of Cipla, Fig(8) and Fig (9) ,between the period of

2000 and 6000, it is clear that the predicted values of RNN
and LSTM does not matches with the pattern of original data.
This can be considered as a change in the behavior of the
system. In case of Cipla, Fig(10), we can observe that CNN is
Fig. 4: Plot for Real value vs Predicted value for INFOSYS capable of capturing the changes in the behavior of the stock
using CNN price for the specified period.
It can be seen that CNN network is almost able to capture the
trends and is giving accurate predictions compared to the other
two models. CNN was able to analyze the change in trend
for Infosys, TCS and Cipla. It should also be noticed that we
trained the network with Infosys data for the period of July-01-

Fig. 5: Plot for Real value vs Predicted value for TCS using
RNN

From Fig(2) and Fig(3) it can be observed that both RNN

and LSTM fails to capture the trends and dynamics towards
Fig. 8: Plot for Real value vs Predicted value for CIPLA using
the end (between the time period 9000 to 11000) ie; there is
RNN
a change in the behavior of the stock pattern for that time
window when compared to the previous windows. In case
of CNN it is evident from the Fig(4), that the network is
capable of capturing the changes in the trend for the time
period 9000 to 11000
In the case of TCS, Fig(5) and Fig(6), RNN and LSTM
networks are not identifying the pattern in the beginning
of the window (during the first 1000 minutes). There is a
change in the trend followed by TCS during that period. This
makes the predictions less accurate, whereas in Fig(7), we Fig. 9: Plot for Real value vs Predicted value for CIPLA using
can see that CNN captures these changes more accurately LSTM

1646
 [5] G. Batres-Estrada, “Deep learning for multivariate financial time series,”
ser. Technical Report, Stockholm, May 2015.
[6] P. Abinaya, V. S. Kumar, P. Balasubramanian, and V. K. Menon,
“Measuring stock price and trading volume causality among nifty50
stocks: The toda yamamoto method,” in Advances in Computing, Com-
munications and Informatics (ICACCI), 2016 International Conference
on. IEEE, 2016, pp. 1886–1890.
[7] J. Heaton, N. Polson, and J. Witte, “Deep learning in finance,” arXiv
preprint arXiv:1602.06561, 2016.
[8] H. Jia, “Investigation into the effectiveness of long short term memory
Fig. 10: Plot for Real value vs Predicted value for CIPLA networks for stock price prediction,” arXiv preprint arXiv:1603.07893,
using CNN 2016.
[9] Y. Bengio, I. J. Goodfellow, and A. Courville, “Deep learning,” Nature,
vol. 521, pp. 436–444, 2015.
[10] H. White, Economic Prediction Using Neural Networks: The Case of
2014 to October-14-2014, even then the testing accuracy for IBM Daily Stock Returns, ser. Discussion paper- Department of Eco-
Infosys is lower when compared to the other companies.This nomics University of California San Diego. Department of Economics,
shows that whatever trend Infosys exhibits during the period of University of California, 1988.
[11] B. G. Malkiel, “Efficient market hypothesis,” The New Palgrave: Fi-
July to October 14 is not present in the test data as such(from nance. Norton, New York, pp. 127–134, 1989.
October-16-2014 to November-28-2014) ie; there is a change [12] X. Ding, Y. Zhang, T. Liu, and J. Duan, “Deep learning for event-driven
in the dynamics.This accounts for the difference in the error stock prediction.” in IJCAI, 2015, pp. 2327–2333.
[13] J. Roman and A. Jameel, “Backpropagation and recurrent neural net-
percentage for Infosys when compared to other models. Also works in financial analysis of multiple stock market returns,” in System
the model is capable of predicting stock price for companies Sciences, 1996., Proceedings of the Twenty-Ninth Hawaii International
other than Infosys. This shows that, the pattern or dynamics Conference on,, vol. 2. IEEE, 1996, pp. 454–460.
[14] M.-C. Chan, C.-C. Wong, and C.-C. Lam, “Financial time series fore-
identified by the model is common to other companies also. casting by neural network using conjugate gradient learning algorithm
and multiple linear regression weight initialization,” in Computing in
IV. CONCLUSION Economics and Finance, vol. 61, 2000.
[15] J. Roman and A. Jameel, “Backpropagation and recurrent neural net-
We propose a deep learning based formalization for stock works in financial analysis of multiple stock market returns,” in System
price prediction. It is seen that, deep neural network archi- Sciences, 1996., Proceedings of the Twenty-Ninth Hawaii International
tectures are capable of capturing hidden dynamics and are Conference on,, vol. 2. IEEE, 1996, pp. 454–460.
[16] E. W. Saad, D. V. Prokhorov, and D. C. Wunsch, “Comparative study
able to make predictions. We trained the model using the data of stock trend prediction using time delay, recurrent and probabilistic
of Infosys and was able to predict stock price of Infosys, neural networks,” IEEE Transactions on neural networks, vol. 9, no. 6,
TCS and Cipla. This shows that, the proposed system is pp. 1456–1470, 1998.
[17] O. Hegazy, O. S. Soliman, and M. A. Salam, “A machine learning model
capable of identifying some inter relation with in the data. for stock market prediction,” arXiv preprint arXiv:1402.7351, 2014.
Also, it is evident from the results that, CNN architecture is [18] K.-j. Kim and I. Han, “Genetic algorithms approach to feature discretiza-
capable of identifying the changes in trends. For the proposed tion in artificial neural networks for the prediction of stock price index,”
Expert systems with Applications, vol. 19, no. 2, pp. 125–132, 2000.
methodology CNN is identified as the best model. It uses the [19] Y. Kishikawa and S. Tokinaga, “Prediction of stock trends by using the
information given at a particular instant for prediction. Even wavelet transform and the multi-stage fuzzy inference system optimized
though the other two models are used in many other time by the ga,” IEICE Transactions on Fundamentals of Electronics, Com-
munications and Computer Sciences, vol. 83, no. 2, pp. 357–366, 2000.
dependent data analysis, it is not out performing the CNN [20] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural
architecture in this case. This is due to the sudden changes computation, vol. 9, no. 8, pp. 1735–1780, 1997.
that occurs in stock markets. The changes occuring in the
stock market may not always be in a regular pattern or may
not always follow the same cycle. Based on the companies
and the sectors, the existence of the trends and the period of
their existence will differ. The analysis of these type of trends
and cycles will give more profit for the investors.To analyze
such information we must use networks like CNN as they rely
on the current information.
R EFERENCES
[1] A. V. Devadoss and T. A. A. Ligori, “Forecasting of stock prices using
multi layer perceptron,” Int J Comput Algorithm, vol. 2, pp. 440–449,
2013.
[2] J. G. De Gooijer and R. J. Hyndman, “25 years of time series forecast-
ing,” International journal of forecasting, vol. 22, no. 3, pp. 443–473,
2006.
[3] V. K. Menon, N. C. Vasireddy, S. A. Jami, V. T. N. Pedamallu,
V. Sureshkumar, and K. Soman, “Bulk price forecasting using spark
over nse data set,” in International Conference on Data Mining and Big
Data. Springer, 2016, pp. 137–146.
[4] G. E. Box, G. M. Jenkins, G. C. Reinsel, and G. M. Ljung, Time series
analysis: forecasting and control. John Wiley & Sons, 2015.

1647