Stock Prediction with Machine Learning

Antony Alexos AALEXOS @ UTH . GR

Antony Evmorfopoulos AEVMORFOP @ UTH . GR
Tilemachos Tsiapras TTSIAPRAS @ UTH . GR

Abstract stock price index movement. ANN has its own way to learn
patterns. ANN emulates functioning of our brain to learn
A good approach to predict future price of a
by creating network of neurons. This study focuses on a
stock, based on its past price and other factors
modification of ANN, a Long Short-Term Memory Neural
was investigated. In this paper we try to predict
Network, which is an optimization of RNNs. Here we use
whether the price of a stock will go up or down
the LSTMs(Hochreiter & Schmidhuber, 1997) in a classi-
on the next day, based on a classification model
fication approach, in order to predict the next day, if the
that we built, using LSTMs from keras library.
stock’s price will be up or down.

1. Introduction 2. Dataset and Features

Predicting stock and stock price index is difficult due to un- The basic data that we used are downloaded from Yahoo
certainties involved. There are two types of analysis which through its API and it is the price of Goldman Sachs stock.
investors perform before investing in a stock. First is the The dates that we chose are from 31/12/2014 to 19/3/2019,
fundamental analysis. In this, investors looks at intrinsic and apart from that we also use the price of Bank of Amer-
value of stocks, performance of the industry and economy, ica, Barclays, Credit Suisse, JPMorgan, Morgan Stanley,
political climate etc. to decide whether to invest or not. NASDAQ, Hang Seng Index, NYSE, Nikkei 225 and VIX.
On the other hand, technical analysis is the evaluation of We also used the technical indicators Moving Average 7
stocks by means of studying statistics generated by market and 21, Exponential Moving Average(EMA), Moving Av-
activity, such as past prices and volumes. Technical ana- erage Convergence Divergence(MACD), Bollinger Bands,
lysts do not attempt to measure a security s intrinsic value Momentum and Log Momentum. The last additions to our
but instead use stock charts to identify patterns and trends dataset are three fourier transformations. Because of the
that may suggest how a stock will behave in the future. Ef- Bollinger Bands we started the dataset 29 days after the
ficient market hypothesis by experts, states that prices of starting data of the data, because until day 29, some of the
stocks are informationally efficient which means that it is data are missing. The idea of adding all of this data is that
possible to predict stock prices based on the trading data. the price of a stock is affected by a majority of factors; so
This is quite logical as many uncertain factors like politi- the more we add, the better prediction we may have.
cal scenario of country, public image of the company will
start reflecting in the stock prices. So, if the information 3. Methods
obtained from stock prices is preprocessed efficiently and
appropriate algorithms are applied then trend of stock or 3.1. Long Short Term Memory
stock price index may be predicted. Long Short Term Memory networks(Wei Bao, 2017) are
Since years, many techniques have been developed to pre- neural networks that are being used for solving time series
dict stock trends. Initially classical regression methods problems. As such, they provide the best possible solu-
were used to predict stock trends. Since stock data can be tion in solving the problem of predicting the prices in stock
categorized as non-stationary time series data, non-linear market. This kind of network is used to recognise pat-
machine learning techniques have also been used. Artifi- terns when past results have influence on the present result.
cial Neural Networks (ANN) is a machine learning algo- For this reason, our experiments and efforts were revolved
rithms which is most widely used for predicting stock and around LSTM networks.
Our first network was simple LSTM network of 50 neu-
University of Pennsylvania, CIS 419/519 Course Project. rons, with one hidden layer. We tried different type of
Copyright 2017 by the author(s). models with this kind of architecture. Firstly, we passed
 PUT YOUR LAST NAMES HERE

our data through the features dimension of the LSTM cell, Cadima, 2016). PCA is a statistical procedure that uses
then through the timestep dimension. Another model we an orthogonal transformation to convert a set of observa-
built was making the LSTM network stateful - with mem- tions of possibly correlated variables into a set of values
ory between the batches of data we passed, making it learn of linearly uncorrelated variables called principal compo-
from every sequence instead of every epoch. Afterwards, nents. This transformation is defined in such a way that
we applied dropout of 0.2 to the model that gave the best the first principal component has the largest possible vari-
results. That was done in order to avoid any overfitting of ance (that is, accounts for as much of the variability in the
our algorithm to our training set, trying to make it gener- data as possible), and each succeeding component in turn
alize better to unseen data. All these models were built in has the highest variance possible under the constraint that
order to fully capture the nature of our problem, trying to it is orthogonal to the preceding components. The resulting
see which algorithm would fit best as a solution. vectors (each being a linear combination of the variables
and containing n observations) are an uncorrelated orthog-
The way we evaluated our model was through a custom
onal basis set. From this analysis we resulted to 5 variables,
metric, named gain. While classic metrics, like mean
which try to capture the most information from our data and
squared error, mean absolute error etc, can be used to eval-
we followed the same procedure, to build the new models,
uate how close is our prediction to the actual price of the
as before.
stock, the problem itself suggests that we should see if the
model is gaining money or not. With that in our mind, if However, in the results of this process, it can be noticed,
we predict that the stock will move to the same direction that even though it wasn’t possible to predict the stock
as the actual move, then we won the absolute value of the price, the predictions were following correctly the direc-
subtraction of the closing price from the opening price. In tion the price was going. For this reason , we transformed
the same way we lost money if we predict the opposite di- the whole problem to a new one. Instead of predicting the
rection from the markets. price directly, we will try now to predict the direction of the
price where the price is heading, making it a classification
Afterwards, we built stacked LSTM models, trying to take
problem.We transform our data to binary, 0 if the price fell
advantage of the memory the LSTM cells provide to a net-
that day, 1 if the price went up. That way the information
work. The architecture is composed from two layers, one
our models need to process is much simpler and therefore,
with 50 neurons followed by one with 30. We followed the
much easier to learn.
same principle as before while building our models.

4. Results and Discussion

Time series data, as the name indicates, differ from other
types of data in the sense that the temporal aspect is impor-
tant. On a positive note, this gives us additional informa-
tion that can be used when building our machine learning
model, that not only the input features contain useful in-
3.2. XGBoost
formation, but also the changes in input/output over time.
We used XGBoost in order to see the importance of our However, while the time component adds additional infor-
features(Dey et al., 2016). From there, we could imple- mation, it also makes time series problems more difficult to
ment a different model which hypothetically be better since handle compared to many other prediction tasks.
we would use only the data that capture better the informa- So the gain metric in the classification approach has given
tion we want to learn, however no such result was achieved us a gain of 818.18, which is a very impressive result, be-
since only 2 out of 30 features were considered as not in- cause at the time of testing the LSTM in the data the Gold-
formative enough. For the record, XGBoost has been a man Sachs stock only gains 38 points. We can observe that
proven model in data science for its speed and accuracy. our algorithm surpasses that by a lot.
In XGBoost the trees are built sequentially such that each
subsequent tree aims to reduce the errors of the previous The accuracy of the algorithm is 0.736986. This is the per-
tree. Each tree learns from its predecessors and updates the centage that we predicted right the days that the stock went
residual errors. Hence, the tree that grows next in the se- either up or down. The accuracy can also be seen from the
quence will learn from an updated version of the residuals. graph below where we see with green color the days that
we predicted right, and with red color the days that we pre-
3.3. PCA dicted wrong.

In order to reduce the dimensionality of our data, we im-

plemented principal components analysis(PCA)(Jolliffe &
 PUT YOUR LAST NAMES HERE

5. Conclusion and Future Work

In conclusion with this work, we have achieved a really
good result. The accuracy of the algorithm may be only
0.736986, but this is above 51% which is considered satis-
fying by most professionals. But accuracy combined with
the gain metric, which is quite high doesn’t mean that we
have completely solved the stock market prediction prob-
lem. We have some future plans to make this algorithm
more reliable and more successful. In the future we will
add more complex and better algorithms like GANs, we
will also add prices targets and stop losses to achieve a bet-
ter result, and finally we will implement the algorithm in
MQL4 in order to do live trading with it.

References
Dey, Shubharthi, Kumar, Yash, Saha, Snehanshu, and
Basak, Suryoday. Forecasting to classification: Predict-
ing the direction of stock market price using xtreme gra-
dient boosting, 10 2016.
Hochreiter, Seep and Schmidhuber, Jurgen. Long short-
term memory. NEURAL COMPUTATION, 9(8):1735–
1780, 1997.
Jolliffe, Ian T. and Cadima, Jorge. Principal component
analysis: a review and recent developments. Philosoph-
ical Transactions of the Royal Society A: Mathematical,
Physical and Engineering Sciences, 374(2065), 04 2016.
Wei Bao, Jun Yue, Yulei Rao. A deep learning framework
for financial time series using stacked autoencoders and
long-short term memory. PLOS ONE, 12(7):1–24, 07
2017.