Identifying Trades Using Technical Analysis and ML/DL Models
Identifying Trades Using Technical Analysis and ML/DL Models
ABSTRACT: The importance of predicting stock market prices cannot be overstated. It is a pivotal
task for investors and financial institutions as it enables them to make informed investment
decisions, manage risks, and ensure the stability of the financial system. Accurate stock market
predictions can help investors maximize their returns and minimize their losses, while financial
institutions can use this information to develop effective risk management policies. However, stock
market prediction is a challenging task due to the complex nature of the stock market and the
multitude of factors that can affect stock prices. As a result, advanced technologies such as deep
learning are being increasingly utilized to analyze vast amounts of data and provide valuable
insights into the behavior of the stock market. While deep learning has shown promise in accurately
predicting stock prices, there is still much research to be done in this area.
KEYWORDS: Stock Market, Technical Indicators, Deep Learning, Price Trends, Closing Prices
I. INTRODUCTION
The onerous and pioneering task of predicting stock market prices is an indispensable task for
any informed investor. The non-stationary, non-linear, random, and noisy behavior of the market
does not help make the task any less challenging. We can use modern technology and program
models that can help predict potential future trends and patterns that emerge from observing several
years of stock market data. These trends and patterns could help investors make informed decisions
about stocks under their purview and maximize their profits. The fitful nature of the markets could
intimidate even the most experienced and protean of investors and therefore it is momentous for the
task to be automated to provide investors some sort of a structure to a mercurial market.
A. Problem
Accurately predicting stock market prices is a grueling and painstaking task. Erudite investors
make use of technical and fundamental analysis, which stem from certain technical indicators
designed to evince which stocks to invest in so as to make a net profit. Even today, a large fraction
of investors and traders manually design and adhere to a set of rules derived from the
aforementioned technical & fundamental analysis. This process, however reliable it might be, is
mind-numbing and transitory based on current market trends. Perhaps a more sophisticated and
comprehensive approach would be to use deep learning models to automate the task and make it
more generic in order to embrace a wider range of trends and patterns.
B. Complexity
A plethora of models exist for predicting stock market prices. From all the available options,
selecting the model that is the right fit for our use case is cumbersome. In addition to selecting the
right model, we also have to pick the right hyperparameters for our model since they govern the
learning rate of the model. This involves a lot of experimentation and trial and error done over a
large dataset to arrive at the right blend. There are also times when a certain model is better suited
for a certain period of interval of time or for a certain sector of the market. This makes the entire
process of stock market prediction formidable and understandably, byzantine.
C. Challenges
The preliminary challenge was to access the publicly available APIs for obtaining accurate yet
reliable stock market data, one needs to have a registered trading account, so that was a major
drawback and also, the number of requests that could be sent to the server were limited. Since we
were thinking of analyzing and making predictions for at least 500 stocks, we needed a thorough
approach to this. Once this was dealt with, the major and the most intimidating challenge was to
calculate and store the values of all the stock market indicators (approx 60) for all the stocks
(approx 500). And the mere volume of the data suggested we use some of the more sophisticated
processing methods that deal with the data.
II. RELATED WORK
Various research studies have attempted to study stock market behavior. Deep learning has
enabled us, from manually designing buying and selling rules using different technical and
fundamental indicators to allowing the models to design complex rules for us using historical data.
The model here acts as a black box. Deep Neural Networks, Support Vector Machines, Random
Forests, and Decision Trees are some of the many models that can be used for this purpose.
Mabrouk et al. (2022) [1] discusses an Intraday Trading Strategy based on GRU and CNN for the
Forex market. The authors used feature selection techniques to reduce the number of inputs from
more than 150 variables to less than 30, keeping only the most important features. Their proposed
model is composed of four layers, with the first layer taking a two-dimensional matrix as input. The
authors evaluated their proposed model using real-world data from the Forex market and compared
it with several baseline models. The results showed that their hybrid model had the best
performance in terms of profit accuracy for predictions in all periods. It also reduced the number of
transactions compared to the baseline models.
Agrawal et al. (2022) presents a deep learning model for predicting stock prices based on
technical indicators. The authors propose a novel architecture that combines convolutional neural
networks (CNNs) and long short-term memory (LSTM) networks to capture both spatial and
temporal features of the input data. The proposed model is evaluated on real-world stock market
data from the National Stock Exchange of India (NSE) and compared with traditional machine
learning models such as support vector regression (SVR) and random forest regression (RFR). The
results show that the proposed model outperforms these traditional models in terms of prediction
accuracy, with an average mean absolute error (MAE) of 0.0125. The authors also conduct
sensitivity analysis to investigate the impact of different hyperparameters on the performance of the
model. Finally, the authors discuss some limitations and potential drawbacks of using deep learning
models for stock prediction, such as overfitting, data quality issues, and interpretability challenges.
Hamoudi and Elseif (2021) [3] presents a novel approach for stock market prediction using
1-Dimensional Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM)
models. The goal of the project is to develop prediction models for high-frequency automated
algorithmic trading using a data set of 130 anonymous intra-day market features and trade returns.
The models are designed to predict the trades resulting in returns in the top ten percentile of all
returns in the training set. The authors introduce two novelties in their approach. First, they frame
the problem as a binary classification task rather than trying to predict the exact value of the return
for a given trading opportunity. Second, they augment the feature matrix with a logical array to
preserve information about missing features at each time step. The results show that their approach
achieves positive returns with very low risk by identifying promising trading opportunities. The
deep neural network models outperform traditional machine learning models such as Artificial
Neural Networks (ANNs) and Support Vector Machines (SVMs).
Ghosh et al. (2021) [4] proposes a novel approach to forecast directional movements of stock
prices for intraday trading by using Long Short-Term Memory (LSTM) and Random Forests (RF)
models.The proposed approach involves using technical indicators such as moving averages,
relative strength index (RSI), and Bollinger Bands as input features to train the models. The LSTM
model is used to capture the temporal dependencies in the data, while the RF model is used to
select the most relevant features and make final predictions. The authors evaluate the proposed
approach on historical data of two major stock indices, namely the S&P 500 and NASDAQ. The
results show that the proposed approach outperforms several benchmark models in terms of
accuracy and profitability, as measured by the trading simulation results.
Faraz et al. (2020) [5] proposes a novel approach for predicting the closing price of the stock
market using autoencoder long short-term memory (AE-LSTM) networks. The model uses an
encoder-decoder-based architecture to extract invariant and abstract features from the raw dataset in
an unsupervised way. Technical analysis techniques such as wavelet transformation and outlier
exclusion are also employed to eliminate market noise and improve prediction accuracy. The
z-score method is used for anomaly detection in the dataset, further improving the accuracy of the
predictions. Overall, this paper presents a promising strategy for predicting stock prices using deep
learning methods and technical analysis.
Tomar et al. (2020) [6] explores the use of deep learning techniques, specifically LSTM neural
networks and their slim variants, for intraday stock trading. The study focuses on predicting the
next-minute price movement of the US SPDR S&P 500 index using a dataset consisting of various
technical indicator features. The paper outlines the methodology used in detail, including
information about the dataset, features used, and feature selection. The results show that the slim
LSTM2 model outperforms other models in terms of accuracy and F1 score. This study provides
insights into how deep learning techniques can be applied to real-world data for intraday stock
trading and could potentially be useful for traders or investors looking to make more informed
decisions based on predictive modeling.
Miao (2020) [7] explores a deep learning approach for stock market prediction using a Long
Short-Term Memory (LSTM) network. The LSTM model is trained on historical stock price data
and tested on future stock prices to predict the trend of the stock market. The study tests different
parameters such as the number of hidden layers, drop-out regularization, and batch size to
determine their effect on result accuracy. The dataset used in this study includes historical stock
prices of Amazon, Google, and Facebook from January 2010 to December 2017. The dataset is
preprocessed by normalizing the data and dividing it into training and testing sets. The results show
that the LSTM technique is widely applied in similar works and achieves promising results in
predicting stock prices. The study finds that changing the number of neurons can influence the
model's performance while changing the time step can also affect accuracy.
Paspanthong et al. (2019) [8] proposes the use of machine learning models to predict future price
movements of SPDR S&P 500 trust and how investors can benefit from these predictions. The
authors implemented multiple algorithms, including logistic regression, support vector machines
(SVM), Long-Short Term memory (LSTM), and Convolutional Neural Networks (CNN), to
determine the trading action in the next minute. They used the predicted results from their models
to generate the portfolio value over time and found that a support vector machine with a polynomial
kernel performs the best among all of their models. The main goal of this paper is not only to assess
the statistical performance of machine learning in forecasting future price movements but also
effectively evaluate the results in terms of actual profits.
Agarwal et al. (2018) [9] presents a thesis on the development of a stock price prediction model
using technical analysis and machine learning. The model uses a two-layer reasoning approach that
employs domain knowledge from technical analysis to guide a second layer of reasoning based on
machine learning. The model is supplemented by a money management strategy that uses the
historical success of predictions made by the model to determine the amount of capital to invest in
future predictions. The performance of the model is evaluated on stocks listed on the Oslo Stock
Exchange, and it is found to outperform the Oslo Benchmark Index (OSEBX) successfully. This
thesis provides valuable insights into the use of technical analysis and machine learning for
predicting stock prices, which can be useful for investors and traders in making informed decisions.
Sezer et al. (2017) [10] proposes a model that optimizes technical analysis parameters using
genetic algorithms (GA). The study uses Genetic Algorithm to optimize RSI parameters for
different market conditions. The optimized feature values are then used as buy-sell trigger points
for the deep neural network data set. The methodology of parameter optimization is an important
aspect of the study, as it enhances the stock trading performance and provides a model that might
be used as an alternative to Buy and Hold and other standard technical analysis models.
Kwon et al. (2007) [11] presents a hybrid neurogenetic approach for stock forecasting that
combines a recurrent neural network and a genetic algorithm to optimize stock trading predictions.
The system was tested on 36 companies in NYSE and NASDAQ for 13 years and showed notable
improvement over the buy-and-hold strategy and the context-based ensemble. The authors also
used a context-based ensemble method to dynamically adjust the predictions based on the test day's
context. This paper provides valuable insights into the use of machine learning techniques for
financial portfolio construction.
III. PROPOSED SYSTEM
A. Problem Statement
To implement a model that analyzes the daily prices of all the stocks in an index and predicts
trends thus helping in identifying profitable trades. This may help traders/investors to automate the
task of shortlisting promising stocks and thus help them boost their profits.
B. Proposed Methodology
To predict buying points in stocks to gain profit, we use historical data of the current NIFTY 500
index of NSE. The data involves Open, High, Low, and Close (OHLC) prices of each trading day
from March 2000 to March 2022 as well as the volume of equity shares traded.
We have used technical analysis exclusively as our trading method. Technical analysis involves
statistical methods and chart patterns to determine price movement. We have used technical
indicators and candlestick patterns that can be derived from the OHLC values. These statistical
tools help in determining a strategy to buy stocks based on certain parameters and constraints.
The strategy is then implemented on the historic data and gives certain buying points and selling
points. These buying and selling points are backtested to check if they are profitable or not. The
output of the strategy is added to our data with values of -1 where the strategy doesn’t give a
buying signal, 0 where the strategy gives a buying signal but is proven to incur a loss when
backtested and 1 where the strategy gives a buying signal and the trade is profitable when
backtested.
Thus the problem has now been transformed into a classification problem where a machine
learning model classifies the buying points of a strategy into profitable or lossy trade. All rows with
a -1 value in the strategy attribute are removed and the remaining rows are further preprocessed and
given as input. The machine learning model classifies the trade and gives output based on the
trained weights thus helping a trader to strengthen his strategy further and maximize profits.
a. Adding Features
Apart from price features and volume, we have added technical indicators, candlestick
patterns, and strategy features. Technical indicators are statistical values derived from the
price or volume features that give an indication of the trend, momentum, or volatility of the
stock. We derived the following technical indicators from the raw data so as to feed them to
the models.
● RSI
The relative strength index or RSI is a momentum-based oscillating technical indicator. It
indicates the strength of buying and selling for a given time period and also tells if the stock
is overbought or oversold.
● MACD
Moving average convergence divergence(MACD) is a trend-based technical indicator that
indicates potential trends using moving averages of the underlying security. It is derived
from the difference between two variable-length exponential moving averages.Following
are some of the technical indicators derived from the closing price of the underlying
security to help train the model
● Bollinger Bands
Bollinger Bands are volatility-based technical indicators. Bollinger bands indicate whether
the stock has been bought or sold beyond certain values of standard deviations thus
indicating the volatility. We have used Bollinger bands with 2 standard deviations.
b. Data Preprocessing
Data Normalization: We use data normalization so as to transform the data such that they
have no/similar dimensions. Variables that are bigger in magnitude tend to steer the model
in their direction which is not good for performance. Data normalization helps prevent such
behavior by ensuring each variable has similar weights/importance. We have used min-max
normalization for data normalization because it ensures that all features will have the same
scale.
Technical Indicators help identify trading opportunities. For example, a moving average
crossover often signifies a change in an ongoing trend. In this instance, applying a moving average
indicator to a stock's price chart can help identifying an area where an ongoing price may change,
thus identifying a trading opportunity.
D. Model
a. Neural Network Classification
Deep neural networks (DNNs) are a type of machine learning algorithm inspired by the
structure and function of the human brain. They consist of multiple layers of interconnected
nodes, or neurons, that process input data and progressively extract higher-level features for
classification or other tasks. DNNs are particularly useful for classification tasks because
they can automatically learn complex decision boundaries from large and high-dimensional
datasets. In other words, they can classify objects or events into different categories based
on patterns and relationships in the data, without being explicitly programmed to do so.
As input to the model, we have provided trades that are shortlisted by the technical trading
strategy. Each row, signifying the buying point of the trade, contains 51 attributes. The
model outputs a single value for each row. This single output is the probability of the trade
being profitable.
For the loss function, we have used binary cross-entropy. We use RMSProp as the
optimizer. The model has one hidden layer with 9 hidden units.
b. XGBoost
XGBoost (short for Extreme Gradient Boosting) is a type of machine learning algorithm
that uses decision trees to model and predict outcomes. It is an ensemble method, which
means it combines the predictions of multiple weaker models (decision trees) to make a
more accurate final prediction.
XGBoost works by iteratively training decision trees on the residuals (errors) of the
previous trees, gradually improving the overall prediction accuracy. It also uses
regularization techniques to prevent overfitting, which can occur when a model is too
complex and fits the training data too closely.
We use XGBoost to classify the trades given by the technical analysis strategy. We use the
following hyperparameters:
c. Decision Tree
Decision Tree, in various papers referred to and in the past have shown constructive results
in binary classification problems. Also analyzing the decision tree leads to better feature
selection for other methods of classification too. We have leveraged the decision tree to
classify buy signals received from our strategy.
The input consisted of 35 features ranging from normalized close values, binary data of
different types of Candle sticks, and encodings of the Stock’s company name and its
respective industry. We used Label Encoding to encode Company and Industry names here.
We explored hash encodings too but for the Decision Tree label, encoding was a better
choice since hash encodings led to an output of 6 bits which is a better input for a Neural
network and not a decision tree. Gini Impurity was used in the Decision tree since it is ideal
for binary classification. We used Graphviz for visualizing the resulting Decision tree.
The resulting Decision Trees were of large size, leading to which we set the max depth to 6.
Precision
2011-12 0.3559
2012-13 0.3572
2013-14 0.4448
2014-15 0.4484
2015-16 0.3263
2016-17 0.4264
2017-18 0.4020
2018-19 0.3480
2019-20 0.3357
2020-21 0.4570
2021-22 0.3858
Precision Recall
Since the performance of the Decision Tree was not promising we discontinued it
further and have not performed a year-wise analysis for the same.
Precision Recall
Using XGBoost:
Precision Recall
We can see from the above tables that the model does a great job in filtering the
trades given by the technical analysis strategy. But using precision and recall as a
metric will not suffice as we do not know profit per trade. It does not do any good if
the model has high precision but only identifies many trades with very small profits
and fewer trades with huge losses. On the other hand, it is okay to have a model with
lower precision but that identifies fewer trades with huge profits and many trades
with small profits.
To have a better idea regarding the performance of the model we can compare the
models by calculating the average profit per trade for the trades that are predicted to
be profitable by the model.
V. CONCLUSION
By using the initial strategy developed by us we were able to achieve an average of 3.1359 %
profit per trade each year since 2011 with an average precision of 0.3897. Using Neural Network
Architecture, we improved it to 3.9883 % ( a 27% improvement from the strategy-only method)
with an average precision of 0.4105 ( a 5% improvement). Further XGBoost logged 4.6019 %
profit per trade each year since 2011 (a 46% improvement) with an average precision of 0.4357(an
11.8% improvement). Thus, according to the proposed methodologies, applying DL/ML
techniques on top of trades given by technical analysis strategies increases average precision and
per-trade profitability significantly.
BIOGRAPHY
Aayush Shah, B. Tech Student, Dept. of Computer Engineering and IT, VJTI College, Mumbai,
Maharashtra, India.
Mann Doshi, B. Tech Student, Dept. of Computer Engineering and IT, VJTI College, Mumbai,
Maharashtra, India
Meet Parekh, B. Tech Student, Dept. of Computer Engineering and IT, VJTI College, Mumbai,
Maharashtra, India.
Nirmit Deliwala, B. Tech Student, Dept. of Computer Engineering and IT, VJTI College, Mumbai,
Maharashtra, India.