Does the use of Technical & Fundamental Analysis

improve Stock Choice?

: A Data Mining Approach applied to the Australian Stock Market

Carol Hargreaves Yi Hao

Institute of Systems Science School of Business Economics
National University of Singapore Wilfrid Laurier University
Singapore Waterloo, Canada
[email protected] [email protected]

Abstract—With the easy access to share information and data, • Large amounts of stock information easily available on
many investors worldwide are interested in predicting stock the internet, through newspapers, magazines, radio and
prices. The prediction of stock prices using data mining
techniques applied to technical variables has been widely television.
researched but not much research to date has been done in
applying data mining techniques to both technical and How can investors go about selecting stocks in a systematic
fundamental information. This paper is based on a personal way so that they select winning stocks and make a profit with
approach to stock selection, using both technical and minimum risk?
fundamental information. In this paper we construct a
framework that enables us to make class predictions about
industrial stock companies’ financial performances. In order to A. Aim of the Present Study
have a systemized approach for the selection of stocks and a high The focus of this paper is to investigate whether stocks
likelihood of the performance of the stock price increasing, a
selected by
Data Mining Approach is applied. A trading strategy is also
designed and the performance of the stocks evaluated. Our two
goals are to validate our stock selection methodology and to • the application of a personal systematic approach or
determine whether our trading strategy allows us to outperform with a data mining approach will outperform the
the Australian market. Simulation results show that our selected Australian stock market
stock portfolios outperform the Australian All-Ordinaries Index.
Our findings justify the use of data mining techniques for • when monitored and managed using a trading
classification and prediction purposes. Further, in conclusion, we strategy will outperform the Australian stock market.
can safely say that our stock selection and trading strategy
outperformed the Australian Ordinary index. B. Hypothesis
The following hypotheses will be tested in this study:
Keywords- stock price prediction; stock selection; stock market;
data mining; decision trees; neural networks (NN); trading strategy
• H1: The stocks selected through a personal
systematic approach outperforms the Australian stock
market
I. INTRODUCTION • H2: The stocks selected through a data mining
A stock market is a private or public market for the trading approach outperforms the Australian stock market
of company stock and derivatives of company stock at an
agreed price. Many financial companies such as stock markets In the remainder of the paper, we explain the methodology
produce large datasets and are looking to find efficient ways to for our study, by discussing our stock selection system,
discover useful information about stocks and the market for followed by our trading strategy. We conclude our paper with
investment decisions. Further, with the easy access to stock the results and the value of our study with possible further
information and data, many private investors worldwide are research ideas.
interested in predicting stock prices and hope to maximize on
the opportunities in the market and become rich. The problem
is: II. METHODOLOGY
• There are so many stocks in the market, Data mining can be described as “making sense of data”.
Over the last few decades, increasingly huge amounts of past
data have been stored electronically and this volume is
expected to grow considerably in the future. According to the
Gartner Group, “Data mining is the process of discovering • Neural Networks can cop pe with “fuzzy patterns” –
meaningful new correlations, patterns and ttrends by sifting patterns that are difficult to
o reduce into precise rules.
through large amounts of data stored in reepositories, using
pattern recognition technologies as well aas statistical and • Neural Networks can be reetrained and thus can adapt
mathematical techniques”. Data mining iis one way of
to changing market behaviour.
predicting if future stock prices will increase or decrease. We
treated our problem as an unsupervised task aand therefore we
• Neural Networks can play y a crucial role in deciding
created the class variable at the beginning annd only after the
class variable was constructed, we trained thhe classifier. We which technical variables to follow when analyzing
used decision trees and neural networks to cllassify our stocks past prices.
as to whether they will increase or decreasee in price. These
stock predictions assisted us in our decision oof whether to buy Based on the above information n, we ran a Neural Network
a stock or not. with our Australian industrial stock
k data using a multi-layered
perceptron Neural Network mod del, trained with a back
propagation algorithm, using the Hyyperbolic Tangent Solution.
A. Decision Trees
Decision trees are a form of multiple vvariable analyses
and are powerful and popular tools for cllassification and III. THE EXPERIMENT - THE STOCK
T SELECTION PROCESS
prediction. The attractiveness of decision trrees lies in their Our method is more from a top-down perspective. Figure 1
ease of interpretation, relative power, robustneess with a variety below, illustrates the path taken too arrive at our 6 stocks for
of data and levels of measurement, and ease of use. Decision selection consideration. Since theree are more than 2000 stocks
trees attempt to find a strong relationship betw
ween input values available in the Australian stock market,
m our first step is to
and target values in a group of observations that form a data narrow down the universe of stocks we want to choose from.
set [3]. In contrast to neural networks, decisioon trees represent
rules and rules can readily be expressed so that humans can
understand them or even directly used in database access To do this, we first identify thee best performing sector by
language like SQL so that records fallingg into particular comparing all sector price trends ag gainst the ASX (Australian
category may be retrieved [6]. Ordinary) index for the latest 3 mon nths. In our experiment, the
industrial sector turned out to o have the best sector
In our study, the decision trees generateed some rules of performance when compared to the Australian Ordinary Index.
how the performance of different industrial company stocks
were predicted as data became available. For a detailed
technical explanation of the two decision trees applied
(CHAID and C5.0) in this study, the reader is referred to [7]
for C5.0 and [3] for CHAID.

B. Neural Networks
Why use Neural Networks for forecastiing stock price?
There are numerous reasons why Neural Neetworks offer an
advantage in the quest to forecast stock pricess. Firstly, there is
no need for any assumptions to be made uused by Efficient
Market Hypothesis (EMH) and no needd for Normality
assumptions. The EMH [4, 5] assumes that thee price of a stock
reflects all of the information available and thhat everyone has
same degree of access to information and thhat when ever a
change in financial outlook occurs, the markket will instantly
adjust the stock price to reflect the new inform
mation.
Many studies [1] have established that non-linearity exists in
financial data and Neural Networks can be ssuccessfully used Figure 1: Stock Selection
S Process.
to model the relationship among this data. Soo the applications
of Neural Networks to financial forecasting haave become very So, we instantly reduced our univeerse of stocks from 2223 to
popular over the last few years. 200 as there are about 200 stocks in
n the Industrial sector. Data
• The main advantage of Neural Netw works is that they on the 200 industrial stocks was then
t collected from Yahoo
can approximate any nonlinear funcction to a degree Finance.
of accuracy with a suitable number off hidden units
 The four stock selection strategies weree applied in this C. Growth Trading Strategy
study. One personal trading strategy, two baased on decision We ran a CHAID decision tree withw the five input metrics
trees (CHAID & C5.0) and one based on N Neural Networks. and the 55 industrial stocks. The onnly metric that was
Each selection strategy selected six stocks and roughly ten important was ‘growth this year’, where
w the rule generated
thousand dollars was invested in the market ffor each selected was ‘growth this year >13.9%’. So thet six stocks with the
stock. To keep the input variables consistennt, five variables highest probability of increasing theeir price and meeting this
were used for all of the strategies: criterion the closest were selected fo
or this portfolio, where the
highest ‘growth this year’ stocks weere preferred.
• Return on Equity
• Return on Assets
D. Growth & Value Trading Strategy
• Analyst Opinion
We ran a Neural Network with the five input metrics and
• Growth this year the 55 industrial stocks. Three metrrics were important ‘growth
• Price this year’, ‘analyst opinion’ and ‘reeturn on assets’. So the six
stocks with the highest probability y of increasing their price,
In the next few sections we will describe each of the four was selected for this portfolio.
stock selection processes. For the stock sellection processes
using a data mining approach, 6 winning stoccks were selected
from 69 industrial stocks whose financial information was IV. TRADING STRATEGY
published. The goal of our trading strategy is to reduce the universe of
stocks to a manageable few. We W use technical analysis
A. Personal Trading Strategy (charts) and fundamental analysis (financial ratios) to assist us
in our selection of stocks with goo od indicators for growth &
We then selected all those industrial stockss that had an value respectively. After the stock selections
s process in Figure
upward trend in the market and were outperforrming the 1, we set the exit strategy as eitheer a stock gain of 10% or
Australian Ordinary Index. There were 34 of tthese stocks. We more, or a loss of 5% or more (ssee Figure 2). To evaluate
then selected the best 6 stocks based on the folllowing criteria: whether our trading strategy perfo orms well, we compare it
with the Australian Ordinary Index.
• Return on Equity (ROE) >15%,
• Return on Assets (ROA)>10%
• Analysts’ opinion < 3
• >20%
Growth estimate for the current year>
• Pure Industrial Firm

As a result, 6 stocks were selected for our ppersonal strategy

(Option 1). Our scoring system assigned w weights to stocks
based on their fundamental ratios, price, analyysts’ opinion and
growth estimates, where the sum of all weiights is one. We
next ranked our stocks from 1 to 6 with 1 as the highest
preferred score to 6 as the lowest preferred scoore (for example,
the higher the ROE the lower the rank). The final score is the
sum product of the stock ranking and assignned weights. The
assumption made is that the smaller the scoree, the more likely
the stock will make profit.
Figure 2: Trading Strategy.
S

B. Price Trading Strategy

We ran a C5.0 decision tree with the five input metrics on V. RESU
ULTS
the 55 industrial stocks. The ‘price’ metric was the most Accuracy can be compared in manym ways. In the simplest
important. More specifically the rule generrated was ‘price way, a decision tree is grown and the predicted classification
>1.05’. So the six stocks that had the higheest probability of is tested against the data set used to
o train the form of the tree.
increasing their price and meeting this criteerion the closest This is called a re-substitution tesst [3]. The performance of
were selected for this portfolio, where the low
wer priced stocks CHAID is determined by a test of siignificance of 0.05. For our
were preferred with as they had a higher probability of CHAID, C5.0 and Neural network k models, we obtained the
increasing price. following results (See Table 1, 2, 3)):
 The accuracy rates for the data mining models are presented in
Table 1: C5.0 Accuracy Table
Table 4 below.
Actual Table 4: Summary Accuracy Table

Predicted Increase Stock Decrease Stock Sum Accuracy C5.0 CHAID Neural Overall
Price Price Measure Network
Decrease Stock 1 23 24 Sensitivity 98% 86% 93% 92%
price
Increase Stock 42 3 45 Specificity 88% 46% 31% 55%
Price
Sum 43 26 69

Overall the models performed very well, with average

sensitivity of 92% and specificity of 55%. The C5.0 decision
Table 2: CHAID Accuracy Table
tree had the best overall performance. Sensitivity is the more
important measure for us as we are more interested in
Actual correctly classifying stocks with increasing trends. So, the
overall accuracy level (sensitivity of 92%), provides support
Predicted Increase Stock Decrease Stock Sum that our data mining models are reliable and have high
Price Price accuracy. els
Decrease Stock 6 12 18
price
Increase Stock 37 14 51 A. Portfolio Results
Price The personal strategy, option 1 portfolio yielded a higher
Sum 43 26 69 return, than the Australian Ordinary Index (AOI) (7.9% versus
1.7%) during a 20 day trading period. This yield is more than
4.5 times higher than the AOI return (See Figure 3).
Both Table 1 and Table 2 show the C5.0 and CHAID
decision tree classification techniques to achieve high
accuracy rates. Personal Portfolio vs Australian
Ordinary Index
Table 3: Neural Network Accuracy Table
10.00%
Actual
Portfolio %
Gain or Loss

5.00% G/L
Predicted Increase Stock Decrease Stock Sum
Price Price
0.00% AOI % G/L
Decrease Stock 3 8 11
price
1 4 7 10 13 16 19
Increase Stock 40 18 58
-5.00%
Price
Sum 43 26 69 Figure 3: Personal Strategy Portfolio versus Australian Ordinary Index.

According to [3], sensitivity and specificity may be The growth value strategy (using a C5.0 decision tree),
computed using equations (1) and (2). option 2 portfolio also yielded a higher return, than the
Australian Ordinary Index (AOI) (19.9% versus 1.7%) during
a 20 day trading period. This yield is more than 11.7 times the
N T P AOI return (See Figure 4). One thing to take note with the low
Sensitivity = (1)
N T P N F N price strategy is the volatility. The volatility is higher than the
personal strategy. It confirms the notion that the higher the
volatility, the higher the possible returns. Stocks with lower
N T P prices are more volatile than others but may produce higher
Specificity = (2) returns.
N T P N F N
 It is quite clear from the above figures, that our hypothesis:
Growth Value Portfolio vs Australian
H1: The stocks selected through a personal systematic
Ordinary Index approach outperforms the Australian stock market
30.00% significantly
PriceOpt %
Gain or Loss

20.00% H2: The stocks selected through a data mining approach

G/L outperforms the Australian stock market significantly
10.00%
AOIndex %
0.00% G/L Further, a statistical test was performed at the 5% level and
it confirmed that the returns of all 4 portfolios were
-10.00% 1 4 7 10 13 16 19 significantly different to the returns of the Australian stock
market.
Figure 4: Growth Value (C5.0) Portfolio versus Australian Ordinary Index.

B. Limitations
Similarly, the price strategy (using a CHAID decision tree),
option 3 portfolio also yielded a higher return, than the The data set size is smaller than usual and missing data was
Australian Ordinary Index (AOI) (20.8% versus 1.7%) during imputed using the median value. The specific form of a
a 20 day trading period. This yield is twelve times the AOI decision tree, particularly at lower levels, cannot be exactly
return (See Figure 5). reproduced when applied to new data. One of the main
problems of a single decision tree model is that small changes
in the data set can produce substantial changes in the model.
Thus, small changes can easily change the size and shape of
Price Strategy Portfolio vs Australian the decision tree. One solution is to grow multiple decision
Ordinary Index trees using a randomisation approach and then combine the
information from the multiple decision trees into one summary
60.00% representation. Another limitation is that there are many other
learning algorithms in the Neural Network to be explored.
Gain or Loss

40.00% Portfolio %
Secondly, we focused on the industrial sector so our stock
G/L selection criteria and trading strategy is currently limited to
20.00%
AOI % G/L this sector.
0.00%
-20.00% 1 4 7 10 13 16 19 VI. CONCLUSION
A. Findings
Figure 5: Price Strategy Portfolio versus Australian Ordinary Index.
In this paper we showed how Data Mining techniques can
be used to build classification models with stock data from the
The growth & value strategy, option 4, also yielded a Industrial Sector of the Australian Stock Market. In
higher return than the Australian Ordinary Index (AOI) conclusion, we can safely say that our stock selection and
(23.2% versus 1.7%). This yield is almost 14 times more than trading strategy outperformed the Australian Ordinary index.
the AOI return (See Figure 6). If we annualize our portfolio under the assumptions that we
could consistently trade in this way each month, then expected
returns for each portfolio would be 94.3% for our personal
Growth Value Portfolio vs Australian strategy, 216% for the low price strategy, 243.5% for the
growth strategy and 263% for the growth value strategy.
Ordinary Index
80.00% Further, we have obtained high accuracy rates for the three
60.00% data mining models under study, the C5.0 decision tree, the
Gain or Loss

Portfolio % CHAID tree and the Neural Network model. The decision tree
40.00% G/L models performed better in classifying the stocks than the
20.00% Neural Network model, probably because the data set size was
AOI % G/L
0.00% smaller than usual and the Neural Network model will perform
-20.00% 1 4 7 10 13 16 19 better for larger sample sizes.

Our results are good with regards to the validation criteria.

Figure 6: Growth Value Portfolio versus Australian Ordinary Index. We have confirmed that the metrics, ROE, ROA, growth this
year, price, and analyst opinion, are good predictors for exit strategy on a loss of 5% or more. Other trading strategies
classifying stocks into two groups, namely, stocks that are may be investigated.
highly likely to increase in price and stocks they are not likely
to increase in price.
REFERENCES
Our findings justify the use of data mining techniques for
classification and prediction purposes. Lastly, our trading
strategy consistently produced results that outperformed the
Australian Ordinary Index, whether our personal strategy or [1] R.Apostolos-Paul, “Neural Networks in the Capital Markets,” 1995.
the low price strategy or the growth strategy or the growth [2] A. Costea, “new Approached and Perspectives in eth use of quantitative
methods”. Journal of Applied Quantitative Methods. Vol.1 No.2 Winter,
value strategy was chosen, because our trading strategy was 2006.
employed we consistently outperformed the Australian market. [3] B. de Ville. “decision trees for Business Intelligence and Data
mining:Using SAS Enterprise Miner. SAS Institute Inc., Cary, NC,
USA.ISBN-13:978-1-59047-567-6, 2006.
B. Further Research [4] E. Fama. “The behaviour of stock market prices.” Journal of Business
Future researchers may include more methods for finding 38, 34-105, 1965.
the best model for predicting stock prices. We used stocks [5] E. Fama. “Random Walks in stock market prices”. Financial Analysts
Journal 21, 55-9, 1965.
from the Industrial Sector however it would be interesting to
[6] E. Hajizadeh; H.D. Ardakani; J. Shahrabi. “Application of data mining
expand our study to see whether our stock selection and techniques in stock markets: A Survey.”, Journal of economics and
trading strategy will work in other sectors. Another interesting International Finance. Vol. 2 (7), pp. 109-118, 2010.
idea we have in mind, is to build an application that applies [7] J.R. Quinlan. “C4.5: Programs for Machine Learning”, Morgan
our stock selection and trading strategy. Finally, our trading Kaufmann Publishers, 1993.
strategy has a 10% upper limitation on the portfolio and an [8] The Gartner Group, www.garner.com