International Journal of Computer Science and Telecommunications [Volume 4, Issue 12, December 2013] 17

A Machine Learning Model for Stock Market

Prediction
ISSN 2047-3338

1 2 3
Osman Hegazy , Omar S. Soliman and Mustafa Abdul Salam
1,2
Faculty of Computers and Informatics, Cairo University, Egypt
3
Higher Technological Institute (H.T.I), 10th of Ramadan City, Egypt
2
[email protected], [email protected]

Abstract– Stock market prediction is the act of trying to regularization parameter and the kernel parameters play an important
determine the future value of a company stock or other financial role in the regression system. Therefore, it is necessary to establish a
instrument traded on a financial exchange. The successful methodology for properly selecting the LS-SVM free parameters, in
prediction of a stock's future price will maximize investor’s such a way that the regression obtained by LS-SVM must be robust
gains. This paper proposes a machine learning model to predict against noisy conditions, and it does not need priori user knowledge
stock market price. The proposed algorithm integrates Particle about the influence of the free parameters values in the problem
swarm optimization (PSO) and least square support vector studied [6].
machine (LS-SVM). The PSO algorithm is employed to optimize The perceived advantages of evolutionary strategies as
LS-SVM to predict the daily stock prices. Proposed model is optimization methods motivated some researchers to consider such
based on the study of stocks historical data and technical stochastic methods in the context of optimizing SVM. A survey and
indicators. PSO algorithm selects best free parameters overview of evolutionary algorithms (EAs) found in [7]. Particle
combination for LS-SVM to avoid over-fitting and local minima swarm optimization (PSO) is one of the most used EAs. PSO is a
problems and improve prediction accuracy. The proposed model recently proposed algorithm by James Kennedy and Russell Eberhart
was applied and evaluated using thirteen benchmark financials in 1995, motivated by social behavior of organisms such as bird
datasets and compared with artificial neural network with flocking and fish schooling [8]. The optimizer which is used in the
Levenberg-Marquardt (LM) algorithm. The obtained results particle swarm optimization algorithm, while making adjustment
showed that the proposed model has better prediction accuracy towards "local" and "global" best particles, is conceptually similar to
and the potential of PSO algorithm in optimizing LS-SVM. the crossover operation used by genetic algorithms [9]. As well
particle swarm optimization includes fitness function, which
Index Terms– Least Square Support Vector Machine, Particle measures the closeness of the corresponding solution to the
Swarm Optimization, Technical Indicators and Stock Price optimum. The main difference of particle swarm optimization
Prediction concept from the evolutionary computing is that flying potential
solutions through hyperspace are accelerating toward "better"
solutions, while in evolutionary computation schemes operate
I. INTRODUCTION directly on potential solutions which are represented as locations in
hyperspace [10]. SVM was used in stock market forecasting in [11].
Financial time series forecasting using SVM optimized by PSO was
S TOCK price prediction has been at focus for years since it can
yield significant profits. Predicting the stock market is not a
simple task, mainly as a consequence of the close to random-
presented in [12]. The Optimization of Share Price Prediction Model
Based on Support Vector Machine‫ ‏‬is presented in [14]. Financial
walk behavior of a stock time series. Fundamental and technical time series forecasting based on wavelet kernel support vector was
analyses were the first two methods used to forecast stock prices. presented in [15]. Computational Intelligence Approaches for Stock
Artificial Neural networks (ANNs) is the most commonly used Price Forecasting was introduced in [16]. A hybrid approach by
technique [1]. In most cases ANNs suffer from over-fitting problem integrating wavelet-based feature extraction with MARS and SVR
due to the large number of parameters to fix, and the little prior user for stock index forecasting was presented in [17]. An interval type-2
knowledge about the relevance of the inputs in the analyzed Fuzzy Logic based system for modeling generation and
problem [2]. summarization of arbitrage opportunities in stock markets was
Also, Support vector machines (SVMs) had been developed as an presented in [18]. Robust stock trading using fuzzy decision trees is
alternative that avoids such limitations. Their practical successes can presented in [19]. Ensemble ANNs-PSO-GA Approach for Day-
be attributed to solid theoretical foundations based on VC-theory [3]. ahead Stock E-exchange Prices Forecasting was presented in [20].
SVM compute globally optimal solutions, unlike those obtained with Index prediction with neuro-genetic hybrid network was proposed in
ANNs, which tend to fall into local minima [4]. [21]. A hybrid fuzzy intelligent agent-based system for stock price
Least squares –support vector machines (LS-SVM) method was prediction was introduced in [22]. Improved Stock Market Prediction
presented in [5], which was reformulated the traditional SVM by Combining Support Vector Machine and Empirical Mode
algorithm. LS-SVM uses a regularized least squares function with Decomposition was presented in [23]. Neural Network Ensemble
equality constraints, leading to a linear system which meets the Model Using PPR and LS-SVR for Stock Market Forecasting was
Karush-Kuhn-Tucker (KKT) conditions for obtaining an optimal proposed in [24]. Computational Intelligence Techniques for Risk
solution. Although LS-SVM simplifies the SVM procedure, the Management in Decision Making was introduced in [25]. Stock

Journal Homepage: www.ijcst.org

 Osman Hegazy et al. 18

market prediction algorithm using Hidden Markov Models was

proposed presented in [26]. Neural Networks and Wavelet De-
I 0 0  Z T  W   0 
 
Noising for Stock Trading and Prediction was introduced in [27].
0 0 0  1T  b   0 
The aim of this paper is to develop a machine learning model that   
hybrids the PSO and LS-SVM model. The performance of LS-SVM 0 0 CI  I  e   0 
(4)
 
0   a   y 
is based on the selection of free parameters C (cost penalty), ϵ
(insensitive-loss function) and γ (kernel parameter). PSO will be  Z 1 I
used to find the best parameter combination for LS-SVM.
The paper is organized as follows: Section 2 presents the Least
square support vector machine algorithm; The Particle swarm Where I represents the identity matrix and
optimization algorithm is introduced in section 3; The proposed
model and its implementation stock prediction are discussed in
section 4 Section 5 introduces excremental results and discussions.
Finally, section 6 is devoted to conclusions of the proposed model. Z  [ ( x1),  ( x2),...,  ( xn )]T (5)

II. LEAST SQUARE SUPPORT VECTOR MACHINE From rows one and three in (3) w  Z T a and Ce  a

Least squares support vector machines (LS-SVM) are least

Then, by defining the kernel matrix K  ZZ T , and the
parameter   C , the conditions for optimality lead to the
1
squares versions of support vector machines (SVM), which
are a set of related supervised learning methods that analyze following overall solution
data and recognize patterns, and which are used
0 1T  b   0 
    
for classification and regression analysis. In this version one
 (6)
finds the solution by solving a set of linear equations instead
1 K   I  a   y 
of a convex quadratic programming (QP) problem for
classical SVMs. Least squares SVM classifiers, were Kernel function K types are as follows:
proposed by Suykens and Vandewalle [28].
Let X is n  p input data matrix and y is n  1 output  Linear kernel K ( x, xi )  xiT x (7)
n
vector. Given the {xi , yi } i 1 training data set, where  Polynomial kernel of degree d:
xi  R and yi  R , the LS-SVM goal is to construct the
p
K ( x, xi )  (1  xiT x / c) d (8)
function f ( x)  y , which represents the dependence of the  Radial basis function RBF kernel :
output y i on the input x i . This function is formulated as:
K ( x, xi )  exp(  x  xi / 2 )
2
(9)
f ( x)  W  ( x )  b
T
(1)

Where W and  (x) : R p  R n are n  1 column  MLP kernel :

vectors, and b  R . LS-SVM algorithm [5] computes the
function (1) from a similar minimization problem found in the K ( x, xi )  tanh(kxiT x   ) (10)
SVM method [3]. However the main difference is that LS- In this work, MLP kernel is used.
SVM involves equality constraints instead of inequalities, and
it is based on a least square cost function. Furthermore, the III. PARTICLE SWARM OPTIMIZATION
LS-SVM method solves a linear problem while conventional ALGORITHM
SVM solves a quadratic one. The optimization problem and
the equality constraints of LS-SVM are defined as follows: PSO is a relatively recent heuristic search method which is derived
from the behavior of social groups like bird flocks or fish swarms.
1 T 1 T PSO moves from a set of points to another set of points in a single
min j(w,e,b)  w wC e (2)
iteration with likely improvement using a combination of
w, e, b 2 2 deterministic and probabilistic rules. The PSO has been popular in
academia and industry, mainly because of its intuitiveness, ease of
yi  wT  ( xi )  b  ei (3) implementation, and the ability to effectively solve highly nonlinear,
mixed integer optimization problems that are typical of complex
engineering systems. Although the “survival of the fittest” principle
Where e is the n  1 error vector, 1 is a n  1 vector with all is not used in PSO, it is usually considered as an evolutionary
entries 1, and C  R  is the tradeoff parameter between the algorithm. Optimization is achieved by giving each individual in the
search space a memory for its previous successes, information about
solution size and training errors. From (2) a Lagranian is
successes of a social group and providing a way to incorporate this
formed, and differentiating with respect to w, b, e, a ( a is knowledge into the movement of the individual.
Largrangian multipliers), we obtain;
International Journal of Computer Science and Telecommunications [Volume 4, Issue 12, December 2013] 19

Therefore, each individual (called particle) is characterized by its

   Data Acquisition and Preprocessing (Stocks Histoical data)
position xi , its velocity vi , its personal best position pi and its
 Feature Extraction and selection (Techincal Indicators)
neighborhood best position pg .
The elements of the velocity vector for particle i are updated as: Optimizing and training LS-SVM with PSO algorithm

ij  ij  c1q( xijpb  xij )  c2 ( x sbj  xij ), j  1,.., n (11) Testing LS-SVM-PSO model with new data

Where w is the inertia weight, xipb is the best variable vector Computing MSE for( LS-SVM, LS-SVM-PSO, and ANN)

sb
encountered so far by particle i , and x is the swarm best
Fig. 2: The Proposed Model
vector, i.e. the best variable vector found by any particle in
the swarm, so far c1 and c 2 are constants, and q and r are The proposed algorithm was tested for many companies
random numbers in the range (0, 1). Once the velocities have which cover all stock sectors in S&P 500 stock market. These
been updated, the variable vector of particle i is modified sectors are Information Technology (Adobe, Hp, and Oracle);
according to: Financials (American Express and Bank of New York);
Health Care (Life Technologies, and Hospera); Energy
xij  xij  ij , j  1,..., n. (12) (Exxon-Mobile and Duck energy); Communications (AT&T);
Materials (FMC Corporation); Industrials (Honey Well).
The cycle of evaluation followed by updates of velocities
Five technical indicators are calculated from the raw
and positions (and possible update of xipb and x sb ) is then datasets:
repeated until a satisfactory solution has been found. PSO
algorithm is shown in Fig. 1.  Relative Strength Index (RSI): A technical momentum
indicator that compares the magnitude of recent gains to
recent losses in an attempt to determine overbought and
oversold conditions of an asset. The formula for
computing the Relative Strength Index is as follows.
RSI = 100- [100 / (1+RS)] (13)
Where RS = Avg. of x days’ up closes / Average of x
days’‫‏‬down closes.
 Money Flow Index (MFI): This one measures the
strength of money in and out of a security. The formula
for MFI is as follows:
Money Flow (MF) = Typical Price * Volume. (14)

Money Ratio (MR) = (Positive MF / Negative MF). (15)

Fig. 1: PSO Algorithm MFI = 100 – (100/ (1+MR)). (16)

 Exponential Moving Average (EMA): This indicator

IV. THE PROPOSED MODEL returns the exponential moving average of a field over a
given period of time. EMA formula is as follows.
The proposed model is based on the study of historical data,
EMA = [α *T Close] + [1-α* Y EMA]. (17)
technical indicators and optimizing LS-SVM with PSO
algorithm to be used in the prediction of daily stock prices. Where T is Today’s close and Y is Yesterday’s close
Levenberg-Marquardt (LM) algorithm is used as a benchmark
for comparison with LS-SVM and LS-SVM-PSO models. The  Stochastic Oscillator (SO): The stochastic oscillator
proposed model architecture contains six inputs vectors defined as a measure of the difference between the
represent the historical data and derived technical indicators current closing price of a security and its lowest low
and one output represents next price. price, relative to its highest high price for a given period
of time. The formula for this computation is as follows:
%K = [(Close price – Lowest price) / (Highest Price – ‫‏‬
‫‏‏‏‏‏‏‏‏‏‏‏‬Lowest Price)] * 100 (18)
 Moving Average Convergence/Divergence (MACD):
This function calculates difference between a short and a
long term moving average for a field. The formulas for
calculating MACD and its signal as follows.
 Osman Hegazy et al. 20

MACD = [0.075*EMA of Closing prices] – ‫‏‏‏‏‏‏‏‏‏‏‏‏‏‏‏‏‏‏‏‬

(19)
‫[‏‏‏‏‏‏‏‏‏‏‏‏‏‬0.15*EMA of closing prices]
Signal Line = 0.2*EMA of MACD (20)

V. RESULTS AND DISCUSSION

LS-SVM-PSO, LS-SVM and ANN algorithms were trained

and tested with datasets form Jan 2009 to Jan 2012. All
datasets are available in [13]. All datasets are divided into
training part (70%) and testing part (30%).
Fig. 3 to Fig. 14 outline the application of Proposed LS-
SVM-PSO model compared with LS-SVM and ANN-BP
algorithms at different data set with different sectors of the
market. In Fig. 3, Fig. 4, and Fig. 5, which present results of
three companies in information technology sector (Adobe, Fig. 3: Results for Adobe Company
Oracle and HP), results show that LS-SVM optimized with
PSO is the best one with lowest error value followed by LS-
SVM algorithm. Fig. 6 and Fig. 7 represent results of
financial sector (American Express, and Bank of New york),
we can remark that the predicted curve using the proposed
LS-SVM-PSO algorithm is most close to the real curve which
achieves best accuracy, followed by LS-SVM, while ANN-
BP is the worst one.
Fig. 8 represents results of using PSO-LS-SVM model in
Honeywell company which represent industrials stock sector,
proposed model still achieves best performance.
Fig. 9 and Fig. 10 outline the application of the proposed
algorithm to hospera and life technologies companies in
health stock sector. From figures one can remark the
enhancement in the error rate achieved by the proposed
model.
Fig. 11 and Fig. 12 outline the results of testing proposed
model on Exxon-mobile and duke energy companies which Fig. 4: Results for Oracle Company
represent energy stock sector. PSO-LS-SVM also the best
especially in fluctuation cases.
Fig. 13 represents results for FMC Corporation in materials
stock sector. The achievements of proposed model is very
promising compared with SVM and ANN
Fig. 14 outlines results for AT&T from communication
stock sector. We can notice from figure the role of proposed
model in reducing the error rate and overcoming local minima
problems which found in ANN results.
Table 1 outlines Mean Square Error (MSE) performance
function for proposed algorithm. It can be remarked that the
LS-SVM optimized with PSO always gives an advance over
LS-SVM and ANN trained with LM algorithms in all
performance functions and in all trends and sectors. Proposed
model performs better than other algorithms especially in
cases with fluctuations in the time series function.
Fig. 15 outlines comparison between PSO-LS-SVM, LS-
SVM and ANN algorithms according to MSE function.
Fig. 5: Results for HP Company
International Journal of Computer Science and Telecommunications [Volume 4, Issue 12, December 2013] 21

Fig. 6: Results for American Express Co Fig. 9: Results for Hospera Company

Fig. 10: Results for Life Technologies company

Fig. 7: Results for Bank of New York

Fig. 11: Results for Exxon Mobile company

Fig. 8: Results for Honeywell company
 Osman Hegazy et al. 22

Table 1: Mean Square Error (MSE) for proposed algorithm

‫‏‬Algorithm
‫‏‬ PSO-LS-SVM LS-SVM NN-BP
‫‏‬Company

Adobe ‫‏‬0.5317 0.5703 0.8982

Oracle ‫‏‬0.6314 ‫‏‬0.8829 ‫‏‬0.9124
HP ‫‏‬0.7725 ‫‏‬1.2537 ‫‏‬1.9812
American Express ‫‏‬0.7905 ‫‏‬1.0663 ‫‏‬2.8436
Bank of New york ‫‏‬0.4839 ‫‏‬1.2769 ‫‏‬1.9438
Coca-Cola ‫‏‬0.6823 ‫‏‬0.9762 ‫‏‬1.7975
HoneyWell ‫‏‬0.9574 ‫‏‬1.3371 ‫‏‬2.1853
Hospera ‫‏‬0.8694 ‫‏‬0.9320 ‫‏‬1.4640
Fig. 12: Results for Duke Energy company Life Tech. ‫‏‬0.7713 ‫‏‬1.3221 ‫‏‬1.3492
Exxon-Mobile ‫‏‬1.1000 ‫‏‬1.6935 ‫‏‬2.4891
AT & T ‫‏‬0.2911 ‫‏‬0.4673 ‫‏‬0.4055
FMC Corp. ‫‏‬1.5881 ‫‏‬2.1034 ‫‏‬3.5049
Duke Energy ‫‏‬0.1735 ‫‏‬0.6097 ‫‏‬0.6010

4
PSO-LSSVM LSSVM NN-BP
3.5
3
2.5
2
1.5
Fig. 13: Results for FMC corporation
1
0.5
0

Fig. 15: MSE of proposed algorithm

VI. CONCLUSIONS

This paper, proposed a machine learning model that

integrates particle swarm optimization (PSO) algorithm and
Fig. 14: Results for AT &T Company
LS-SVM for stock price prediction using financial technical
indicators. These indicators include relative strength index,
money flow index, exponential moving average, stochastic
oscillator and moving average convergence/divergence The
PSO is employed iteratively as global optimization algorithm
to optimize LS-SVM for stock price prediction. Also, PSO
algorithm used in selection of LS-SVM free parameters C
(cost penalty), ϵ (insensitive-loss function) and γ (kernel
parameter). The proposed LS-SVM-PSO model convergence
International Journal of Computer Science and Telecommunications [Volume 4, Issue 12, December 2013] 23

to the global minimum. Also, it is capable to overcome the SVR for stock index forecasting”, ELSEVIER, Decision
over-fitting problem which found in ANN, especially in case Support System, 2012.
of fluctuations in stock sector. PSO-LS-SVM algorithm [18] Bernardo, D.:“An interval type-2 Fuzzy Logic based system
parameters can be tuned easily. The performance of the for model generation and summarization of arbitrage
opportunities in stock markets”, IEEE Computational
proposed model is better than LS-SVM and compared Intelligence (UKCI), 2012 12th UK Workshop, pp. 1-7., 2012.
algorithms. LS-SVM-PSO achieves the lowest error value [19] Ochotorena, C.N. :“Robust stock trading using fuzzy decision
followed by single LS-SVM, while ANN-BP algorithm is the trees”, IEEE Conference on Computational Intelligence for
worst one. Financial Engineering & Economics (CIFEr), pp. 1-8, 2012.
[20] Yi X., Jin X., Fengbin L. & Shouyang W. :“Ensemble ANNs-
REFERENCES PSO-GA Approach for Day-ahead Stock E-exchange Prices
Forecasting”, International Journal of Computational
[1] Olivier C., Blaise Pascal University: “Neural network Intelligence Systems, Volume 6, Issue 1, pp.96-114 2013.
modeling for stock movement prediction, state of art”. 2007 [21] Nayak, S.C. :“Index prediction with neuro-genetic hybrid
[2] Leng, X. and Miller, H.-G. : “Input dimension reduction for network: A comparative analysis of performance”, IEEE
load forecasting based on support vector machines”, IEEE International Conference on Computing, Communication and
International Conference on Electric Utility Deregulation, Applications (ICCCA), pp. 1-6, 2012.
Restructuring and Power Technologies (DRPT2004), 2004. [22] Fazel Z.,Esmaeil H., Turksen B.: “A hybrid fuzzy intelligent
[3] Vapnik, V., “The nature of statistical learning”, second agent-based system for stock price prediction”, International
edition, ©Springer, 1999. Journal of Intelligent Systems, Volume 27, Issue 11, pages
[4] Cherkassky, V. and Ma, Y., “Practical Selection of SVM 947–969, 2012.
Parameters and Noise Estimation for SVM regression”. [23] Honghai Y., Haifei L.: “Improved Stock Market Prediction by
Neural Networks, vol., 17, pp. 113-126, 2004. Combining Support Vector Machine and Empirical Mode
[5] Suykens, J. A. K., Gestel, V. T., Brabanter, J. D., Moor, B.D Decomposition”, Fifth International Symposium on
and Vandewalle, J. “Least squares support vector machines”, Computational Intelligence and Design (ISCID), pp. 531 -
World Scientific, 2002. 534 , 2012.
[6] ANDRÉS M., GENARO DAZA, S., CARLOS D., GERMÁN [24] Lingzhi W., Jiansheng W.: “Neural Network Ensemble Model
C. : “Parameter Selection In Least Squares-Support Vector Using PPR and LS-SVR for Stock Market Forecasting”,
Machines Regression Oriented, Using Generalized Cross- Springer, Advanced Intelligent Computing Lecture Notes in
Validation” , Dyna, year 79, Nro. 171, pp. 23-30. Medellin, Computer Science Volume 6838, pp 1-8, 2012.
February, 2012. [25] İhsan K., Cengiz K., Selçuk Ç. :“Computational Intelligence
[7] Carlos A. Coello‫‏‬, Gary B. Lamont‫‏‬, David A. van Veldhuizen‫‏‬: Techniques for Risk Management in Decision Making”,
“Evolutionary Algorithms for Solving Multi-Objective Springer, Handbook on Decision Making Intelligent Systems
Problems”,Springer, 2007. Reference Library Volume 33, pp 9-38, 2012.
[8] D. N. Wilke. “Analysis of the particle swarm optimization [26] Gupta, A. :“Stock market prediction using Hidden Markov
algorithm”, Master's Dissertation, University of Pretoria, Models”, IEEE Engineering and Systems (SCES), 2012
2005. Students Conference on, pp.1-4, 2012.
[9] Khalil A.S.: “An Investigation into Optimization Strategies of [27] Lipo W., Shekhar G.: “Neural Networks and Wavelet De-
Genetic Algorithms and Swarm Intelligence”. Artificial Life Noising for Stock Trading and Prediction”, Springer, Time
(2001). Series Analysis, Modeling and Applications Intelligent
[10] Kennedy J., Spears W.M.: “Matching Algorithms to Systems Reference Library Volume 47, pp 229-247, 2013.
Problems”, An Experimental Test of the Particle Swarm and [28] Suykens, J.A.K.; Vandewalle, J. "Least squares support vector
Some Genetic Algorithms on the Multimodal Problem machine classifiers", Neural Processing Letters, 9 (3), 293-
Generator. Processes. Current as of December 15th, 2003. 300., 1999.
[11] Chun C, Qinghua M, Shuqiang L.: “Research on Support
Vector Regression in the Stock Market Forecasting”
©Springer, Advances in Intelligent and Soft Computing
Volume 148, , pp 607-612, 2012.
[12] Guo Z., Wang H., Liu Q. : “Financial time series forecasting
using LPP and SVM optimized by PSO” © Springer, Soft
Computing Methodologies and Applications , December 2012.
[13] http://finance.yahoo.com.
[14] Xie, G.: “The Optimization of Share Price Prediction Model
Based on Support Vector Machine”, International Conference
on Control, Automation and Systems Engineering (CASE),
pp.1-4., 30-31 July 2011.
[15] Huang C.; Huang L.; Han T.: “Financial time series
forecasting based on wavelet kernel support vector machine”
IEEE, Eighth International Conference on Natural
Computation (ICNC), 2012.
[16] Jui Y.: “Computational Intelligence Approaches for Stock
Price Forecasting”, IEEE International Symposium on
Computer, Consumer and Control (IS3C), pp. 52 - 55, 2012.
[17] Ling J., Chih C., Chi L, Chih H., “A hybrid approach by
integrating wavelet-based feature extraction with MARS and