A New Wrapped Ensemble Approach for Financial Forecast

2.2.1 Moving Average

The moving average (MA) is based on the concept of average cost, which is part of the Dow Jones theory. It connects the average price of a period to a line to display the history of the stock price’s fluctuations and then reflect the future trends of the stock price. The SMA formula for a period of n is presented in Formula (1):

2.2.2 Relative Strength Index

The RSI, an indicator proposed by Welles Wilder in 1978 [25], is used to measure the relative strength of securities. It constraints its value to 0 to ∼100; generally, if its value is higher than 70, we can speculate that the market is in condition of overbought; conversely, we can get the signal that the market is oversold if its value is lower than 30. The RSI is computed using Formula (2):

where AvgRise is the average rise value during n periods and AvgFall is the average fall value during n periods. Welles Wilder has proved that parameter n is generally 14. This article also adopts this parameter.

2.2.3 Stochastic Oscillator

The term stochastic oscillator, namely, %K and %D, was first used by George C. Lane in 1950 [12]. It was used to forecast time price trend reversal by comparing the range of the close price with the other prices. It is made up with two lines, the fast (%K) and the slow (%D) [Formulas (3) and (4), respectively]. Closing levels that are consistently near the top of the range indicate accumulation (buying pressure) and those near the bottom of the range indicate distribution (selling pressure).

In the above formula, RSV is the raw stochastic value, C_n is the close price of the n^th day, H_n is the highest price of the n^th day, and L_n is the lowest price of the n^th day; in this article, we adopt the most common parameters, α=1/3 and n=9.

2.2.4 Moving Average Convergence and Divergence

The MACD indicator, proposed by Gerald Appel in 1970 [1], is a technical tool that is used to judge the intensity, direction, energy, and trend cycle of stock prices and then to support the decision of sell and buy. The MACD often consists of two lines (DEM and DIF) and one chart (an MACD bar). To use the MACD indicator, there are three parameters that have to be set, and in this article, we used 12 and 26 for the calculation of the DIF and 9 for the DEM.

For this indicator, we did not use the MACD bar directly but we took the DIF and the DEM as two input variables.

2.3.1 Genetic Algorithm

The main idea of GA comes from the theory of evolution proposed by Darwin, i.e., in natural selection, only the fittest can survive. It works with a set of candidate solutions called population and generates successive populations of alternate solutions that are represented by a chromosome [7]. If associated with the characteristics of exploitation search, GA can efficiently deal with large search spaces and hence has a less chance of obtaining a local optimal solution compared with other algorithms [8]. In the current work, we adopted the GA algorithm to help obtain the optimal feature combination, as did Huang and Wang [8] in 2006. However, we only used the algorithm for feature selection, not for parameter optimization, because we wanted to use simpler and more rapid methods, such as PSO, to optimize the parameters.

2.3.2 Particle Swarm Optimization

PSO is a population-based stochastic optimization technique developed by Eberhart and Kennedy in 1995 [6] and inspired by animal social behavior, as exemplified by a flock of birds or a school of fish. The adjustment toward pbest and gbest by the particle swarm optimizer is conceptually similar to the crossover operation used by GAs. It uses the concept of fitness, as do all evolutionary computation paradigms [20]. Compared with other algorithms, PSO provides faster and better results with lower cost.

The PSO we used adjusts the velocities using the following formula:

In this article, we used the PSO to achieve better parameters for forecast algorithm, and we set self-learning and group learning coefficients c₁ and c₂ as 0.2, the initial particle number as 60, and the weight coefficient w as 1.0.

2.3.3 RBF Network

The essence of the RBF network is to convert the inputs from one space to another. It is divided into two states, finishing a mapping: ξ:R^m→R^p. The architecture is displayed in Figure 1. Each neuron i of the hidden layer consists of a center μ_i and width σ_i. The Euclidean distance between the input signal S and the center of the neuron is first evaluated; then an activation function, i.e., the basis function (BF), is applied [8]. The RBF network has the best unique approximation and better convergence speed. In this article, the parameters are wrapped in the model [16].

Figure 1

Architecture of an RBF Network.

3.2.1 Inputs for Sequence Data

When building the sequence data model, we only use the daily close price sequence data. However, we will not use the close price sequence directly; on the contrary, we will deal with the data using the financial method given in Formula (11) in order to decrease the errors coming from data noise. The method that we adopted was also used by others to make sequence prediction.

In Formula (11), we did not use the simple return; instead, we used the log return; R_t+1 is the simple return of day (t+1), P_t is the price of day t. The log return has many advantages compared with the simple return. First, with the log return, it is easier to calculate the multiple-periods return; all we need to do is calculate the simple addition of each period return. Second, the log return has a statistical property that is easy to deal with.

3.2.2 Inputs for Indicator Data

To build the indicator data model, we used the daily open price, highest price, lowest price, and close price data. However, we did not directly use the sequence data because the original data contain a lot of noise; on the contrary, we adopted some technical indicators, which we introduced above. In addition, we also used other indicators adopted in pioneer research. We extracted some features from the price of history data as inputs for each day t; some of the inputs were also used in the works of Kwon and Moon [10] and Teixeira and de Oliveira [21].

Parts of the inputs are reported in Table 1, where P_o(t), P(t), P_h(t), and P_L(t) indicates the stock’s open, close, high, and low price, respectively, on day t; V(t) is the volume of trades on day t; Bollinger_High and Bollinger_Low are the upper and lower Bollinger bands, respectively; MA_s and MA_l are the short- and long-term simple moving average applied to price; and VMA_s and VMA_l are the simple moving average of short- and long-term forecast applied to volume.

Before using the inputs introduced in Table 1, we also normalized the inputs to avoid the effects between the bigger and the smaller numbers. Here we adopt the Z-score standard method using the following formula:

where Z_ij, X_ij, and σ_i representing the number after normalization, the number before normalization, the average value of the inputs i, and the standard deviation of inputs i, respectively.

3.2.3 Wrapped Hybrid RBF Network

In a previous study, the forecast and optimization processes are often separated [16], and then, their models are only fitted with the fixed period, i.e., if the periods are changed, their model may fail. Thus, in this article, we propose a wrapped hybrid RBF model (WHR) based on the new ensemble method proposed above. The WHR is made up of three parts, the feature selection model, the forecast model, and the parameter optimization model; however, because the three models have a unique object that is subject to features and parameters, the three parts are not independent but are an organic whole. In this model, the features and parameters are all dynamically changed, and each change contributes to a systematic error. The goal in this hybrid model is to search for a relatively optimal global model.

In the feature selection parts shown in Figure 2, we adopted the GA, but unlike other studies that use complex coding, we used common binary coding to represent the features to be included or excluded. Binary coding not only perfectly and simply represents the feature but also improves the efficiency. In our experiment, the chromosome length is 74, which is the number of features, the crossing rate is 0.25, and the mutation rate is 0.01, and it is worth mentioning that we used a mean square error (MSE) as the fitness function of GA.

Figure 2

The Wrapped Hybrid Forecast Model.

In the parameter optimization parts shown in Figure 2, the PSO algorithm was used. PSO is a highly efficient search algorithm that has a fast convergence rate and fewer parameters to set, and more importantly, it does not need any encoding or decoding. In our experiment, the self-learning factor and the group learning factor were all set as 2.0, whereas the maximal inertia weight was set as 0.9 and the minimal inertia weight as 0.4. The object function is similar to the fitness function in the GA for the wrapped model with unique training target.

The forecast model in Figure 2 was designed based on the new ensemble method for stock data proposed in Section 3.1. First, we used the indicator RBF model to estimate the future price then we took advantage of the time series RBF model to estimate the future residual error; then the final price was obtained using Formula (10).

As a whole, our model uses a uniform evaluation indicator for feature selection and parameter optimization, and the search process consists of two cycling, one for the parameter search and another for the feature search. As for the stop of the search process, it was set at the point when the value of the evaluation indicator does not lessen for a time during the search; the time we used in our experiment is 100.

3.2.4 Evaluation Indicator

In a previous study, all types of evaluation indicators were used, and the common evaluation indicators are listed in Formulas (13)–(16), which are mean absolute error (MAE), mean absolute percent error (MAPE), mean square error (MSE), and root mean square error (RMSE). Pai and Lin [18] have applied them to one of their studies on stock price forecast; Lu and Wu [15] also used them in their study on financial forecast in 2011.

However, these traditional evaluation indicators have their limitations; unlike prediction in the traditional field, where we do not need to consider the direction of the prediction result, stock price prediction has its own characteristics and the accuracy of stock price movement direction counts more than traditional prediction. If we ignore the data given above, we may obtain a model that has low system error but is useless. Some researchers also use the direction accuracy indicator and the system error indicator at the same time to overcome the limitations of the traditional indicator; however, their method has changed the problem to a multiobjective optimization problem, thus bringing more problems.

In this article, we designed a new indicator to provide the model with comprehensive evaluation not only to overcome the limitations of the traditional indicator but also to maintain it as a single-objective optimization problem. We call this the weighted mean absolute error (WMAE):

where α is the penalty factor, d_t is the real value, and z_t is the prediction value. In this article, α was set as 5.0. Based on the traditional indicator, the WMAE adds a penalty factor to amplify the loss brought about by mistakes in direction judgment.

4.1.1 Data Set

The data in this article were from the China Shanghai and Shenzhen stock exchange database (http://chart.yahoo.com/table.csv?s=000300.SS&a=6&b=12&c=1998&d=6&e=9&f=2013&g=d&q=q&y=0&z=000300.SS&x=.csv). For our experiment, we used 7 years of data (from January 2, 2006, to December 31, 2012).

4.1.2 Sliding Time Window

In our experiment, we used the sliding time window, which is usually used in the analysis of time series data. Tsai and Hsiao [22] have used this method in their study on feature selection. Teixeira and de Oliveira [21] also used this method in stock forecasting, but their window model was static and did not have a dynamic property; thus, their model will only be fit up to a certain period and will not adapt to changes in market situation.

To improve the adaptivity of the model, we tried to improve the traditional window used in most studies. Table 2 displays the detail of the sliding time window, which comprised training data and test data but without validation data. The reason is that the step-forward forecast is different from the multistep-forward forecast, and if we use the data between train data and test data as the validation data, we will unconsciously have some information loss; however, the stock price is usually closely linked with the most recently generated data; thus, information loss is rather great. The approach we proposed, however, avoids that problem. Based on professional experience, the trading data, as reflected by the candlestick chart, will repeat itself, and if we consider the train data and validation data as a whole, then we can use the train set to train and check the model. In Table 2, we assumed that the length of the training set T (which will be automatically determined by the algorithm according to the market) is five, and we can use the data of T day to train and check the model and make a forecast.

4.1.3 Experimental Procedure and Result Analysis

In our experiment, we used the Hushen 300 index to make a forecast. First, we used sequence data to experiment on the wrapped RBF algorithm and forecast the probable value of yield, and then based on real data, we calculated the residual error, which is defined as MAE [Formula (13)]. Next, we used the indicator data to train and test the model, and to smooth and standardize the data, we used the yield rather than the rate of return as the forecast label. Meanwhile, we constructed the train record for sequence data based on the forecast result of the value of yield, then trained and tested the model, and then forecasted the residual error of indicator forecast. Lastly, the forecast value of the ensemble model was calculated by the forecast data and the forecast residual error. According to Formula (10), we separated the forecast value into two parts. In our experiment, we used the indicator data to forecast the value of yield and the sequence data to forecast the residual error of the earlier indicator forecast.

To make a comparison, we also performed the experiment using a single method, without using the wrapped method or the ensemble method. We used a diagram to visualize the forecast data, and the results are shown in Figure 3, which displays the volatility of real and forecast data.

Figure 3

Volatility Forecast Map.

However, if solely based on Figure 3, we could not determine the better result. Thus, to determine which result is better, we introduced a new concept, the cumulative absolute error (CAE):

Using Formula (18), we can draw the CAE map (Figure 4), which shows the CAEs of the different approaches. Using the map, we can see that the wrapped ensemble method (WEM) is better than other models.

Figure 4

Cumulative Absolute Errors Map.

To see the forecast of the real price, we need to convert volatility to price. The calculation method of price, given in Formula (19), is opposite to the method of volatility:

where P_t+1 is the future price, r_t is the volatility, and P_t is the latest price.

The comparisons of model precision are given in Table 3, which shows that the WEM has the best performance. The sequence wrapped RBF (SWRBF) and the indicator wrapped RBF perform better than the sequence wrapped SVR (SWSVR) and the indicator wrapped MLP (IWMLP), and this is the reason why we used the RBF network algorithm to build the ensemble model. Our experiment proved that the wrapped ensemble model performs better than the model without ensemble and that the wrapped model has a better parameter. In addition, in comparison with the bagging model, i.e., the bagging of RBF, MLP, and SVM, we can see that our model is still superior.