Journal of Computational Methods in Sciences and Engineering 16 (2016) 599–615 599

DOI 10.3233/JCM-160643
IOS Press

Chaotic time series prediction by artificial

neural networks

Marjan Kuchaki Rafsanjani∗ and Meysam Samareh

Department of Computer Science, Faculty of Mathematics and Computer, Shahid Bahonar University
of Kerman, Kerman, Iran

Abstract. In this paper, we used four types of artificial neural network (ANN) to predict the behavior of chaotic time series.
Each neural network that used in this paper acts as global model to predict the future behavior of time series. Prediction process
is based on embedding theorem and time delay determined by this theorem. This ANN applied to the time series that generated
by Mackey-glass equation that has a chaotic behavior. At the end, all neural networks are used to solve this problem and their
results are compared and analyzed.

Keywords: Artificial neural network (ANN), chaotic systems, Mackey-glass (MG), prediction, time series

1. Introduction

Time series are events (or observations) that occur over time. Chaotic time series are decisive system,
but it has a lot of complexity [1]. The behavior of the chaotic time series has high similarity to the natural
systems, and it shows the dynamic of natural systems [2]. One special feature of chaotic systems is the
butterfly effect it means that, a small change in the initial state will have a large impact on the final state
of the chaotic systems. Prediction these systems can be useful where its behavior is described by chaotic
time series and a proper database of natural system’s behavior be available. This field of science has
recently attracted the attention of researchers [3]. Nonlinear time series prediction is applied in various
sciences, such as oceanography, meteorology, medicine, geophysics, as well as in economics, finance
and management [4].
Many methods are used to forecast the nonlinear and dynamic (chaotic) time series, in the an-
alytical methods like autoregressive (AR), threshold AR, autoregressive moving average (ARMA),
nonlinear auto regressive moving average (NARMA) and integrated autoregressive moving average
(ARIMA) [5–8]. Excepting these methods, recently the intelligent methods are used as predictors. For
example (ANNs), heuristic methods, Support vector machine (SVMs), nearest neighbor regression, hid-
den Markov model, fuzzy inference system (FIS) and case base reasoning [9–13].
This paper is organized as follows. Section 2 gives a primary description of time series and their pre-
dictions. Section 3 reviews different types of artificial neural networks, including multilayer perceptron,
recurrent, dynamic and radial based function neural networks. In Section 4 the experiments and results
will be discussed and finally conclusion is in Section 5.

∗
Corresponding author: Marjan Kuchaki Rafsanjani, Department of Computer Science, Faculty of Mathematics and Com-
puter, Shahid Bahonar University of Kerman, Kerman, Iran. E-mail: [email protected].

c 2016 – IOS Press and the authors. All rights reserved

1472-7978/16/$35.00 
600 M. Kuchaki Rafsanjani and M. Samareh / Chaotic time series prediction by ANNs

2. Time series prediction

Predict of a dynamic and nonlinear system (natural) is approximation of future behavior based on past
behavior of the system. That means that by having x0 , . . . , xn as time series that represent the behavior
of the natural system calculate the future value xn+h . The h determines the number of prediction steps
that known as prediction horizon. One step ahead is predicting a next value of the time series and multi
steps ahead or long-term prediction is a long prediction horizon [14]. Taken’s embedding theorem in a
chaotic system is used for information that show the evolution of the system over time [15]. Equation (1)
shows progress of a time series:

x(n + 1) = F [x(n)] (1)

Where x(n) is a vector space ℜ that represents the phase of the chaotic system at the time step n, F [·]
is the nonlinear function that F : ℜn → ℜn . The future value of the chaotic system at time t + τ is as
Eq. (2):

x(t) → F [x(t)] = x(t + τ ) (2)

The behavior of nonrandom complex system manufacturing a curve that its popular attractor. If all
values of a natural system such as time series is not available, nevertheless could be determined the
state of the system, then under use the embedding theorem the attractor determines the phase system.
Equation (3) is used for predicting the future of the chaotic system [16–19].

Y (t) = [x(n), x(n − τ ), x(n − 2τ ), . . . , x(t − (d − 1)τ )] (3)

Where x(n) is the value sample of the chaotic time series, d is the embedding parameter and τ is the
embedding time delay.

3. Artificial neural network (ANN)

In this section, the ability of ANNs will have checked to simulate the natural system. The ability of the
ANNs for detection the nonlinear relation between inputs and outputs in the training set prepares them
for natural system’s reconstruction [20]. ANNs are powerful tools for function approximation that have
the additional capability of learning from training data and is resistant to noise and incomplete inputs. It
should be noted that ANNs have free parameters that must be tuned and this tune is a research field.

3.1. Multilayer perceptron

Multilayer perceptron (MLP) can be used in pattern recognition and function approximation problems,
and also with an additional time delay input it can use for predicting time series problems. An MLP
with bias and transfer function Sigmoid in hidden layers can approximate every function with finite
discontinuity points. MLPs are trained commonly with back propagation algorithm (BP) so that, errors
propagate among neural network and to allow the ANN to update their weights and biases [21–23].
Inputs to this ANN are generated by Mackey-glass (MG) and targets are prepared based on embedding
theorem and their architecture is feed forward; in this architecture there are three units that in the first
unit where inputs are delivered to it, in the second unit or the hidden unit the first unit is connected to
 M. Kuchaki Rafsanjani and M. Samareh / Chaotic time series prediction by ANNs 601

w 11 w11′
x1
w12 w12′
v1
x2 v2
• • •
• • •
• • •
vn

wnn ′
wnn
xn
input − layer hidden − layers output − layer

Fig. 1. The configuration of feed forward neural network [26].

the third where it receives weight from before unit and deliver the next unit and finally, after learning
′ ’s
the third unit delivers the output [24,25]. See the Fig. 1, in this figure xi ’s are the inputs, the wij ’s, wij
and vi ’s are the that weights are updated at every iteration, attention to these points is important that the
output layer has an output and if necessary the output unit can have more neurons. The second parameter
that must be noted is the concept of Biases that was removed in each layer to avoid the complexity in
the Fig. 1 and finally the hidden layer has several layers that here just is plotted one layer. This figure
summarizes in Eq. (4):
⎡   ⎤
  
Y= v1 F ′ ⎣ ′
wjl F wij xi + b1j + b2k ⎦ + b31 . (4)
1 j i

That Y is the value of output, [wij ] is the weights from input neuron to the hidden layer, [wjl ′ ] is

the weights from hidden layer to the hidden layer, [vl ] is defined similarly, bij is the biases and the F
is the Sigmoid function for hidden layer and F ′ could be linear or a Sigmoid function for the output
layer [26–28].

3.2. Recurrent neural network

Recurrent neural networks (RNNs) are similar MLP, but with this difference, that it has feedback or
delay in their architecture. This form of ANNs commonly named dynamic ANNs versus static ANNs
where only the output unit determines the ANN output and there do not exist any feedback from the
output layer (or delay) to others layer [24,25]. RNNs are robust and have a better performance than
static ANNs, but their train process is difficult and complex [29]. As RNNs or dynamic ANNs have a
memory so can learn sequential patterns and variable at time patterns [30].

3.2.1. Elman
The Elman neural network is one member of the class the RNNs whereas this ANN is a feed forward
neural network, moreover a recurrent unit that this unit determines the step-ahead time delay in different
application. Roughly speaking, the Elman is the BPFF neural network with two layers that has addi-
tional feedback from the output of the hidden layer to the input of the hidden layer [30]. This connected
back (feedback) is useful to recognize the temporal patterns and also creation and recognize the variable
patterns in time [31,32]. This kind of RNNs is capable of using appropriate transfer functions in its hid-
den layer and output layer approximates every function with finite discontinuous point [33]. The Elman
602 M. Kuchaki Rafsanjani and M. Samareh / Chaotic time series prediction by ANNs

w 11 w11′
x1
w12 w12′
w1 v1
x2
w2 v2
• • •
• • •
• • •
vn
wnn ′
wnn
xn hidden − layers output − layer
input − layer wn

Fig. 2. The configuration of elman [16].

neural network is used for prediction of time series with six delays follow embedding theorem [16,34].
In the Fig. 2 similar the Fig. 1, the inputs and weights marked, and in this figure also the biases have
been removed. The only difference of recurrent neural network with feed forward neural network is the
having a recurrent and delay unit; and this unit acts as a tapped delay and its input is the output of the
hidden layer and output acts as a feedback to the inputs of the hidden layer. This unit can be several
neurons for each neuron of hidden layer that in figures of this paper are shown by one output neuron.
This figure is summarized in the Eq. (5).
⎡  ⎤
   
Yk = F ⎣ bi + vji F ′ bj + wmj xm + ′
wlj wj (k − 1) ⎦ (5)
i j m l

The parameters of Eq. (5) are determined in Fig. 2, where bi are biases, F ′ and F are the hidden, output
function and the neural network weights is according the Eq. (4) [16,31,35]. Note that the equation has
several outputs.
3.3. Dynamic neural network
As described in Section 3.2 the dynamic neural networks (DNN) is similar to the RNNs by difference
that the RNNs have a feedback from some layers to other layers, but in the DNNs there exist a time
delay on the input of some layers, however some neural networks have a mixture of both especial time
delay and feedback. Both ANNs are members of a recurrent class of ANN, but in this paper DNNs are
considered as two separate groups [24,25,29].

3.3.1. Layer recurrent neural network

The layer recurrent neural (LRN) network is the simplex type of the DNNs that introduced by Mr.
Elman. In the LRNs every layer, excluding the end layer there is the feedback mid a one unit delay.
The Elman neural network is a two layer feed forward neural network and this ANN is shown in Fig. 2
that train with an approximation of BP algorithm, but the LRN is similar to the Elman with this dif-
ference, that LRNs can be multilayer and also every layer except output layer has a delay unit, see the
Section 3.2.1. The figure of the architecture is the same Fig. 2 by the difference that the delay unit
acts on every of the middle layer (hidden layer) and this neural network can be multilayer. LRNs have
application in filtering and modeling [25].
 M. Kuchaki Rafsanjani and M. Samareh / Chaotic time series prediction by ANNs 603

w11′′
x1
w1 w12′′
w1′ v1
x2
v2
w2 w2′
• • •
•
• •
•
delay − unit wn′ • • vn
wn

′′
wnn
xn
output − layer
input − layer
hidden − layers

Fig. 3. The configuration of FTDNN [25].

3.3.2. Focus time delay neural network

Focus time delay neural network or summarizes FTDNN is the simplest type of the DNNs consists
a feed forward neural network with a time delay in the first layer; this ANN is a class of the DNNs
namely focus neural network [36]. A feed forward neural network that was used in this neural network
could be multilayer and Sigmoid function is used as a transfer function, but only the first layer has a
delay time [37]. One of the special features of the FTDNNs is a lake of requirement to the dynamic BP
to compute the gradient, because the delay layer there is only to the input layer, it does not include the
feedback connections, this argumentum is that the rate of learning FTDNNs is faster than other RNNs.
Figure 3 is a diagram of the FTDNN; in this figure the inputs firstly entered to the delay unit and then
delayed to the hidden layer is applied, also biases and the delay unit is ignored and output layer can
include several neurons that here is plotted squarely.

3.3.3. Time delay neural network

FTDNNs are using a time delay in the first layer that could use time delay in the other layer where this
neural network is distributed time delay neural network or TDNN [38]. The first application of ANNs
was acoustic signals detection that primary purpose is to work with sequential data, so it’s used in speech
recognition. As this neural network could use the time delay in each layer except first layer so this cause,
the neural network train process is very slow and should use the dynamic BP algorithm for learning [14].
TDNNs can be multilayer that each layer has the time delay wherein the first phase input layer, the time
delay effects on the inputs, in the other phase the time delay effects on the output of the last phase and
gives the data to the next phase, but there is no feedback on this type of neural networks [39], If in Fig. 3
for each layer be a delay unit, then this new neural network is a TDNN.

3.3.4. Nonlinear autoregressive with exogenous input

So far, three types of neural networks were studied, where the time delay unit can be in some layers,
every neural network that has nonlinear feedback and with exogenous input namely the dynamic recur-
rent neural network. ANNs that have the connection feedback to the other layer named nonlinear auto
regressive with exogenous input (NARX). NARXs are a main class of discrete time nonlinear system
where is modeled as Eq. (6):
y(t + 1) = F [y(t), y(t − 1), . . . , y(t − dy + 1); u(t), u(t − 1), u(t − du + 1)] (6)
604 M. Kuchaki Rafsanjani and M. Samareh / Chaotic time series prediction by ANNs

u1 w1 w11′′
x1
u2 w2 v1
x2
• v2
•
wn xn • •
• •
• • •
un

y1 w1′ x1′
y2 w2′ output − layer
x 2′
•
• wn′ x n′ • • v2n
• • •
yn delay − unit • •
′′nn
w2,2
input − layer hidden − layers delay − unit

Fig. 4. The configuration of NARX [25].

That u(n), y(n) ∈ ℜ and dy , du  1, (dy  du ) where dy is the output delay and du is the input
delay and the F [·] is the unknown function that could be approximated or is a black box similarity the
MLPs [16,39]. NARX is a powerful class of ANNs as computational power is similar to the Turing
machine [14,40] and this ANNs very suitable to predict the output signal using input signal and used in
the nonlinear filtering for the abandon the noise and this is used to modeling the dynamic systems [14,
41,42].
Now, discussion is focused on the differ architecture of NARXs where there is two class configuration,
the first is parallel that has feedback from output to the ANN and in here the output of this ANN can be
an approximation of the nonlinear dynamic system [14,16]. The second configuration is series-parallel
where the true outputs used instead of feedback, so the inputs and approximate outputs applied to the
neural network by delay and there is not feedback. This configuration is feed forward and could use the
static back propagation [14,16,43]. In Fig. 4, the inputs are same the variable that expressed in Eq. (4),
in this configuration where the output feedback with delay to the hidden layer, but if the feedback and yi
imposed, then this architecture converted to the series-parallel architecture, in the figure also the biases
imposed.
NARX equation is presented in Section 3.3, because this ANN is more general than the other dynamic
ANNs, and the equation of other is similar to the NARX equation; see the Eq. (7).
⎡ ⎛ ⎞⎤
  
y(n + 1) = F ⎣b0 + vi F ′ ⎝ bi + wji u(n − j) + ′′
wki y(n − k)⎠⎦ (7)
i j k

Where the parameters of the Eq. (7) are determined in Fig. 4 that F ′ and F are the hidden and output
function, the bi s are biases and finally the neural network weights are according the Eq. (4) [14,16,28].

3.4. Radial basis functions

Since a problem in the original space is not linearly separable, but if this problem mapped to the
other space may be converted to the linearly separable problem. This is the idea behind of radial basis
 M. Kuchaki Rafsanjani and M. Samareh / Chaotic time series prediction by ANNs 605

input − weight
x1
w11′

w1 w′12
x2
w2
•
• •
•
• •
•
• •
wn output − layer

xn w′nn

input − layer hidden − layers

Fig. 5. The configuration of RBF [25]. Fig. 6. Mackey-glass chaotic time series.

functions or RBFs. RBFs include the functions that the output depends only on the distance of the inputs
from the central point, and a RBF has such like Eq. (5) as a univariate function, but could combine some
RBF:

f (x) = Φ(x − µ) (8)

Where x, µ are the n-dimensional and µ called the central of the RBF and · is Euclidean distance.
These functions can use for interpolation and could mention function as RBFs such as Gaussian func-
tions, multi-quadric functions, thin-plate, logistic basis function and so on [44].

3.4.1. Radial basis functions neural network

RBF neural networks based mapping input space to feature space where the problem has become a
problem is linearly separable. RBF neural network than the BPFFs are needed to the more neuron, but
design it requires less time than the BPFFs. Its performance will be good when big training data are
available [45]. Two used functions are in the following, radial basis (RB) and exact radial basis (RBE).
RBE could create the trained ANN by zero error. RB iteratively adds neurons to the ANN as long as
stop criterion is true. The RBFNNs are commonly two layers. RBFNNs have very application such as
classification, clustering, function approximation and time series prediction [46–48]. Figure 5 shows a
RBFNN, a different of RBFs with other ANNs is the compute the distance of between the inputs, and
also uses of the radial based function as activation function. In this figure, the biases are imposed. The
equation of RBFs is similar to the Eq. (4) by difference that the radial based function operates on inputs
before applying the weights.

3.4.2. Generalized regression neural network

Generalized regression neural networks or GRNNs are similar to the RBFNNs, but by few differences,
it is a nonlinear regression based on neural network [49]. GRNNs architecture are similar the RBFs
videlicet have additional hidden layer input and output layer, where in this layer composed two sections
that in the first that called radial base layer used from RBF and in the second unit that called special linear
layer used the dot product elements and utilized the linear function for transfer to output layer [50,51].
GRNNs as a RBF have very application that more used for function approximation and prediction. The
architecture of GRNN is similar to the RBFNN with a difference that hidden layer is linear.
606 M. Kuchaki Rafsanjani and M. Samareh / Chaotic time series prediction by ANNs

Fig. 7. The Mackey-glass attractor.

4. Experiments, results and analysis

Now, it’s time that these ANNs simulate and contrast the outputs; in this paper, for simulation we used
MATLAB (R2011b) software and predicted the Mackey-glass (MG) time series as a chaotic time series,
MG time series is already predicted by other methods for improving the result [16,18,26,31,52–64].
Mackey-glass time series is used for medical purposes [14,16], it’s used here for generating data of
dynamical system, the Mackey glass chaotic time series equation is:
dx(t) 0.2x(t − τ )
= − 0.1x(t) (9)
dt (1 + x(t − τ ) ∧ 10)
Where x(0) = 1.2, τ = 17 and the default x(t) = 0 when t < 0. The diagram of MG is plotted in Fig. 6.
And so the figure of attractor of this equation is as follows (Fig. 7).
There are very measures of evaluating the performance of the ANNs that in this paper, we used two
following measures for evaluating ANNs performance:
Mean square error or MSE and root mean square error or RMSE that are calculated by following
equations:
1
MSE = (yi − ti )2 (10)
N
1
RMSE = (yi − ti )2 (11)
N
Where N is the length of data, yi is the predict value and ti is the target value [14,31].
To continue, each figure is made up of four parts: where the left and top is the predict diagram. The
right and top is the regression of the all data, and the top of this subfigure is the value of the correlation
coefficient. The left and bottom of the figure is the error diagram, and the top of this subfigure is the
values of the MSE and RMSE. The right and bottom is the error instances and the statistic distribution
(normal distribution) is marked on it and the values of the mean and standard deviation shown on top of
subfigure.
The results of the ANNs are analyzed according to the Section 3: MLP neural network was first intro-
duced and this ANN has parameters that should determine. The ANN is two layers with feed forward
 M. Kuchaki Rafsanjani and M. Samareh / Chaotic time series prediction by ANNs 607

Fig. 8. Long-term prediction of Mackey-glass time series with MLP.

architecture that used six neurons and Sigmoid transfer function (TF) in the hidden layer. Also, it used
the Levenberg-Marquardt learning algorithm to the learning process, and used linear TF in the output
layer (see Fig. 8).
In the Fig. 8, the prediction chart is shown where output and the target are drawing closer together; the
value of correlation coefficient is 0.99985 this means that the most of data is truly fitted. The error graph
is shown where the values of MSE and RMSE also are marked. Note that the value of MSE is very low,
but the value of RMSE is not very low that is about 0.003 namely the value of MSE may be deceptive.
In this subfigure error value is (between −0.01 and 0.01), and finally the error instances and its relation
to the normal distribution, the value of µ and σ is shown.
The second ANN explained in Section 3.2 was Elman NN, and the parameters of this ANN are as
follows: this is a two layer NN with a delay to the hidden layer, which this delay helps to the finding
the variable patterns over time; this delay is a weakness, because the learning process is slow. This uses
six neurons in the hidden layer and use Sigmoid and linear TF in the hidden and output layer and uses
gradient descend with momentum in the learning process. The results of this ANN are shown in the
Fig. 9.
The Elman has more error than the MLP, of course the most of errors related to the initial state of time
series that whatever goes ahead the error is reduced, so that the errors is (between −0.025 and 0.025)
and values of the MSE and RMSE are shown on top of error graph. The performance of this ANN is
relatively good. In addition to the performance of ELMAN is weaker than MLP and also its learning
process is difficult and time consuming. What can be inferred from Fig. 9 is the performance of the
Elman is worse the MLP; the value of RMSE is better criterion for comparison of the performance of
the ANNs. Also the value of correlation coefficient (CC) is shown and the values of µ and σ where worse
values of the MLP.
The third ANNs that was introduced in Section 3.2 is a layer recurrent neural network (LRNN) that
have two layers where all layers except the end layer have a loop feedback with a time delay unit, this
608 M. Kuchaki Rafsanjani and M. Samareh / Chaotic time series prediction by ANNs

Fig. 9. Long-term prediction of Mackey-glass time series with Elman.

Fig. 10. Long-term prediction of Mackey-glass time series with LRN.

change converts the neural network to slow neural network in the learning process. This ANN used six
neurons in the hidden layer and uses Sigmoid and linear TF and also used Levenberg-Marquardt to the
learning process (Fig. 10).
 M. Kuchaki Rafsanjani and M. Samareh / Chaotic time series prediction by ANNs 609

Fig. 11. Long-term prediction of Mackey-glass time series with FTDNN.

Fig. 12. Long-term prediction of Mackey-glass time series with TDNN.

In this section, the dynamic ANNs are examined that one of them was LRN where its performance is
good and even its error is less than the MLP and Elman, but its learning process is very difficult and need
the more time for completion this process. Its architecture is causing this problem; however, its output
is good. The values of the MSE and RMSE both are low, but better analysis of this ANN performance
610 M. Kuchaki Rafsanjani and M. Samareh / Chaotic time series prediction by ANNs

Fig. 13. Long-term prediction of Mackey-glass time series with NARX.

requires that consider MSE, RMSE values together. Also the CC value, µ and σ indicative the good
performance of this ANN, however, it should be noted that the training of this ANN is very difficult and
need very time to training.
The second dynamic ANN that introduced was focus time delay neural network or FTDNN this ANN
is a feed forward NN that uses a unit time delay in the first layer and other parameters of this ANN are
same the parameters of the previous ANN. The results of experiments of this ANN are shown in Fig. 11.
As the figures, this ANN has good output and also the process of learning both is simple and has
simple time complexity, of course its output rather than the LRN is slightly weaker; the values of MSE,
RMSE, CC, µ and σ also is determined.
The third recurrent network was the time delay neural network or TDNN where all of the settings are
same setting of the FTDNN with this difference that there exists unit delay in all of layer. So, this reason
called this neural network as distributed time delay neural network and also the learning process need
more time; the results are shown in Fig. 12.
Figure 12 shows that this ANN has an acceptable performance, however, the learning process is rela-
tively difficult. So, its error graph is not good. The prediction error at first is large and gradually this error
is less (between −0.05 and 0.05), and MSE, RMSE, CC, µ and σ values indicate relatively this ANN
has good performance. Forasmuch as the performance of this ANN is not very good and the procedure
of training is difficult, using this ANN not recommended.
The last ANN that in Section 3.3 was introduced was NARX that settings of this ANN also same the
settings other dynamic neural network and in here parallel architecture has been used; the results are
shown in Fig. 13. The figure shows that the NARX has good performance and it has less error, the values
of the MSE, RMSE, CC, µ, σ are determined, and the training procedure is too simple and not required
the much time for running. So far, this ANN has the best performance between the dynamic ANN and
even between the MLP and Elman.
 M. Kuchaki Rafsanjani and M. Samareh / Chaotic time series prediction by ANNs 611

Fig. 14. Long-term prediction of Mackey-glass time series with NAR.

Fig. 15. Long-term prediction of Mackey-glass time series with RBF.

As well as there is another neural network similar NARX called NAR that this ANN is similar to the
NARX, but by nuances, the results of this ANN are shown in Fig. 14. In the Fig. 14, the information
contains values of previous parameter, these results are not better NARX’s results, but this ANN is a
good alternative for NARX and other dynamic ANN.
RBFs was introduced as the last group of ANNs in Section 3.4, the ANN architecture is two layers that
612 M. Kuchaki Rafsanjani and M. Samareh / Chaotic time series prediction by ANNs

Fig. 16. Long-term prediction of Mackey-glass time series with RBFE.

Fig. 17. Long-term prediction of Mackey-Glass time series with GRNN.

uses 50 neurons in the hidden layer and the values of goal and spread is 0.3 this ANN and hidden layer
TF is a radial base function Eq. (8). The output layer TF is a linear function. May be used two functions
for experiment the RBFs where called radial base and radial base exact, these two neural networks have
nuances, in here show the results of two functions in Figs 15 and 16.
Comparing Figs 15 and 16, determine that error in RBFE is less than RBF, because the RBF neurons
 M. Kuchaki Rafsanjani and M. Samareh / Chaotic time series prediction by ANNs 613

Table 1
Summary of information obtained from all ANNs
MSE RMSE R M Σ
MLP 1.5963e-005 0.0039954 0.99985 −8.3225e-007 0.0039971
Elman 0.00027216 0.016497 0.9974 1.32e-005 0.016504
LRN 2.9439e-007 0.00054258 1 −5.4943e-008 0.00054281
FTDNN 1.607e-006 0.0012677 0.99998 1.7414e-006 0.0012682
TDNN 0.00067167 0.025917 0.9937 0.0007885 0.025916
NARX 1.7982e-007 0.00042405 1 2.1249e-006 0.00042423
NAR 1.1825e-006 0.0010874 0.99999 3.6787e-006 0.0010879
RBF 3.35e-005 0.0057879 0.99968 −7.9239e-005 0.0057898
RBFE 2.3028e-005 0.0047988 0.99978 4.5069e-006 0.0048008
GRNN 9.3879e-005 0.0096891 0.9991 0.00035788 0.0096866

are less than RBFE. As well as, the values of MSE, RMSE and other parameters are close to the perfor-
mance of MLP. And there exists a minimal difference between RBF and RBFE where this difference is
shown in the Figs 15 and 16.
The last ANN that be checked is GRNN that was introduced in Section 3.4, this ANN setting is same
the settings of the RBF by the difference that the spread this ANN is 0.04, the results of the experiment
are shown in Fig. 17. The performance of this ANN rather than to other RBF (RBF and RBFE) is
relatively good despite being slightly weaker, but the learning process is simple, as well as the values of
MSE, RMSE and other parameters specified on the Fig. 17; the tolerance of error is (between −0.03 and
0.03).
Summary of all of the ANNs results is listed in Table 1. These results were previously reported in
figures; if MSE and RMSE values are less, then the ANN performance is better. The R value represents
the correlation between outputs and targets and is closer to 1 represents nearly outputs and targets.
Finally µ and σ are mean and deviation of error instances, these measures are evaluated with a standard
normal distribution.

5. Conclusion and future work

In this paper, several categories of ANNs for prediction MG behavior are introduced, as all results
were analyzed. Of each category a method based on the results of experiments is selected. From dynamic
ANN it appears that the NARX is the better from others. We got four models; MLP, Elman, NARX and
RBFE according to the Section 4, it appears that the NARX ANN is the better from others. All of ANNs
have advantages and disadvantages that mention to the tradeoff between accuracy and time complexity,
that NARX is a good proposal for this especial problem. In addition to all the benefits of ANN, the
main problem is configuration and tuning them for every problem. For future work, we can compare the
results of ANNs to other predictor and computational intelligences for example support vector machine
(SVM) or combination the ANN by own or other methods.

References

[1] J. Zhang and K.F. Man, Time series prediction using recurrent neural network in multi-dimension embedding phase
space, IEEE International Conference on Systems, Man, and Cybernetics 2 (1998), 11–14.
[2] H.D. Abarbanel, T.W. Frison and L. Tsimring, Obtaining order in a world of chaos, IEEE Signal Processing Magazine
15(3) (1998), 49–65.
614 M. Kuchaki Rafsanjani and M. Samareh / Chaotic time series prediction by ANNs

[3] E. Ott, Chaos in dynamical systems, 2nd edition, Cambridge, U.K. Cambridge University Press, 2002.
[4] C.H. Skiadas, I. Dimotikalis and C. Skiadas, Chaos theory modeling, simulation and applications, World Scientific
Publishing, 2011.
[5] D.S. Poskitt and A.R. Tremayne, The selection and use of linear and bilinear time series models, International Journal
of Forecasting 2(1) (1986) 101–114.
[6] H. Tong, threshold models in nonlinear time series analysis, Springer Verlag, 1983.
[7] H. Tong, Non – linear time series: A dynamical system approach, Oxford University Press, 1990.
[8] H. Tong and K.S. Lim, Threshold autoregression limit cycles and cyclical data, Journal of the Royal Statistical Society
42(3) (1980), 245–292.
[9] J.G. De Gooijer and R.J. Hyndman, 25 years of time series forecasting, International Journal of Forecasting 22(3)
(2006), 443–473.
[10] J.G. De Gooijer and K. Kumar, Some recent developments in non-linear time series modelling, testing, and forecasting,
International Journal of Forecasting 8(2) (1992), 135–156.
[11] P.J. Werbos, Generalization of backpropagation with application to a recurrent gas market model, Neural Networks 1(4)
(1988), 339–356.
[12] E. Alpaydin, Introduction to Machine Learning, The MIT Press, 2010.
[13] L.J. Herrera, H. Pomares, I. Rojas, A. Guillen, A. Prieto and O. Valenzuela, Recursive prediction for long term time
series forecasting using advanced models, Neurocomputing 70 (2007), 2870–2880.
[14] J.P. Menezes and G.A. Barreto, Long – term time series prediction with the NARX network: An empirical evaluation,
Neurocomputing 71 (2008), 3335–3343.
[15] M. Ardalani-Farsa and S. Zolfaghari, Chaotic time series prediction with residual analysis method using hybrid Elman –
NARX neural networks, Neurocomputing 73 (2010), 2540–2553.
[16] H. Abarbanel, Analysis of observed chaotic data, Spinger-Verlag, New York, 1996.
[17] Y. Fu, C. Wu, J. Jeng and C. Ko, ARFNNs with SVR for prediction of chaotic time series with outliers, Expert Systems
with Applications 37 (2010), 4441–4451.
[18] M. Norgaard, O. Ravn, N.K. Poulsen and L.K. Hansen, Neural Networks for Modelling and Control of Dynamic Systems,
Springer, Berlin, (2000).
[19] R. Lawrence, Using neural networks to forecast stock market prices, University of Manitoba, 1997.
[20] F. Liu, C. Quek and G.S. Ng, Neural network model for time series prediction by reinforcement learning, Proceedings
of the International Joint Conference on Neural Networks, Montreal, Canada, (2005).
[21] Y.R. Park, T.J. Murray and C. Chen, Predicting sun spots using a layered perceptron neural network, IEEE Transactions
on Neural Networks 1(2) (1996), 501–505.
[22] P. Tenti, Forecasting foreign exchange rates using recurrent neural networks, Applied Artificial Intelligence 10 (1996),
567–581.
[23] S. Haykin, Neural networks: A comprehensive foundation, Macmillan College Publishing Company, New York, 1994.
[24] M.T. Hagan, H.B. Demuth and M.H. Beale, Neural Network Design, 1996.
[25] J.F. Kolen and S.C. Kremer, A Field guide to dynamical recurrent networks, Wiley – IEEE Press, 2001.
[26] L.P. Maguire, B. Roche, T.M. McGinnity and L.J. Mc Daid, Predicting a chaotic time series using a fuzzy neural network,
Information Sciences 112 (1998), 125–136.
[27] J. Zhang and K.F. Man, Time series prediction using recurrent neural network inmulti – dimension embedding phase
space, IEEE International Conference on Systems, Man, and Cybernetics 2 (1998), 11–14.
[28] R. Zemouri, R. Gouriveau and N. Zerhouni, Defining and applying prediction performance metrics on a recurrent NARX
time series model, Neurocomputing 73 (2010), 2506–2521.
[29] A. Aussem, F. Murtagh and M. Sarazin, Dynamical recurrent neural networks towards environmental time series predic-
tion, International Journal Neural Systems 6(2) (1995), 145–170.
[30] R. Chandra and M. Zhang, Cooperative coevolution of elman recurrent neural networks for chaotic time series prediction,
Neurocomputing 86 (2012), 116–123.
[31] A. Aussem, F. Murtagh and M. Sarazin, Dynamical recurrent neural networks and pattern recognition methods for time
series prediction: Application to seeing and temperature forecasting in the context of ESO’s VLT Astronomical Weather
Station, Vistas iv Astronomy 38 (1995), 357–374.
[32] J. Zhang and K.F. Man, Time series prediction using recurrent neural network in multi–dimension embedding phase
space, IEEE International Conference on Systems, Man, and Cybernetics 2 (1998) 11–14.
[33] T. Koskela, M. Lehtokangas, J. Saarinen and K. Kaski, Time series prediction with multilayer perceptron, FIR and elman
neural networks, in: The Proceedings of the World Congress on Neural Networks (1996), 491–496.
[34] D.E. Rumelhart and J.L. McClelland, Parallel distributed processing; exploration in the microstructure of cognation,
MIT Press Cambridge, 1986.
[35] A.B. Geva, Scale net – multiscale neural – network architecture for time series prediction, IEEE Transaction on Neural
Network 9(5) (1998).
 M. Kuchaki Rafsanjani and M. Samareh / Chaotic time series prediction by ANNs 615

[36] A. Wabel, T. Hanzawa, G. Hinton, K. Shikano and K.J. Lang, Phoneme recognition using time delay neural network,
IEEE Transaction on Acoustics, Speech, Signal Processing 37(3) (1989).
[37] J.C. Principe, N.R. Euliano and W.C. Lefebvre, Neural adaptive systems: Fundamentals through simulations, Wiley, New
York, 2000.
[38] B.A. Pearlmutter, Gradient calculations for dynamic recurrent neural networks: A survey, IEEE Tansactions on Neural
Networks 6(5) (1995), 1212–1228.
[39] H.T. Siegelmann, B.G. Horne and C.L. Giles, Computational capabilities of recurrent NARX neural networks, IEEE
Transaction Systems 27(2) (1997), 208–215.
[40] K.S. Narendra and K. Parthasarathy, Identification and control of dynamical systems using neural networks, IEEE Trans-
action on Neural Networks 1(1) (1990), 4–27.
[41] S. Chen, S.A. Billings and P.M. Grant, Nonlinear system identification using neural networks, International Journal
Control 11(6) (1990), 1191–1214.
[42] T. Lin, B.G. Horne, P. Tino and C.L. Giles, A delay damage model selection algorithm for NARX neural networks, IEEE
Transaction Signal Processing 45(11) (1997), 2719–2730.
[43] B.A. Whitehead and T.D. Choate, Cooperative – competitive genetic evolution of radial basis function centers and widths
for time series prediction, IEEE Transactions on Neural Networks 7(4) (1996), 869–880.
[44] V.M. Rivas, J.J. Merelo, P.A. Castillo, M.G. Arenasb and J.G. Castellano, A evolving RBF neural networks for time –
series forecasting with EvRBF, Information Sciences 165 (2004), 207–220.
[45] F. Schwenker, H.A. Kestler and G. Palmm, Three learning phases for radial – basis – function networks, Neural Networks
14 (2001), 439–458.
[46] D.S. Broomhead and D. Lowe, Multi variable functional interpolation and adaptative networks, Complex Systems 11
(1988), 321–355.
[47] M. Awad, H. Pomares, I. Rojas and O.S. Hamdon, Prediction of time series using rbf neural networks: A new approach
of clustering, The International Arab Journal of Information Technology 6(2) (2009).
[48] Z. Min and L. Shiling, Application of GRNN and uniform design to nonlinear system modeling, Computer Measurement
and Control 15(9) (2007), 1189–1191.
[49] D.A. Specht, General regression neural network, IEEE Transactions on Neural Network 2 (1991), 568–576.
[50] Z. Libiao, Z.C. Guang, M. Ming and L. Xiaohua, Solutions of multi-objective optimization problems based on particle
swarm optimization, Journal of Computer Research and Development 41(7) (2004), 1286–1291.
[51] J. Zhang, H.S. Chung and W. Lo, Chaotic time series prediction using a neuro – fuzzy system with time – delay coordi-
nates, IEEE Transaction on Knowledge and Data Engineering 20(7) (2008).
[52] H. Yuxia and Z. Hongtao, Chaos optimization method of SVM parameters selection for chaotic time series forecasting,
Proceedings of the International Conference on Solid State Devices and Materials Science, Physics Procedia 25 (2012),
588–594.
[53] B. Samanta, Prediction of chaotic time series using computational intelligence, Expert Systems with Applications 38
(2011), 11406–11411.
[54] J. Wang, D. Chi, J. Wu and H. Lu, Chaotic time series method combined with particle swarm ptimization and trend
adjustment for electricity demand forecasting, Expert Systems with Applications 38 (2011), 8419–8429.
[55] D. Karunasinghe and S. Liong, Chaotic time series prediction with a global model: Artificial neural network, Journal of
Hydrology 323 (2006), 92–105.
[56] A.E. Omidvar, Configuring radial basis function network using fractal scaling process with application to chaotic time
series prediction, Chaos, Solitons and Fractals 22 (2004), 757–766.
[57] H.J. Song, C.Y. Miao, Z.Q. Shen, W. Roel, D.H. Maja and C. Francky, Design of fuzzy cognitive maps using neural
networks for predicting chaotic time series, Neural Networks 23 (2010), 1264–1275.
[58] A. Aussem, Dynamical recurrent neural networks towards prediction and modeling of dynamical systems, Neurocom-
puting 28 (1999), 207–232.
[59] J. McNames, Local averaging optimization for chaotic time series prediction, Neurocomputing 48 (2002), 279–297.
[60] K.W. Lau and Q.H. Wu, Local prediction of non – linear time series using support vector regression, Pattern Recognition
41 (2008), 1539–1547.
[61] H. Mirzaee, Long – term prediction of chaotic time series with multi – step prediction horizons by a neural network with
levenberg – marquardt learning algorithm, Chaos, Solitons and Fractals 41 (2009), 1975–1979.
[62] S. Li-yun, Prediction of multivariate chaotic time series with local polynomial fitting, Computers and Mathematics with
Applications 59 (2010), 737–744.
[63] V.A. Gromov and A.N. Shulga, Chaotic time series prediction with employment of ant colony optimization, Expert
Systems with Applications 39 (2012), 8474–8478.
[64] X. Wu and Z. Song, Multi – step prediction of chaotic time – series with intermittent failures based on the generalized
nonlinear filtering methods, Applied Mathematics and Computation 219 (2013), 8584–8594.