ARTICLE IN PRESS

Mechanical Systems and Signal Processing 23 (2009) 1434–1446

Contents lists available at ScienceDirect

Mechanical Systems and Signal Processing

journal homepage: www.elsevier.com/locate/jnlabr/ymssp

Nonlinear model identification of an experimental ball-and-tube

system using a genetic programming approach
Leandro dos Santos Coelho , Marcelo Wicthoff Pessôa
Industrial and Systems Engineering Graduate Program, PPGEPS, Pontifical Catholic University of Paraná, Imaculada Conceic- ão, 1155, Zip code 80215-901,
Curitiba, Paraná, Brazil

a r t i c l e i n f o abstract

Article history: Most processes in industry are characterized by nonlinear and time-varying behavior.
Received 11 December 2007 The identification of mathematical models typically nonlinear systems is vital in many
Received in revised form fields of engineering. The developed mathematical models can be used to study the
1 February 2009
behavior of the underlying system as well as for supervision, fault detection, prediction,
Accepted 3 February 2009
Available online 20 February 2009
estimation of unmeasurable variables, optimization and model-based control purposes.
A variety of system identification techniques are applied to the modeling of process
Keywords: dynamics. Recently, the identification of nonlinear systems by genetic programming
System identification (GP) approaches has been successfully applied in many applications. GP is a paradigm of
Nonlinear models
evolutionary computation field based on a structure description method that applies the
Evolutionary algorithm
principles of natural evolution to optimization problems and its nature is a generalized
Genetic programming
hierarchy computer program description. GP adopts a tree structure code to describe an
identification problem. Unlike the traditional approximation methods where the
structure of an approximate model is fixed, the structure of the GP tree itself is
modified and optimized and, thus, there is a possibility that GP trees could be more
appropriate or accurate approximate models. This paper focuses the GP method for
structure selection in a system identification applications. The proposed GP method
combines different techniques for tuning of crossover and mutation probabilities
with an orthogonal least-squares (OLS) algorithm to estimate the contribution of
the branches of the tree to the accuracy of the discrete polynomial Nonlinear
AutoRegressive with eXogenous inputs (NARX) model. The nonlinear system identifica-
tion procedure, based on a NARX representation and GP, is applied to empirical case
study of an experimental ball-and-tube system. The results demonstrate that the GP
with OLS is a promising technique for NARX modeling.
& 2009 Elsevier Ltd. All rights reserved.

1. Introduction

Developing models from observed data, or function learning, is a fundamental problem in engineering systems. Many
engineering approaches such as model-based control systems require accurate process models. However, the majority of
physical systems contain complex nonlinear relations, especially in case of hard nonlinearities and chaotic behavior, which
are difficult to model with conventional techniques.

 Corresponding author. Tel./fax: +55 41 327113 45.

E-mail addresses: [email protected] (L.S. Coelho), [email protected] (M.W. Pessôa).

0888-3270/$ - see front matter & 2009 Elsevier Ltd. All rights reserved.
doi:10.1016/j.ymssp.2009.02.005
 ARTICLE IN PRESS
L.S. Coelho, M.W. Pessôa / Mechanical Systems and Signal Processing 23 (2009) 1434–1446 1435

System identification is the procedure of constructing a mathematical model from input–output data for a dynamic
system under testing and characterizing the system behaviors. The identification of dynamic nonlinear systems, which
pose problems and require solutions distinct from their linear counterparts, is a hard task as demonstrated by the effort
devoted by researchers in the last decades. Several techniques have been proposed for identification of nonlinear systems
[1–4]. The most widespread identification techniques proposed in literature including bilinear models [4], Wiener–Ham-
merstein block-oriented models [5], Volterra series [6], Nonlinear AutoRegressive Moving Average eXogenous (NARMAX)
models [7], artificial neural networks [8], fuzzy models [9], neuro-fuzzy models [10], principal component analysis [11],
evolutionary algorithms [12,13], and others [1–4].
In recent years, genetic programming (GP) [14–22], a member of the evolutionary computation field, has been applied to
fault detection, modeling and identification of nonlinear systems. GP is a stochastic process for automatically generating
computer programs and was introduced by John Koza [23], based on the idea of genetic algorithms. An advantage of GP is
that it can evolve a solution automatically from the training data, and does not require an assumption regarding the
mathematical model of the structure or the size of the decision tree-based solution.

1.1. Goals

This paper investigates the design of a GP based on different tuning approaches of crossover and mutation probabilities
and the orthogonal least-squares (OLS) algorithm [24] to estimate the contribution of the branches of the tree to the
accuracy of the discrete polynomial Nonlinear AutoRegressive with eXogenous inputs (NARX) model.
The effect of different tuning approaches of crossover and mutation probabilities is compared and that provides
interesting insight into GP design combined with OLS algorithm to identify NARX models. To illustrate the power of the
proposed GP methodology in NARX identification, an experimental ball-and-tube system [25] has been considered.
The procedure identification using GP based on OLS for NARX nonlinear identification validated in this paper was
inspired in [26]. Therefore, there are differences between the results presented in this paper and in paper [26]. The paper
[26] proposes a GP based on OLS for NARX nonlinear identification. On the other hand, the contributions of this paper are:
(i) evaluation of seven GP approaches to tune the mutation and crossover probabilities, (ii) performance index for
identification results based on multiple correlation index, and (iii) evaluation of practical case of an experimental ball-and-
tube system.

1.2. Organization

The remainder of this paper is organized as follows. In Section 2, the fundamentals of system identification are
presented. The theoretical background of GP method is introduced in Section 3. Description of ball-and-tube system and
the identification results are both commented in Section 4. Finally, the conclusion and further research are discussed in
Section 5.

2. System identification

System identification is concerned with choosing mathematical models to characterize the input–output behaviors of
an unknown system using observed data of process. In terms of system identification, linear system identification is a field
that has evolved considerably during the last decades [27,28]. The linear mathematical model is useful if the underlying
physical process exhibits qualitatively similar dynamic behavior to the linear model in the operating point of interest.
However, it is often difficult to represent the behavior of the system over its full range of operation using linear
mathematical models.
For these reasons, there is much current research interest in models for nonlinear identification. Nonlinear identification
techniques may provide a solution to this problem. Meanwhile, it is important to note that several practical examples of
nonlinear dynamic behavior have been reported in the recent engineering literature [1–13,29–32].
This paper investigates a GP method combined with OLS algorithm to NARX modeling. In this context, the procedure for
identification adopted is summarized by the following steps:

(i) design an experiment to obtain the process input/output data sets pertinent to the model application;
(ii) examine the quality of measured data, removing trends and outliers;
(iii) construct a set of candidate models based on information from the experimental data sets (or simulation data sets).
This step is the model structure identification;
(iv) select a particular model from the set of candidate models and estimate the model parameter values using the
experimental data sets (or simulation data sets).
(v) evaluate how good the model is using a performance criterion;
(vi) if a satisfactory model is still not obtained in step (v) then repeat the procedure either for step (i) or step (iii),
depending on the problem.
 ARTICLE IN PRESS
1436 L.S. Coelho, M.W. Pessôa / Mechanical Systems and Signal Processing 23 (2009) 1434–1446

3. Genetic programming method

GP is an evolutionary optimization method that produces functional programs to solve a given task. In the GP paradigm,
problems in systems identification are viewed as the discovery of computer programs through a search process (global
optimization) based on the rules of natural selection and genetics. Due to its population-based nature, GP approaches can
avoid being trapped in a local optimum, and consequently have the ability to find global suitable solutions.
In GP, solutions to a problem can be represented in different forms, but are usually interpreted as computer programs.
The computer programs represent candidate solutions to a problem. The typical structure of each individual can be seen as
a tree-shaped structure to represent the individuals in the evolving population. Each candidate solution or individual of
which has a fitness.
The mentioned trees are typically encoded as S-expressions, the syntactic form in Lisp (LISt Processing language)
programming language and the basic units of trees are called nodes. The GP procedure employs usually a context-free
grammar declared in Backus–Naur-Form (BNF). The BNF-grammar consists of nonterminal nodes and terminal nodes and is
represented by the set {N, T, P, S}, where N is the set of nonterminals (function set), T is the set of terminals, P is the set of
production rules and S is a member of N corresponding to the starting symbol [33].
The leaf nodes are input variables from the terminal set T, and internal nodes are operators from the function set F. Each
candidate solution is built from combining of all possible functions and terminals. The function set is the operators and
functions such as arithmetic operators of addition, subtraction, multiplication, and division as well as a conditional
branching operator. The terminal set is provided, consisting of zero argument instructions, typically representing external
inputs from the environment or constants, such as (3.33+u(k6))*(y(k1)), where u and y are, respectively, the input and
the output signals of process, and k is the time. The example of a tree in GP for (3.33+u(k6))*(y(k1)) is shown in Fig. 1.
The evolution procedure in GP proceeds in similar way to the standard genetic algorithm, i.e., GP starts with the random
generation of an initial population of candidate solutions (trees in GP case). The following sequence of operations in GP
include: (i) evaluate the fitness of each of the candidate solutions to find out how well this candidate solution can reach the
objective, (ii) apply the genetic operators, and (iii) verify if the adopted stopping criterion was reached.
Determining the stopping (termination) criterion, for example, is usually a simple process, e.g. stop after a certain
number of generations have elapsed or stop when a solution of a high enough quality is found. In this work is adopted a
predefined number of generations.
A flowchart of GP is presented in Fig. 2. The genetic operators in GP are described in the next section.

3.1. Operators in GP

The evolution of GP is performed using a set of stochastic evolutionary operators, which manipulate the candidate
solutions in the population. The main operators for GP are reproduction, crossover and mutation.
Reproduction operator: The reproduction operation is performed by copying candidate solutions to the next population
without any change in terms of a certain probability Pr. Tournament selection is adopted here to select two candidate
solutions (called parents). According to this strategy, two candidate solutions among the current population are randomly
selected, and the one with higher fitness win the right to mate. The process is repeated for the other parent.
Crossover operator: The crossover is employed to exploit the current candidate solutions. On the basis of the given fitness
values, to perform the standard crossover operator in the GP, two candidate solutions are selected from the population and
are put into the mating pool. A crossover point is then randomly chosen in each of the two parent programs (trees), then
the subtrees below the crossover points are swapped, and two new (offspring) programs are created. The two offspring
programs are then put into the population in the next generation. Fig. 3 shows an example of crossover operation. The
crossover probability is Pc.
Mutation operator: The mutation is employed to explore in the search space to produce new candidate solutions. In other
words, the purpose of the mutation operator in GP is to improve population diversity by generating entirely new candidate
solutions segments, in order to explore new regions of the search space. A number of duplicates of each offspring are
created, and their terminal nodes mutate by replacing the original values with the neighbor values randomly selected from

Fig. 1. Example of a tree in GP representing a nonlinear model structure.

 ARTICLE IN PRESS
L.S. Coelho, M.W. Pessôa / Mechanical Systems and Signal Processing 23 (2009) 1434–1446 1437

Fig. 2. Flowchart of a GP approach.

Fig. 3. An example of crossover operation.

the component value set (see Fig. 4). Like the mutation operator in genetic algorithms, it is a form of asexual reproduction
based on only one parent. The mutation probability is Pm.

3.2. Control parameters

GP may have parameters, for instance, the reproduction, crossover, mutation probabilities, maximum size for trees, the
tournament size of selection, and the population size. The values of these parameters determine the quality of the solution
obtained and the efficiency of the search [34].
The performance of GP is sensitive to the choice of control parameters. Choosing suitable parameter values is,
frequently, a problem-dependent task and requires previous experience of the user. Proper control parameters are
recommended to some certain values to provide the algorithm better performance from the two aspects of effectiveness
 ARTICLE IN PRESS
1438 L.S. Coelho, M.W. Pessôa / Mechanical Systems and Signal Processing 23 (2009) 1434–1446

Fig. 4. An example of mutation operation.

and efficiency according to the computational experiments. Despite its crucial importance, there is no consistent
methodology for determining the control parameters of EA, which are, most of the time, arbitrarily set within some
predefined ranges [4].
Two major forms of setting parameter values must be mentioned: parameter tuning and parameter control [34]. The
former means the commonly practiced approach that tries to find good values for the parameters before running the
algorithm, then tuning the algorithm using these values, which remain fixed during the run. The latter means that values
for the parameters are changed during the run.
In this paper, the setting of control parameters of crossover and mutation probabilities in GP is provided by different
approaches. These approaches are described as follows:
Approach 1—GP1: This is the classical GP with fixed crossover and mutation rates. The Pc and Pm values are set to 0.90
and 0.40, respectively.
Approach 2—GP2: The parameters Pc and Pm are modified by the following equations in each generation based on
uniform distribution:
Pc ¼ 0:40Uð0; 1Þ þ 0:50 (1)

Pm ¼ 0:35Uð0; 1Þ þ 0:05 (2)

where U(0,1) is a uniformly distributed random value between 0 and 1.
Approach 3—GP3: The parameter Pc is decremented and the Pm is increment with evolution of generations. The value of
Pc and Pm are modified using the following equations:
g
PcðgÞ ¼ ðC f  C i Þ þ Ci (3)
g max

g
PmðgÞ ¼ ðM f  Mi Þ þ Mi (4)
g max
where g is the number of generation, Ci, Cf, Mi and Mf are constants (adopted Ci ¼ 0.90, Cf ¼ 0.50, Mi ¼ 0.05, and Mf ¼ 0.35),
and gmax is the maximum generations of evolutionary procedure in GP (stopping criterion).
Approach 4—GP4: The values of Pc and Pm are modified by same equations as those of GP3. However, here, it is adopted
Ci ¼ 0.50, Cf ¼ 0.90, Mi ¼ 0.35, and Mf ¼ 0.05.
Approach 5—GP5: The application of chaotic sequences instead of random sequences in GP is an alternative strategy to
diversify the population and improve the GP’s performance and other evolutionary algorithms in preventing premature
convergence to local minima. One of the simplest dynamic systems evidencing chaotic behavior is the iterator called the
logistic map [35], whose equation is given by
zðtÞ ¼ mzðt  1Þ½1  zðt  1Þ (5)
where t is the sample, and m is a control parameter, 0pmp4. The behavior of the system of Eq. (5) is greatly changed with
the variation of m. The value of m determines whether z stabilizes at a constant size, oscillates between a limited sequence
of sizes, or behaves chaotically in an unpredictable pattern. A very small difference in the initial value of z causes
substantial differences in its long-time behavior. Eq. (5) is deterministic, displaying chaotic dynamics when m ¼ 4 and
z(1)e{0, 0.25, 0.50, 0.75, 1} [36]. In this case, x(t) is distributed in the range (0,1) provided the initial z(1)A(0,1) and
z(1) ¼ 0.48, as was adopted here. In this case, the values of Pc and Pm in GP method are modified using Eq. (5).
Approach 6—GP6: The update of parameters Pc and Pm during the evolutionary cycle is based in the following
equations:
(
1:01Pcðg  1Þ if bf ðgÞ4bf ðg  1Þ
PcðgÞ ¼ (6)
0:99Pcðg  1Þ if bf ðgÞpbf ðg  1Þ
 ARTICLE IN PRESS
L.S. Coelho, M.W. Pessôa / Mechanical Systems and Signal Processing 23 (2009) 1434–1446 1439

(
1:01Pmðg  1Þ if bf ðgÞpbf ðg  1Þ
PmðgÞ ¼ (7)
0:99Pmðg  1Þ if bf ðgÞ4bf ðg  1Þ

where 0oPc(g), Pm(g)o1 and bf is the best fitness (maximization problem). The values of Pc(0) and Pm(0) are 0.90 and 0.40.
Approach 7—GP7: The update of parameters Pc and Pm during the evolutionary cycle is based in the following equations:
(
minð1; 1:01Pcðg  1ÞÞ if bf ðgÞ4bf ðg  1Þ
PcðgÞ ¼ (8)
maxð0:1; 0:99Pcðg  1ÞÞ if bf ðgÞpbf ðg  1Þ

(
minð0:5; 1:01Pmðg  1ÞÞ if DdðgÞo0
PmðgÞ ¼ (9)
maxð0:05; 0:99Pmðg  1ÞÞ if DdðgÞp0

where bf is the best fitness (maximization problem). The values of Pc(0) and Pm(0) are 0.90 and 0.40. The Dd is given by
   
 f̄ ðgÞ   f̄ ðg  1Þ 
   
DdðgÞ ¼   , (10)
bf ðgÞ bf ðg  1Þ

where f̄ is the mean of fitness in population.

3.3. NARX Identification using GP

The flexibility of computer program representation, combined with the powerful capabilities of evolutionary search,
make GP a powerful methodology to solve a great variety of problems.
Due to the ability of GP in searching for structures, various techniques have been exploited by a number of researchers
in order to increase the performance of GP as a modeling tool technique for system identification purposes [14–22].
The GP can optimize both the model structure as well as its parameters in applications of system identification mainly
nonlinear identification. Among this class of models, the identification of discrete-time, nonlinear ARX (NARX) is
considered in this paper. In the NARX model, the model regressors are input u(k) and output y(k) observations for the
discrete-time k. The NARX model is given by
yðkÞ ¼ f ðyðk  1Þ; . . . ; yðk  ny Þ; uðk  t d  1Þuðk  t d  2Þ; . . . ; uðk  t d  nu ÞÞ þ eðkÞ. (11)
where y(k) is system output for the time k, f is an unknown static nonlinear mapping, td is the delay time (dead-time), nu
and ny are the maximum and output lags (often referred to as model orders), respectively, while e(k) represents the
modeling error and is assumed to be Gaussian noise. The above single-input–single-output (SISO) system representation
can be assumed without a loss of generality since the extension to multiple-input–single-output (MISO) and multiple-
input–multiple-output (MIMO) systems is straightforward.
Letting y^ ðkÞ ¼ f ðyðk  1Þ; . . . ; yðk  ny Þ; uðk  dÞ; uðk  d  1Þ; . . . ; uðk  d  nu ÞÞ, where y^ ðkÞ is predicted output for the time
k, the NARX identification problem amounts to reconstructing the nonlinear mapping f(  ): <b ! <, b ¼ nu+ny form the set
(y(k), u(k)), k ¼ 1, 2,y,n [37]. Since the mentioned set provides a nonuniform and incomplete sampling of the domain of f, a
generalization problem arises than can be solved by resorting to GP.
Linear-in-parameters mathematical models can be formulated as
X
M
yðkÞ ¼ pi F i ðxðkÞÞ (12)
i¼1

where F1,y,FM are nonlinear functions (they do not contain parameters), x(k) is the regressor vector that consists of lagged
input(s) and output(s) given by x(k) ¼ [y(k1),y,y(kny),u(kd1),y,u(kdnu)], and p1,y,pM are model parameters
[26]. The problem of model structure selection for linear-in-parameters models is to find the proper set of nonlinear
functions.
There are a vast number of possible structures; hence, in practice, it is impossible to evaluate all of them. Even if the set
of possible structures is restricted only to polynomial models given by [26]
X
m X
m X
m X
m X Y
m
yðkÞ ¼ p0 þ pi1 xi1 ðkÞ þ pi1 i2 xi1 ðkÞxi2 ðkÞ þ    þ  pi1 ;:::;id xij ðkÞ (13)
i1 ¼1 i1 ¼1 i2 ¼i1 i1 ¼1 id ¼id1 j¼1

where p0 is a constant term. The construction of a NARX model consists of the selection of many structural parameters,
which have significant effect to the performance of the designed model.
The number of parameters (number of polynomial terms) is given by [26]
ðd þ mÞ!
np ¼ (14)
d!m!
Analytical resolution of parameter estimation problem of NARX is a complex task. The GP method combined with the
OLS for the structure selection of NARX models that are linear-in-parameters is addressed in this context. Generally, GP
 ARTICLE IN PRESS
1440 L.S. Coelho, M.W. Pessôa / Mechanical Systems and Signal Processing 23 (2009) 1434–1446

creates nonlinear models in addition to linear-in-parameters models. To avoid nonlinear-in parameters models, the
parameters must be removed from the set of terminals; i.e., it contains only variables: T ¼ {x1(k),y,xm(k)}, where xi(k)
denotes the ith regressor variable. Hence, a population member represents only the Fi nonlinear functions (see Eq. (12))
[26,38].
During the operation of GP, the algorithm generates a lot of potential solutions in the form of a tree structure. These
trees may have terms (subtrees) that contribute more or less to the accuracy of the model. In this context, the parameters
are assigned to the model after ‘‘extraction’’ of the Fi function terms from the tree, and they are determined using the OLS
[24] algorithm.
The OLS introduces the error reduction ratio (ERR), which is a measure of the decrease in the variance of the output by a given
term. The adopted procedure is the following [26]: first, the trees (the individual members of the population) are decomposed to
subtrees (function terms of the linear-in-parameters models); then, the ERRs of these function terms are calculated; finally, the
less significant term(s) is/are eliminated. This ‘‘tree pruning’’ method is realized in every fitness evaluation before the calculation of
the fitness values of the trees. The main goal of the application of this approach proposed in [26] is to transform the trees to
simpler trees that are more transparent, but whose accuracies are close to those of the original trees. This method always
guarantees that the elimination of one or more function terms of the model can be done by ‘‘pruning’’ of the corresponding
subtrees, so there is no need for structural rearrangement of the tree after this operation.

4. Experiments and results

In this section, it turns to the description of experimental process and analysis of the results obtained by the GP
approaches.

4.1. Process description

The process used in this work is a ball-and-tube system, a low-cost prototype. The ball-and-tube system consists in a
plastic tube, a direct current (DC) motor and a ball of table tennis. The aim of this project is to control the height of the ball
by applying a flux of air through the tube base, considering that this air flux is generated by the application of voltage in the
DC motor as shown in (Fig. 5).
The project of ball-and-tube system contains three main modules: (i) ball-and-tube structure, (ii) hardware, and (iii)
software. The hardware module of ball-and-tube prototype involves the distance sensor, DC motor, interface circuit, and
firmware. The interface circuit is composed by the serial communication circuit, the distance sensor multiplexer circuit,
and the pulse width modulator (PWM) circuit, as shown in Fig. 6.
A personal computer with an acquisition data board to transmit the control signal and to register the value of the height
of the sphere in the tube (by the use of a group of sensors based on phototransistors) is utilized. Through the acquisition
data board, a voltage is converted into an output voltage to be acquired by the acquisition data board. In Fig. 6 it is
presented the ball-and-tube prototype with 55 cm of height and 4.5 cm of diameter. The chosen DC motor (fan generation)
allows change of the input voltage from 0 to 20 V, those converted to values between 0 and 5 V TTL (Transistor-Transistor
Logic) level to facilitate the construction of the power electronics circuit.
A computer with a data acquisition board for generating the input signal was used to obtain system measurements. In
the identification procedure based on the NARX model, 200 samples of input (voltage applied to the DC motor) and output
(ball height) were collected with a time sampling of 200 ms.
The ball-and-tube system is open-loop stable. In this case, the data were collected while a proportional controller was
active (closed-loop identification). According to Landau [39], identification in closed-loop appears, in practice, to be one of
the most convenient methods to provide good models for controller design; therefore, one is not surprised that this
problem has generated significant interest in the identification literature [40–48]. Closed-loop identification is motivated
by the need to identify a process operating under output feedback. This is generally required if the process to be identified
is unstable or poorly damped. The main advantage of closed-loop identification is that the process is under automatic
control at the desired operating point. Hence, closed-loop identification is often preferred. Furthermore, closed-loop
identification offers a number of practical advantages over open-loop identification by allowing for easier validation of the
designed controller and on-site retuning, obtaining of models for controller design, controller maintenance, controller
redesign, and controller order reduction.
The assumption in this paper is that data are collected while the proportional control system is active, i.e., in closed-loop
experiments based on approach proposed in [49]. The operation range used in experiments was between 9 cm (0.094 V)
and 53 cm (4.707 V) of full height of tube (see details of data in Fig. 7).

4.2. Parameter settings

Each GP approach was implemented in MATLAB (MathWorks). All the programs were run under a Pentium dual-core
processor with 1.73 GHz and 2 GB of random access memory (RAM). In order to eliminate stochastic discrepancy, in each
case study, 50 independent runs were made involving 50 different initial trial solutions for each tested GP approach.
 ARTICLE IN PRESS
L.S. Coelho, M.W. Pessôa / Mechanical Systems and Signal Processing 23 (2009) 1434–1446 1441

Fig. 5. Ball-and-tube system.

Fig. 6. Hardware module of ball-and-tube prototype.

The parameters of the GP approaches are set as follows: population size is 30, generation gap ¼ 0.8, selection adopted is
the tournament with elitist strategy, the tournament size is equal to 6, the maximum number of generations is 200, and the
maximum allowed tree depth is 5. The design parameter adopted for the OLS algorithm is ERR ¼ 0.01.
The system identification by NARX model based on GP is appropriate if a performance index is in values permissible for
the user’s needs. In this case, the fitness function for maximization proposes is given by the multiple correlation index also
known as the R2 coefficient, R2est , of estimation phase. The fitness function is calculated using the expression of R2est given by

Pne 2
^
k¼1 ½yðkÞ  yðkÞ
R2est ¼ 1  P ne 2
(15)
k¼1 ½yðkÞ  ȳ
 ARTICLE IN PRESS
1442 L.S. Coelho, M.W. Pessôa / Mechanical Systems and Signal Processing 23 (2009) 1434–1446

5
output signal
4.5 input signal

3.5

2.5

1.5

0.5

0
0 50 100 150 200 250 300 350
sample

Fig. 7. Input and output signals applied to ball-and-tube prototype.

where R2est is the multiple correlation index of the estimation phase (training phase), ne is the number of samples of the
estimation phase, y(k) is the output of the real system, y^ ðkÞ is the output estimated by the NARX model obtained by GP, and
ȳ is the mean value of the system’s output.
For the validation phase (testing phase) for verification of generalization capability of optimized NARX model, we have
employed the R2val index. The R2val has the form
Pnv 2
^
k¼neþ1 ½yðkÞ  yðkÞ
R2val ¼ 1  P nv 2
(16)
k¼neþ1 ½yðkÞ  ȳ

where R2val is the multiple correlation index of the validation phase, and nv is the number of samples of the validation phase.
When the value R2 ¼ 1.0 (estimation or validation phases), it indicates the model’s accurate approach to the system’s
measured data. A R2 value between 0.9 and 1.0 is considered sufficient for applications in identification and model-based
controller designs [50].
Experiments for the estimation phase of the mathematical model of the ball-and-tube system are carried out using
samples 1 to 251 (ne ¼ 250). For the validation phase, the GP model uses the input and output signals of samples 252 to
330 (nv ¼ 330). The results of nonlinear identification for the NARX model are given in the next subsection.

4.3. Results and discussion for the ball-and-tube system

The maximum input and output order selected for identification with GP are td ¼ 1, nu ¼ 4, and ny ¼ 4. All of the
numeric parameters in GP design are determined empirically.
In this case, the NARX model with series–parallel type is used for one-step ahead forecasting of the GP model in the case
study of ball-and-tube system. A representation of series–parallel is presented in Fig. 8.
Table 1 presents the simulation results (R2est best of 50 runs) and computational cost (mean time) using GP approaches
for NARX modeling. As indicated in Table 1, the results of the optimized NARX are precise, providing an appropriate
experimental mathematical model for the ball-and-tube prototype.
In Table 1, the best mean and maximum values of R2est function from the 50 runs was obtained using GP7. In this case, the
mean R2est function using GP7 was R2est ¼ 0:9758 in NARX model. The equation with R2est best for the NARX model obtained
by GP2 approach was

yðkÞ ¼ 0:066508 þ 1:13344uðk  2Þ  1:410991uðk  3Þ þ 0:53346uðk  1Þuðk  3Þyðk  4Þ

þ 0:194370yðk  1Þ þ 0:190366yðk  1Þ (17)
4
The model of Eq. (17) presents an error mean equal to 1.9684  10 and maximum value of absolute error equal to
0.5633.
From Tables 2 and 3, it can be concluded that a best solution found by GP7 has a slight advantage over the GP2, GP3 and
GP6. The best result of the GP7 approach for the ball-and-tube system is shown in Fig. 9.
The correlation test proposed in [51,52] was employed to test model
pffiffiffiffi validity obtained by GP7. In this context, the
correlation tests lie within 95% confidence limit, defined as 1:95= N , where N is a number of data. Fig. 10 shows the
 ARTICLE IN PRESS
L.S. Coelho, M.W. Pessôa / Mechanical Systems and Signal Processing 23 (2009) 1434–1446 1443

Fig. 8. Representation of series–parallel system for NARX model with GP optimization.

Table 1
Results obtained by the maximization of R2est using NARX model and GP (50 runs).

Optimization Mean time for GP R2est maximum R2est mean R2est minimum R2est standard
method optimization (s) (best) (worst) deviation

GP1 33.5 0.9733 0.9705 0.9589 0.0047

GP2 34.2 0.9746 0.9712 0.9602 0.0045
GP3 31.3 0.9745 0.9711 0.9589 0.0050
GP4 35.3 0.9733 0.9715 0.9689 0.0046
GP5 25.1 0.9669 0.9640 0.9589 0.0017
GP6 34.0 0.9735 0.9713 0.9589 0.0046
GP7 35.9 0.9758 0.9723 0.9645 0.0026

Table 2
Results obtained by the maximization of R2val using NARX model and GP (50 runs).

Optimization method R2val maximum (best) R2val mean R2val minimum (worst) R2val standard deviation

GP1 0.9496 0.9481 0.9437 0.0019

GP2 0.9510 0.9490 0.9432 0.0020
GP3 0.9499 0.9483 0.9431 0.0023
GP4 0.9495 0.9487 0.9439 0.0017
GP5 0.9471 0.9462 0.9432 0.0008
GP6 0.9501 0.9486 0.9431 0.0020
GP7 0.9601 0.9494 0.9449 0.0013
 ARTICLE IN PRESS
1444 L.S. Coelho, M.W. Pessôa / Mechanical Systems and Signal Processing 23 (2009) 1434–1446

Table 3
Best result in terms of R2est obtained using NARX model and GP in 50 runs.

Optimization method R2est R2val np

GP1 0.9733 0.9496 6

GP2 0.9746 0.9510 6
GP3 0.9735 0.9498 5
GP4 0.9733 0.9492 5
GP5 0.9669 0.9470 7
GP6 0.9735 0.9503 6
GP7 0.9758 0.9601 6

3
ball-and-tube output
2.5 estimated output using NARX model

1.5

0.5

-0.5
0 20 40 60 80 100 120 140 160 180 200
sample

0.8

0.6

0.4

0.2
error signal

-0.2

-0.4

-0.6

-0.8
0 20 40 60 80 100 120 140 160 180 200
sample

Fig. 9. Best result of R2est for NARX modeling obtained by GP7.

correlation tests. It can be observed that all the tests fall within 95% confidence limits except for the f(u2,e2) plot that is
marginally outside the limits. Correlation tests R2est and R2val suggest that the mathematical model represented by Eq. (17) is
unbiased and adequate to represent the identified system.

5. Concluding remarks and further research

The essence of system identification is to approximate automatically the behavior of an unknown process using an
appropriate mathematical model. Due to the nonlinear nature of many systems, physical and otherwise, there has been
extensive research covering the field of nonlinear system identification.
 ARTICLE IN PRESS
L.S. Coelho, M.W. Pessôa / Mechanical Systems and Signal Processing 23 (2009) 1434–1446 1445

Fig. 10. Model-validity test for the NARX model obtained by GP7.

Identification of nonlinear system suffers many problems including determination of the structure and parameters of
the system. Many methods of system identification are based on parameter estimation, and mainly rely on least-mean-
squares method. Recently, soft computing-based system identification approaches have been proposed, mainly fuzzy
systems, artificial neural networks, and evolutionary computation methods.
This paper focuses the application of seven tuning approaches of GP method, an evolutionary computation method,
combined with OLS algorithm for structure selection of a NARX model. Unlike conventional optimization techniques that
manipulate the parameters of an initial estimate of the solution, in GP a population of potential models is maintained, with
the probability of a given model surviving into the next generation being proportional to how well it predicts the output
data. Meanwhile, the main disadvantage of GP method in NARX modeling is that often a large number of lagged variables
have to be included in the terminal set and also that it has to known beforehand how many time lags are required to
capture the relevant dynamic behavior.
The identification procedure based on GP approaches and NARX model was validated to identify an experimental ball-
and-tube system. The results demonstrated that the GP method, mainly the GP7 and GP2 approaches, is a consistent
estimator when applied to identification of NARX models.
Future research is to investigate the performance of GP approaches for solving optimization problems in model-based
control design.

Acknowledgments

This work was supported by the National Council of Scientific and Technologic Development of Brazil—CNPq—under
Grant 309646/2006-5/PQ. The authors are thankful to the anonymous referees and the editor for their valuable comments
and suggestions.

References

[1] G.B. Giannakis, E. Serpedin, A bibliography on nonlinear system identification, Signal Processing 81 (3) (2001) 533–580.
[2] S.A. Billings, Identification of nonlinear systems—a survey, IEE Proceedings 127 (6) (1980) 272–285.
[3] L. Ljung, Black-box models from input–output measurements, in: Proceedings of the 18th IEEE Instrumentation and Measurement Technology
Conference, Budapest, Hungary, 2001, pp. 138–146.
[4] R. Haber, H. Unbehauen, Structure identification of nonlinear dynamic systems—a survey on input/output approaches, Automatica 26 (4) (1990)
651–677.
[5] L.A. Aguirre, M.C.S. Coelho, M.V. Corrêa, On the interpretation and practice of dynamical differences between Hammerstein and Wiener models, IEE
Proceedings, Part D: Control Theory and Applications 152 (4) (2004) 349–356.
[6] K.I. Kim, E.J. Powers, A digital method of modelling quadratically nonlinear systems with a general random input, IEEE Transactions on Acoustics,
Speech, and Signal Processing 36 (11) (1988) 1758–1769.
[7] S. Chen, S.A. Billings, Representations of nonlinear systems: the NARMAX model, International Journal of Control 49 (3) (1989) 1013–1032.
 ARTICLE IN PRESS
1446 L.S. Coelho, M.W. Pessôa / Mechanical Systems and Signal Processing 23 (2009) 1434–1446

[8] C.X. Wong, K. Worden, Generalised NARX shunting neural network modelling of friction, Mechanical Systems and Signal Processing 21 (1) (2007)
553–572.
[9] P.M. Pawar, R. Ganguli, Genetic fuzzy system for online structural health monitoring of composite helicopter rotor blades, Mechanical Systems and
Signal Processing 21 (5) (2007) 2212–2236.
[10] Y. Lei, Z. He, Q. Hu, Fault diagnosis of rotating machinery based on multiple ANFIS combination with GAs, Mechanical Systems and Signal Processing
21 (5) (2007) 2280–2294.
[11] M.H. Tan, J.K. Hammond, A non-parametric approach for linear system identification using principal component analysis, Mechanical Systems and
Signal Processing 21 (4) (2007) 1576–1600.
[12] Q. Chen, K. Worden, P. Peng, A.Y.T. Leung, Genetic algorithm with an improved fitness function for (N)ARX modeling, Mechanical Systems and Signal
Processing 21 (2) (2007) 994–1007.
[13] W.D. Chang, An improved real-coded genetic algorithm for parameters estimation of nonlinear systems, Mechanical Systems and Signal Processing
20 (1) (2006) 236–246.
[14] T.L. Lew, A.B. Spencer, F. Scarpa, K. Worden, A. Rutherford, F. Hemez, Identification of response surface models using genetic programming,
Mechanical Systems and Signal Processing 20 (8) (2006) 1819–1831.
[15] G.J. Gray, Y. Li, D.J. Murray-Smith, K.C. Sharman, Structural system identification using genetic programming and a block diagram oriented simulation
tool, Electronic Letters 32 (15) (1996) 1423–1424.
[16] D.C. Dracopoulos, S. Kent, Genetic programming for prediction and control, Neural Computing & Applications 6 (4) (1995) 214–228.
[17] K. Rodrı́guez-Vásquez, P.J. Fleming, Multi-objective genetic programming for nonlinear system identification, Electronics Letters 34 (9) (1998)
930–931.
[18] G.N. Beliagiannis, L.V. Skarlas, S.D. Likothanassis, K.G. Perdikouri, Nonlinear model structure identification of complex biomedical data using a
genetic-programming-based technique, IEEE Transactions on Instrumentation and Measurement 54 (6) (2005) 2184–2190.
[19] W.-X. Yang, Establishment of the mathematical model for diagnosing the engine valve faults by genetic programming, Journal of Sound and Vibration
293 (1–2) (2006) 213–226.
[20] L. Zhang, A.K. Nandi, Fault classification using genetic programming, Mechanical Systems and Signal Processing 20 (3) (2007) 1273–1284.
[21] L. Zhang, L.B. Jack, A.K. Nandi, Fault detection using genetic programming, Mechanical Systems and Signal Processing 19 (2) (2005) 271–289.
[22] P. Chen, M. Taniguchi, T. Toyota, Z. He, Fault diagnosis method for machinery in unsteady operating condition by instantaneous power spectrum and
genetic programming, Mechanical Systems and Signal Processing 19 (1) (2005) 175–194.
[23] J.R. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection, MIT Press, Cambridge, MA, USA, 1992.
[24] M. Korenberg, S.A. Billings, Y.P. Liu, P.J. McIlroy, Orthogonal parameter estimation algorithm for non-linear stochastic systems, International Journal
of Control 48 (1) (1998) 193–210.
[25] R. Assunc-ão, L.S. Coelho, B-spline neural network design using artificial immune network applied to identification of a ball-and-tube prototype. Soft
Computing in Industrial Applications, Advances in Soft Computing 39 (2007) 92–101.
[26] J. Mádar, J. Abonyi, F. Szeifert, Genetic programming for the identification of nonlinear input–output models, Industrial and Engineering Chemistry
Research 44 (2005) 3178–3186.
[27] L. Ljung, System Identification—Theory for the User, second ed., PTR Prentice-Hall, Upper Saddle River, USA, 1999.
[28] J. Schoukens, R. Pintelon, Identification of Linear Systems: A Practical Guideline to Accurate Modeling, Pergamon Press, New York, NY, USA, 1991.
[29] G. Kerschen, K. Worden, A.F. Vakakis, J.C. Golinval, Past, present and future of nonlinear system identification in structural dynamics, Mechanical
Systems and Signal Processing 20 (3) (2006) 505–592.
[30] J. Sjöberg, Q. Zhang, L. Ljung, A. Beneviste, B. Delyon, P.-Y. Glorennec, H. Hjalmarsson, A. Juditsky, Nonlinear black-box modelling in system
identification: a unified overview, Automatica 31 (12) (1995) 1691–1724.
[31] T. Abdelazim, O.P. Malik, Identification of nonlinear systems by Takagi–Sugeno fuzzy logic grey box modeling for real-time control, Control
Engineering Practice 13 (12) (2005) 1489–1498.
[32] T. Söderström, P. Stoica, System Identification, International Series in Systems & Control, PTR Prentice-Hall, Upper Saddle River, USA, 1994.
[33] A. Tsakonas, A comparison of classification accuracy of for genetic programming-evolved intelligent structures, Information Sciences 176 (6) (2006)
691–724.
[34] A.E. Eiben, R. Hinterding, Z. Michalewicz, Parameter control in evolutionary algorithms, IEEE Transactions on Evolutionary Computation 3 (2) (1999)
124–141.
[35] R. May, Simple mathematical models with very complicated dynamics, Nature 261 (1976) 459–467.
[36] L.S. Coelho, V.C. Mariani, Combining of chaotic differential evolution and quadratic programming for economic dispatch optimization with valve-
point effect, IEEE Transactions on Power Systems 21 (2) (2006) 989–996.
[37] G.D. Nicolao, G.F. Trecate, Consistent identification of NARX models via regularization networks, IEEE Transactions on Automatic Control 44 (11)
(1999) 2045–2049.
[38] J. Mádar, J. Abonyi, F. Szeifert, Genetic programming for system identification, in: Proceedings of the Fourth IEEE International Conference on
Intelligent Systems Design and Applications (ISDA 2004) Conference, Budapest, Hungary, 2004.
[39] I.D. Landau, Identification in closed loop: a powerful design tool (better design models, simpler controllers), Control Engineering Practice 9 (1) (2001)
51–65.
[40] S. Lakshminarayanan, G. Emoto, S. Ebara, K. Tomida, S.L. Shah, Closed loop identification and control loop reconfiguration: an industrial case study,
Journal of Process Control 11 (2001) 587–599.
[41] U. Forssell, Closed-loop identification methods, theory, and applications, Ph.D. Dissertation, Department of Electrical Engineering, Linköping
University, Linköping, Sweden, 1999.
[42] B. Huang, On-line closed loop model validation and detection of abrupt parameter changes, Journal of Process Control 11 (6) (2001) 699–715.
[43] B. Huang, S. Shah, Closed-loop identification: a two step approach, Journal of Process Control 7 (6) (1997) 425–438.
[44] M. Gilsona, P.M.J. Van den Hof, Instrumental variable methods for closed-loop system identification, Automatica 41 (2) (2005) 241–249.
[45] P.M.J. Van den Hof, Closed-loop issues in system identification, Annual Reviews in Control 22 (1998) 173–186.
[46] P.M.J. Van den Hof, R.J.P. Schrama, Identification and control—closed loop issues, Automatica 31 (12) (1995) 1751–1770.
[47] U. Forssell, L. Ljung, Closed-loop identification revisited, Automatica 35 (7) (1999) 1215–1241.
[48] Y. Zhou, A. Han, S. Yan, X. Chen, A fast method for online closed-loop system identification, International Journal of Advanced Manufacturing
Technology 31 (1) (2006) 78–84.
[49] M. Yuwana, D.E. Seborg, A new method for on-line controller tuning, AIChE Journal 28 (3) (1982) 434–440.
[50] B. Schaible, H. Xie, Y.C. Lee, Fuzzy logic models for ranking process effects, IEEE Transactions on Fuzzy Systems 5 (4) (1997) 545–556.
[51] S.A. Billings, Q.M. Zhu, Nonlinear model validation using correlation tests, International Journal of Control 60 (6) (1994) 1107–1120.
[52] S.A. Billings, W.S.F. Voon, Correlation based model validity tests for non-linear models, International Journal of Control 44 (1) (1996) 233–244.