Neural

Networks

PERGAMON Neural Networks 14 (2001) 23±35

www.elsevier.com/locate/neunet

Optimal control by least squares support vector machines

J.A.K. Suykens*, J. Vandewalle, B. De Moor
Department of Electrical Engineering, ESAT-SISTA, Katholieke Universiteit Leuven, Kardinaal Mercierlaan 94, B-3001 Leuven, Heverlee, Belgium
Received 23 September 1998; accepted 31 August 2000

Abstract
Support vector machines have been very successful in pattern recognition and function estimation problems. In this paper we introduce the
use of least squares support vector machines (LS-SVM's) for the optimal control of nonlinear systems. Linear and neural full static state
feedback controllers are considered. The problem is formulated in such a way that it incorporates the N-stage optimal control problem as well
as a least squares support vector machine approach for mapping the state space into the action space. The solution is characterized by a set of
nonlinear equations. An alternative formulation as a constrained nonlinear optimization problem in less unknowns is given, together with a
method for imposing local stability in the LS-SVM control scheme. The results are discussed for support vector machines with radial basis
function kernel. Advantages of LS-SVM control are that no number of hidden units has to be determined for the controller and that no centers
have to be speci®ed for the Gaussian kernels when applying Mercer's condition. The curse of dimensionality is avoided in comparison with
de®ning a regular grid for the centers in classical radial basis function networks. This is at the expense of taking the trajectory of state
variables as additional unknowns in the optimization problem, while classical neural network approaches typically lead to parametric
optimization problems. In the SVM methodology the number of unknowns equals the number of training data, while in the primal space
the number of unknowns can be in®nite dimensional. The method is illustrated both on stabilization and tracking problems including
examples on swinging up an inverted pendulum with local stabilization at the endpoint and a tracking problem for a ball and beam system.
q 2001 Elsevier Science Ltd. All rights reserved.
Keywords: Neural optimal control; Support vector machines; Radial basis functions

1. Introduction application to control of a space robot. Plumer (1996)

studied the case of optimal ®nal time. In Suykens, De
Since the introduction of backpropagation through time Moor, and Vandewalle (1994) and Suykens, Vandewalle,
(Werbos, 1990) and dynamic backpropagation (Narendra & and De Moor (1996) the problem of swinging up of an
Parthasarathy, 1991), the area of neural optimal control has inverted pendulum and double inverted pendulum with
been focusing on gradient based approaches or backpropa- stabilization at the endpoint has been formulated as a para-
gation type solutions for dynamical systems involving metric optimization problem in the unknown weights of
neural networks. Among the many modelbased approaches, feedforward or recurrent neural controllers.
the N-stage optimal control problem (Bryson & Ho, 1969) In this paper we discuss N-stage optimal control by least
has been studied in combination with several structures for squares support vector machines (LS-SVM's). Support
the neural controller. For example, Nguyen and Widrow vector machines have been recently introduced for solving
(1990) applied backpropagation to the problem of backing pattern recognition and function estimation problems
up a trailer±truck based on a neural network emulator for (SchoÈlkopf, Burges, & Smola, 1999; Vapnik, 1995,
the plant. In Saerens, Renders, and Bersini (1993) a full 1998a). In this method one maps the data into a higher
static state feedback controller has been studied in the dimensional input space and one constructs an optimal
context of optimal control, thereby relating the Lagrange separating hyperplane in this space. Hereby one exploits
multiplier sequence to the backpropagation algorithm. Pari- Mercer's theorem, which avoids an explicit formulation of
sini and Zoppoli (1994) assumed a linear structure preser- this nonlinear mapping. The solution is written as a
ving principle for the state-tracking problem with weighted sum of the data points. In the original formulation
one solves a quadratic programming problem, which yields
* Corresponding author. Tel.: 132-1-632-1802; fax: 132-1-632-1970.
many zero weights. The data points corresponding to the
E-mail address: [email protected] (J.A.K. Suykens). non-zero weights are called support vectors. Kernel function
0893-6080/01/$ - see front matter q 2001 Elsevier Science Ltd. All rights reserved.
PII: S 0893-608 0(00)00077-0
24 J.A.K. Suykens et al. / Neural Networks 14 (2001) 23±35

parameters can be chosen such that a bound on the general- optimal trajectory that one seeks. Furthermore, standard
ization error is minimized, expressed in terms of the VC neural network approaches always work in a primal weight
dimension. One has the possibility to use polynomials, space, while in SVM methodologies the computations are
splines, radial basis function (RBF) networks or multilayer done in a dual space such that the number of unknowns
perceptrons as kernels. Although one preferably applies equals the number of training data points (and not the
Mercer's condition, this is not absolutely necessary as is number of weights in the primal space, which can be in®nite
shown e.g. in Suykens and Vandewalle (1999a), where dimensional).
standard multilayer perceptron classi®ers with a ®xed We illustrate the LS-SVM control method on a number
number of hidden units are trained using a SVM method- of simulation examples including swinging up an inverted
ology. Being based on the structural risk minimization prin- pendulum with local stabilization at the endpoint and a
ciple and capacity concept with pure combinatorial tracking problem for a ball and beam system. In Suykens
de®nitions, the quality and complexity of the SVM solution et al. (1994), methods for realizing a transition between
does not depend directly on the dimensionality of the input two states with local stabilization at the target point was
space. discussed. This was illustrated on swinging up an inverted
In the SVM control method we make use of a least pendulum with local stabilization at the endpoint. For the
squares version of support vector machines. Originally, full static state feedback case a Linear Quadratic Regula-
Vapnik's epsilon insensitive loss function has been tor (LQR) (Franklin, Powell, & Workman, 1990) was
employed for the function estimation problem. A least designed for balancing the pole in its upright position.
squares interpretation on the other hand has been given by This result was used in order to impose a set of
Saunders, Gammerman, and Vovk (1998) for function esti- constraints on the interconnection weights of a multilayer
mation problems, Suykens and Vandewalle (1999b) for perceptron controller. A drawback of the approach was
classi®cation, and MuÈller et al. (1997) for time±series that the number of hidden units for the neural controller
prediction. In this case, the problem formulation involves had to be chosen ad hoc. In this paper we present a solu-
equality instead of inequality constraints and the support tion by SVM control which takes into account an LQR
values are proportional to the errors at the data points, design in the LS-SVM control with RBF kernel. The
which simpli®es the problem. Sparseness gets lost when number of hidden units is determined here by the number
the function estimation corresponds to this form of ridge N in the N-stage optimal control problem formulation.
regression, but this can be imposed afterwards by doing Instead of local linear LQR design one has also the possi-
pruning based upon the support value spectrum (Suykens, bility to apply robust linear control methods (Boyd &
Lukas, & Vandewalle, 2000a,b). Barratt, 1991) such as H1 control and m theory instead,
In the optimal control method by LS-SVM's, the N-stage in order to take into account uncertainties (parametric
optimal control problem and the optimization problem uncertainties, unmodelled dynamics) and noise.
related to the LS-SVM controller are incorporated within For the example of a ball and beam system (Hauser,
one problem formulation. A model is assumed to be avail- Sastry, & Kokotovic, 1992) a tracking problem with SVM
able for the plant, the controller is designed based upon this control is discussed. Again a LQR design is incorporated in
model and afterwards applied to the plant, assuming that the order to impose local stability of the origin for the autono-
certainty equivalence principle holds (similar to Nguyen mous closed-loop system. Imposing robust local stability for
Widrow's emulator approach). The solution is characterized the closed-loop system has also been successful in the appli-
by a set of nonlinear equations. However, the set of cation of NLq neural control theory, introduced in Suykens,
nonlinear equations will typically contain a large number De Moor, and Vandewalle (1997) and Suykens et al. (1996),
of unknowns. Therefore an alternative formulation for LS- to the control of a real-life ball and beam system (Verrelst
SVM control is given in less unknowns. This formulation et al., 1998). Within NLq theory suf®cient conditions for
also enables to incorporate local stability constraints. A main global asymptotic stability and input/output stability with
difference between standard neural network approaches ®nite L2-gain are available, which can also be employed in
(Bishop (1995) MLP, RBF) and LS-SVM control is that in order to impose robust local stability of the origin for the
the former one solves a parametric optimization problem in closed-loop system. A similar approach is followed in the
the unknown interconnection weights while in the latter also LS-SVM control for the ball and beam system, where robust
the state vector sequence is part of the unknown parameter local stability can be realized based upon the work of Sezer
vector in the optimization problem. However, standard and Siljak (1998).
methodologies suffer from problems like the choice of the This paper is organized as follows. In Section 2 we
number of hidden units needed in order to accomplish a formulate the N-stage optimal control problem. In
given control task. More speci®cally, in the case of RBF Section 3 we review work on support vector machines. In
networks one has a curse of dimensionality when one de®nes Section 4 we discuss optimal control by least squares
a regular grid for the centers (hidden units) in state space, as support vector machines. In Section 5 we discuss an alter-
explained e.g. in the method of Sanner and Slotine (1992). In native formulation together with stability constraints. In
the LS-SVM control case, the centers will follow from the Section 6 we present examples.
 J.A.K. Suykens et al. / Neural Networks 14 (2001) 23±35 25

2. The N-stage optimal control problem 3. The support vector method of function estimation

In the N-stage optimal control problem one aims at In this section we review basic ideas of the support vector
solving the following problem (Bryson & Ho, 1969): method of function estimation, that are essential for its intro-
duction within the neural control problem as will be consid-
X
N
ered in the sequel. For further details on SVM's we refer to
minJN xk ; uk † ˆ r xN11 † 1 h x k ; uk † 1†
kˆ1
Cherkassky and Mulier (1998), Haykin (1994), SchoÈlkopf et
al. (1999), Smola (1999), Smola, SchoÈlkopf, and Muller
subject to the system dynamics (1998), Vapnik (1995, 1998a,b), Vapnik, Golowich, and
Smola (1997), SchoÈlkopf et al. (1997) and Smola and
xk11 ˆ f xk ; uk †; k ˆ 1; ¼; N x1 given†; SchoÈlkopf (1998).
Consider regression in the following set of functions
where r ´† and h ´; ´† are positive de®nite functions. A typi-
cal choice is the quadratic cost h xk ; uk † ˆ xTk Qxk 1 F X† ˆ WT w X† 1 B 5†
uTk Ruk ; r xN11 † ˆ xTN11 QxN11 withQ ˆ QT . 0; R ˆ RT .
0: The functions r ´†; h ´; ´†; f ´; ´† are assumed to be twice with given training data {Xi ; Yi }M
iˆ1 where M denotes the

continuously differentiable. xk [ Rn denotes the state number of training data, Xi [ Rm are the input data and
vector, uk [ R is the input of the system. The methods Yi [ R are the output data. The nonlinear mapping w :
discussed in this paper are not restricted to single input Rm ! Rnh maps the input data into a so-called high dimen-
systems. sional feature space (which can be in®nite dimensional) and
In order to ®nd the optimal control law, one constructs the W [ Rnh ; B [ R: In the support vector method one aims at
Lagrangian minimizing the empirical risk
1 XM
X
N
Remp W; B† ˆ uY 2 WT w Xi † 2 Bu1 6†
LN xk ; uk ; lk † ˆ JN xk ; uk † 1 lTk ‰xk11 2 f xk ; uk †Š 2† M iˆ1 i
kˆ1
subject to elements of the structure Sn, de®ned by the
with Lagrange multipliers lk [ Rn : The conditions for inequality WT W # cn : The loss function employs
optimality are given by (Fletcher, 1987; Gill, Murray, & Vapnik's 1 -insensitive model:
Wright, 1981) (
0; if uY 2 F X†u # 1
8
> 2LN 2h

2f T
 uY 2 F X†u1 ˆ
>
>
>
> 2xk
ˆ
2xk
1 lk21 2
2xk
lk ˆ 0; k ˆ 2; ¼; N adjoint equation† uY 2 F X†u 2 1; otherwise:
>
>
>
>
>
>
>
2LN
ˆ
2r
1 lN ˆ 0 adjoint final condition†
7†
< 2xN11 2xN11
>
> 2LN 2h 2f The estimation problem is formulated then as the optimiza-
>
> ˆ 2 lTk ˆ 0; k ˆ 1; ¼; N variational condition†
>
>
>
>
2uk 2uk 2uk tion problem:
>
>
>
> 2LN (M )
:
2 lk
ˆ xk11 2 f xk ; uk † ˆ 0; k ˆ 1; ¼; N system dynamics†:
p 1 T
X p X M
min J 1 W; j ; j† ˆ W W 1 g j i 1 j i 8†
3† W;B;j p ;j 2 iˆ1 iˆ1

For the case of a quadratic cost function subject to linear subject to the constraints
system dynamics with in®nite time horizon, the optimal 8
>
> Yi 2 WT w Xi † 2 B # 1 1 j ip ; i ˆ 1; ¼; M
control law can be represented by full static state feedback >
>
>
< 2Yi 1 WT w Xi † 1 B # 1 1 j i ;
control. However, in general, the optimal control law cannot i ˆ 1; ¼; M
be represented by state feedback as the optimal control law > j p $ 0;
>
> i i ˆ 1; ¼; M
may also depend on the co-state. Nevertheless, one is often >
>
:
interested in ®nding a suboptimal control strategy of this j i $ 0; i ˆ 1; ¼; M
form. In the context of neural control, Saerens et al.
where j i ; j ip are slack variables
P and pg is a positive real
(1993) considered, in addition to Eq. (1),
constant. One obtains W ˆ M iˆ1 ai 2 ai †w Xi † where
uk ˆ g x k † 4† api ; ai are obtained by solving a quadratic program and are
the Lagrange multipliers related to the ®rst and second set of
where g ´† is parametrized by a neural network architecture, constraints. The data points corresponding to non-zero
and discussed the link with the backpropagation algorithm. values for api 2 ai † are called support vectors. Typically,
In this paper, we will also consider a control law given in many of these values are equal to zero. Finally, one obtains
Eq. (4), by relating Eq. (4) to support vector machines. the following model in the dual space
Because SVM methodology is not a parametric modelling
X
M
approach, this is less straightforward in comparison with F X† ˆ api 2 ai †K Xi ; X† 9†
standard neural networks such as MLP's and RBF's. iˆ1
26 J.A.K. Suykens et al. / Neural Networks 14 (2001) 23±35

where the kernel function K corresponds to equations after elimination of W and 1 i

2 3" # " #
K Xi ; X† ˆ w Xi †T w X† 10† 0 1~T B 0
4 5 ˆ 15†
according to Mercer's condition. One has several possibili- 1~ V 1 g I 21 a Y
ties for the choice of this kernel function, including linear,
with
polynomial, splines, RBF. In the sequel of this paper we will
focus on RBF kernels. X ˆ ‰X1 ; ¼; XM Š;
The in¯uence of the number of support vectors on the
generalization performance has been theoretically studied Y ˆ ‰Y1 ; ¼; YM Š;
in Vapnik (1995). An extension studied in the context of
the 1 -insensitive loss function is
(M ) 1~ ˆ ‰1; ¼; 1Š;
p 1 T
X pp X M
p
J1;p W; j ; j† ˆ W W 1 g ji † 1 ji† 11†
2 iˆ1 iˆ1 a ˆ ‰a1 ; ¼; aM Š
where p ˆ 1 corresponds to Eq. (8). General convex cost and
functions have been investigated in Smola (1999). Further-
more, links between these loss functions and regularization V ij ˆ K Xi ; Xj † ˆ w Xi †T w Xj †; i; j ˆ 1; ¼; N: 16†
theory have been studied in Smola (1999) and Smola et al. The support values a i are proportional to the errors at the data
(1998). points Eq. (14), while in the case of Eq. (8) most values are
For the sequel we employ a least squares version of the equal to zero. Hence sparseness is lost in the least squares case
support vector method, which has been investigated by but it can be imposed by pruning the support value spectrum
Saunders et al. (1998) and Suykens and Vandewalle (Suykens et al., 2000a,b). Although other norms such as
(1999b), for function estimation and classi®cation problems, Huber's loss function are more appropriate concerning robust-
respectively. It corresponds to p ˆ 2 and the following form of ness (Smola, 1999), least squares norms are used in many
ridge regression (Golub & Van Loan, 1989) applications in identi®cation and control theory such as in
the context of prediction error algorithms (Ljung, 1987). In
1 1X M
min J LS W; B; j† ˆ W TW 1 g j2 12† this paper, we restrict to the least squares case, because of the
W;B;j 2 2 iˆ1 i equality constraints in the problem formulation (which is more
tractable than a SVM version with inequality constraints) in
subject to the equality constraints
order to introduce SVM's within the context of optimal
Yi ˆ WT w Xi † 1 B 1 j i ; i ˆ 1; ¼; M: control.

One de®nes the Lagrangian

4. Optimal control by support vector machines
X
M
T
LLS W; B; j; a† ˆ J LS W; B; j† 2 a i W w Xi † 1 B Now, we relate the training data {Xi ; Yi }M
iˆ1 in Eq. (5)
iˆ1
to the state space and action space in Eq. (1), i.e.
1 j i 2 Yi † {xk ; uk }Nkˆ1 : We state the following N-stage optimal
control problem:
13†
where ai are Lagrange multipliers (which can be either posi- 1 T 1 XN
minJ xk ; uk ; w; ek † ˆ JN xk ; uk † 1 w w1g e2
tive or negative due to the equality constraints as follows from 2 2 kˆ1 k
the Kuhn±Tucker conditions (Fletcher, 1987)). The condi- 17†
tions for optimality
8
subject to the system dynamics
>
> 2LLS X
M
>
> ˆ0!Wˆ ai w X i † xk11 ˆ f xk ; uk †; k ˆ 1; ¼; N x1 given†
>
> 2W
>
> iˆ1
>
>
>
and the control law
>
> 2LLS X
M
>
< ˆ0! ai ˆ 0
2B uk ˆ w T w x k † 1 e k ; k ˆ 1; ¼; N 18†
>
iˆ1 14†
>
> 2LLS where JN is de®ned in Eq. (1), w [ R ; w ´† : R ! nh n
>
>
> ˆ 0 ! ai ˆ gj i ; i ˆ i; ¼; M
>
>
> 21i Rnh with nh the number of hidden units (which can be
>
>
>
> 2L in®nite dimensional) 1 of w ´†: The actual control signal
>
:
LS
ˆ 0 ! WT w Xi † 1 B 1 j i 2 Yi ˆ 0; i ˆ i; ¼; M
2 ai
1
The number of hidden units of the LS-SVM with kernel function K is
can be written as the solution to the following set of linear not nh but N.
 J.A.K. Suykens et al. / Neural Networks 14 (2001) 23±35 27

applied to the plant is assumed to be wT w xk †: In the The set of nonlinear Eq. (21) is of the form
linear control case one has w xk † ˆ xk with nh ˆ n: In
F1 xk ; xN11 ; uk ; w; ek ; lk ; ak † ˆ 0 23†
the sequel we will employ RBF kernels. The support
vector method, as it has been originally introduced, for k ˆ 1; ¼; N with x1 given and can be numerically solved
exploits Mercer's condition such that one does not in the unknown variables. In Appendix A a formulation
have to construct w ´†: Although the use of this kernel without ek variables is given. In comparison with function
function is preferred, one could also evaluate w ´† expli- estimation or classi®cation problems, one looses the advan-
citly, as it has been demonstrated in Suykens and tage of solving a quadratic program or a linear least squares
Vandewalle (1999a) for multilayer perceptron classi®ers. problem. Nevertheless, one is still able to exploit some of
For the RBF case, Eq. (18) becomes then the interesting SVM features.
 As in SVM theory, Mercer's
Xnh
1  Pcondition can be applied with-
uk ˆ wi exp 2 2 xk 2 ci 22 1 ek 19† in Eq. (21), by replacing w ˆ k ak w xk † in the equations. For
s
iˆ1 kernels satisfying Mercer's condition one can impose
where ci [ Rn are chosen centers and s is a chosen
K xk ; xl † ˆ w xk †T w xl †: 24†
constant. One can take for example the set {c i }niˆ1 h

N
equal to {xk }kˆ1 in order to avoid additional unknown For RBF kernels one takes (Vapnik, 1995)
parameters for the centers which is also motivated by
the work on regularization networks by Poggio and K xk ; xl † ˆ exp 2hixk 2 xl i22 † 25†
Girosi (1990). In standard SVM theory for static func- with h a positive real constant. One sees that Eq. (25) does not
tion estimation problems s can be selected so as to contain the centers ci as in Eq. (19). After this elimination of w,
minimize an upper bound on the generalization error. a set of nonlinear equations of the form
These bounds are not applicable in the context of
SVM control due to the fact that the input patterns to F2 xk ; xN11 ; uk ; lk ; ak † ˆ 0 26†
the activation function are not independent from each
for k ˆ 1; ¼; N with x1 given is obtained. More speci®cally,
other. Hence s should be chosen as hoc (typically avoid
after exploiting Mercer's condition, one has
values which are too small) or could be taken as an
8  
additional unknown within the cost function J. >
> 2h 2f T XN
2K xk ; xl †
>
> 1 lk21 2 lk 2 ak al k ˆ 2; ¼; N
In order to ®nd the optimal control law we construct the >
>
> 2x k 2x k lˆ1
2xk
>
>
Lagrangian >
> 2 r
>
> 1 lN ˆ 0
>
> 2x
1 T 1 XN >
< N11
L x k ; uk ; w; ek ; lk ; ak † ˆ JN xk ; uk † 1 w w1g e2 2h 2f : 27†
2 2 kˆ1 k >
>
> 2uk
2 lTk
2uk
1 ak ˆ 0; k ˆ 1; ¼; N
>
>
>
>
>
> xk11 2 f xk ; uk † ˆ 0; k ˆ 1; ¼; N
X
N X
N >
>
>
1 lTk ‰xk11 2 f xk ; uk †Š 1 T
ak ‰uk 2 w w xk † 2 ek Š: >
>
>
XN
>
: uk 2 al K xl ; xk † 2 ak =g ˆ 0; k ˆ 1; ¼; N
kˆ1 kˆ1
lˆ1
20†
For RBF kernels one has
The conditions for optimality are given by
8 2K xk ; xl †
ˆ 22h xk 2 xl † exp 2hixk 2 xl i22 †:
 
>
>
>
2L
ˆ
2h
1 lk21 2
2f T
lk 2 ak
2
‰wT w xk †Š ˆ 0; k ˆ 2; ¼; N adjoint equation† 28†
>
>
>
>
>
2xk 2xk 2xk 2xk 2xk
>
> 2L 2r
>
> ˆ 1 lN ˆ 0; adjoint final condition†
>
>
>
>
>
>
2xN11 2xN11 The actual control signal applied to the plant becomes
>
> 2L 2h 2f
>
>
> ˆ 2 lTk 1 ak ˆ 0; k ˆ 1; ¼; N variational condition†
>
>
>
>
<
2uk 2uk 2uk
X
N
2L XN
a k w xk † ˆ 0 uk ˆ al K x l ; x k † 29†
> 2w ˆ w 2
>
support vectors†
>
> kˆ1 lˆ1
>
> 2L
>
>
>
> ˆ gek 2 ak ˆ 0 k ˆ 1; ¼; N support values†
>
>
>
>
>
2ek
where {xl }Nlˆ1 ; {al }Nlˆ1 are obtained from solving the set of
>
> 2L
>
>
>
>
> 2lk
ˆ xk11 2 f xk ; uk † ˆ 0; k ˆ 1; ¼; N system dynamics† nonlinear Eq. (27) and xk is the actual state vector at time k. The
>
>
>
>
>
:
2L
ˆ uk 2 wT w xk † 2 ek ˆ 0; k ˆ 1; ¼; N SVM control†:
data {xl }Nlˆ1 are used as support vector data for the control
2ak
signal. Furthermore, note that ek has been taken equal to zero
21† in Eq. (29).
For RBF kernels one has
X  
2 T 1 2 5. Alternative formulation and local stability
‰w w xk †Š ˆ 22 wi exp 2 2 ixk 2 ci i2
2xk s
i 22†
In this section we give an alternative formulation to the
2
£ xk 2 ci †=s : LS-SVM control approach in less unknowns, together with a
28 J.A.K. Suykens et al. / Neural Networks 14 (2001) 23±35

method for imposing local stability at the origin for the be formulated as:
autonomous closed-loop system.
X
N
Based upon Eqs. (21) and (27) we consider the optimiza- min JN xk ; xeq ; uk † 1 l a2k 32†
tion problem xk ;ak ;Q
kˆ1

X
N subject to
min JN xk ; xrk ; uk † 1l a2k 30† 8 !
xk ; a k >
> XN
eq eq
kˆ1 < xk11 ˆ f xk ; al K ‰xl ; x Š; ‰xk ; x Š† ; x1 given
lˆ1
>
>
subject to : T
A PA 2 P , 0
8
>
> x ˆ f x k ; uk † x1 given; where P ˆ QT Q; A ˆ 2f =2x^k ux^k ˆxeq and Z , 0 denotes a
< k11
X
N negative de®nite symmetric matrix.
> al K ‰xl ; xrl Š; ‰xk ; xrk Š†:
: uk ˆ
> Imposing robust local stability for the closed-loop system
lˆ1 has been successfully applied within the context of NLq neural
control theory (Suykens et al., 1997, 1996). Within NLq theory
In this formulation a reference state vector xrk is considered suf®cient conditions for global asymptotic stability and input/
and g ! 1 is taken P with respect to Eq. (27). A regulariza- output stability with ®nite L2-gain are available, which can also
tion by the term k a2k is included (see also Smola (1999)) be employed in order to impose robust local stability of the
where l is a positive real constant. Instead of solving a set origin for the closed-loop system. Based upon NLq stability
of nonlinear equations one solves then a constrained criteria, dynamic backpropagation can be modi®ed by impos-
nonlinear optimization problem with less unknowns than ing matrix inequality constraints as in Eq. (32). For Eq. (32)
in the approaches of the previous section. From the formu- also robust local stability can be imposed, e.g. based upon
lation (30) the difference between standard neural network (Sezer & Siljak, 1998).
controllers and LS-SVM control is clear. In the former one
solves a parametric optimization problem in the unknown
interconnection weights while in the latter also the state 6. Simulation examples
vector sequence is part of the unknown parameter vector
in the optimization problem. Furthermore, standard neural 6.1. Example 1
network approaches always work in the primal weight
space, while in SVM methodology the computations are Here we illustrate the LS-SVM optimal control method of
done in the dual space such that the number of unknowns Section 4 on an example reported in Narendra and Mukho-
equals the number of training data points (and not the padhyay (1997). Given the nonlinear system
8
number of weights in the primal space, which can be in®nite > uk 1 x2;k
>
> x1;k11 ˆ 0:1x1;k 1 2
dimensional). >
< 1 uk 1 x2;k †2
For RBF kernels the parameter h in Eq. (25) can be taken ! 33†
as an additional unknown to the cost function (30). When >
> u2k
>
> x2;k11 ˆ 0:1x2;k 1 uk 2 1
applying an optimization method the constraints in Eq. (30) : 1 1 x21;k 1 x22;k
will only hold with a certain tolerance. These small numer-
ical errors may lead to differences between the state vectors we consider a state vector tracking problem with
as solution to Eq. (30) and the simulation of the closed-loop h xk ; uk † ˆ xk 2 xrk †T Q xk 2 xrk † 1 uTk Ruk ; r xN11 † ˆ
system with LS-SVM control, especially in the control of xN11 2 xrN11 †T Q xN11 2 xrN11 † where xrk is the reference
unstable systems. For control of stable systems this problem trajectory to be tracked. We aim at tracking the ®rst
will be less critical. A simulation of the closed-loop system state variable and choose Q ˆ diag{1; 0:001}; R ˆ 1
with LS-SVM control for xrk ˆ ‰sin 2pk=20†; cos 2pk=20†Š with k ˆ 1; N and
N ˆ 20: The given initial state is x1 ˆ ‰0; 0Š: As control
! law is taken wT w ‰xk ; xrk Š†; which results only into a
X
N
x^k11 ˆ f x^k ; al K ‰xl ; xrl Š; ‰x^k ; xrk Š† x^1 ˆ x1 given slight modi®cation of Eqs. (21) and (27). In Fig. 1,
lˆ1 simulation results are shown for the method (27) that
31† makes use of Mercer's condition. The set of nonlinear
equations was solved using Matlab's optimization tool-
is always needed in order to validate the results, where box (function leastsq) with unknowns xk, uk, l k, a k, s
{xl }Nlˆ1 and {al }Nlˆ1 are obtained as a solution to Eq. (30). (variables ek eliminated) for g ˆ 100: The unknowns
In order to impose local stability at a ®xed equilibrium were randomly initialized with zero mean and standard
point x eq, one locally linearizes the autonomous closed-loop deviation 0.3. The plots show the simulation results for
P
simulation model (31) by evaluating 2f =2x^k at x eq. A LS- the closed-loop system x^ k11 ˆ f x^k ; Nlˆ1 al K xl ; x^k †† with
SVM control design with local stability constraint can then RBF kernel. The controller is generalizing well with
 J.A.K. Suykens et al. / Neural Networks 14 (2001) 23±35 29

Fig. 1. (Top) Optimal control by a least squares support vector machine with RBF kernel and application of Mercer's condition. (Full line) ®rst state variable to
be tracked; (Dashed line) actual state variable by closed-loop simulation of the system. The data k ˆ 1; ¼; 20 were used for training of the controller with the
origin as initial state. (Bottom) simulation result for another randomly chosen initial state x1. The LS-SVM controller shows a good generalization performance
with respect to other initial states and beyond the time horizon.

respect to other initial conditions than the origin (for with

which it has been trained) and beyond the time horizon 2 3
of N ˆ 20: The method (22) without the use of x2
6 7
Mercer's condition gave similar results by taking the 6 4 mlx2 sin x 2 mg sin 2x † 7
6 3 4 3 2 3 7
centers {ci} equal to {xk}. 6 7
6 4 2 7
6 3 mt 2 m cos x3 7
F x† ˆ 6 7;
6 x4 7
6 7
6 7
6.2. Example 2: inverted pendulum 6 m g sin x 2 ml x2 sin 2x † 7
4 t 3 2 4 3 5
l 43 mt 2 m cos2 x3 †
In this example we illustrate the LS-SVM control method
2 3
of Section 5 on the problem of swinging up an inverted 0
pendulum with local stabilization at the endpoint, which 6 1 7
6 4 7
has been investigated in Suykens et al. (1994). 6 3 4 7
6 2 7
A nonlinear state space model for the inverted pendulum 6 3 mt 2 m cos x3 7
G x† ˆ 6 7:
(Fig. 2) system is given by 6 0 7
6 7
6 cos x3 7
4 5
2 4
x_ ˆ F x† 1 G x†u 34† 2
l 3 mt 2 m cos x3 †
30 J.A.K. Suykens et al. / Neural Networks 14 (2001) 23±35

The state variables x1, x2, x3, x4 are respectively position and with
velocity of the cart, angle of the pole with the vertical and
rate of change of the angle. The input signal u is the force X
N

applied to the cart's center of mass. The symbols m, mt, l, g usvm

k ˆ al K xl ; xk † 39†
lˆ1
denote respectively mass of the pole, total mass of cart and
pole, half pole length and the acceleration due to gravity. where Llqr is the resulting feedback matrix from LQR design
We take m ˆ 0:1; mt ˆ 1:1; l ˆ 0:5: Remark that in the and
autonomous case x ˆ ‰0; 0; 0; 0Š (pole up) and x ˆ
‰0; 0; p; 0Š (pole down) are equilibrium points. The linear- 2usvm
k
ized system around the target equilibrium point (the origin) LD ˆ 40†
2xk xk ˆ0
is given by

x_ ˆ Ax 1 Bu 35† is a modi®cation to the LS-SVM control law such that locally

at the origin the overall controller uk is acting as a LQR control-
with ler, in a continuous time sense. The mixture between a contin-
2 3 uous time LQR result and the discrete time SVM control law
0 1 0 0 may look surprising at ®rst sight, but here it is a most conve-
6 mg 7 nient way to design an overall LS-SVM controller for the
60 0 2 4 07
6 7
6 3 mt 2 m 7 given continuous time model. An approach according to Eq.
Aˆ6
6
7;
7 (32) would be much more complicated here due to the Runge±
60 0 0 17
6 7 Kutta integration rule. An RBF kernel function is used and
4 mt g 5
0 0 0 expression (22) is applied in order to compute Eq. (40).
l 43 mt 2 m†
The resulting closed-loop simulation model is given by
2 3 36†
0 !
6 7 X
N
6 4 1 7 x^k11 ˆ F x^k ; Llqr 2 LD †x^k 1 al K xl ; x^k † 41†
6 7
6 3 4
3 mt 2 m
7 lˆ1
6 7
Bˆ6 7:
6 0 7
6 7 with x^1 ˆ x1 ˆ 0 given. In Eq. (41) {xl }Nlˆ1 and {al }Nlˆ1 are
6 7
4 1 5 the solution to the constrained optimization problem (30).
2 4
l 3 mt 2 m† As cost function in Eq. (30) has been de®ned

In Suykens et al. (1994) for the neural controller, a LQR X

N
2
controller (Boyd & Barratt, 1991; Franklin et al., 1990) was JN ˆ ixk i2 42†
designed ®rst based upon the linearized model (eventually, kˆN 2 5

also robust linear controllers could be designed, such as

based upon H1 control and m theory, if additional robust- with xrk ˆ 0 and l ˆ 0: As time horizon has been taken N ˆ
ness with respect to noise and uncertainties would be 100 (5 s). The constrained optimization problem has been
needed). This was done in order to impose a set of solved using Matlab's optimization toolbox by SQP
constraints on the choice of the interconnection weights of (sequential quadratic programming) (function constr). The
a multilayer perceptron controller. As a result a swinging up values xk and ak were initialized by taking small random
neural control has been obtained which ensures that the pole values. The parameter h of the RBF kernel was taken as
remains locally stabilized in its upright position. additional unknown in the optimization problem with start-
In order to solve the swinging up problem using LS-SVM ing point 0.1. In order to emphasize the equations of the
control, we follow a somewhat similar approach for the LS- constraints, they have been multiplied with a factor 1000.
SVM control case now. Based on the continuous time model This is needed due to fact that the system to be controlled is
(34) we design a LQR unstable and small differences between xk (as solution to
R controller for Q ˆ I; R ˆ 0:01 in the
Eq. (30)) and x^k (in the simulation model (31)) may cause
LQR cost function 1 0 x T
Qx 1 u T
Ru† dt (Matlab function
lqr). Then the continuous time model (34) is discretized by large differences otherwise. Simulation results of the closed-
using a fourth order Runge±Kutta integration rule with loop simulation model are shown in Figs. 3 and 4.
constant step h ˆ 0:05 into a discrete time model of the form

xk11 ˆ F xk ; uk † 37† 6.3. Example 3: ball and beam

where uk is assumed to be constant in the time intervals [kh, Here we discuss the LS-SVM control method of Section 5
(k 1 1)h] (zero order hold). The control law is taken as on a ball and beam system, which has been described in
Hauser et al. (1992).
uk ˆ Llqr 2 LD †xk 1 usvm
k 38† The continuous time system description of the ball and
 J.A.K. Suykens et al. / Neural Networks 14 (2001) 23±35 31

Fig. 2. Swinging up an inverted pendulum by a LS-SVM controller with local stabilization in its upright position. Around the target point the controller is
behaving as a LQR controller. (Top) inverted pendulum system; (Bottom) simulation result which visualizes the several pole positions in time.

beam system (Fig. 5) is given by Eq. (34) with objective we consider tracking of the ball position for a refer-
2 3 2 3 ence input d t† ˆ 2 cos pt=5†:
x2 0 As for the inverted pendulum example, a LQR design is
6 7 6 7
6 B x x2 2 G sin x † 7 607 incorporated in the LS-SVM control law. Here we impose
6 1 4 3 7 6 7
F x† ˆ 6
6
7;
7 G x† ˆ 66 7
7 43† local stability for the autonomous closed-loop system,
6 x 7 607 which is motivated by the application of NLq neural control
4 4 5 4 5
0 1 theory (Suykens et al., 1997, 1996), to the control of a real-
life ball and beam system (Verrelst et al., 1998). We proceed
where x ˆ ‰x1 ; x2 ; x3 ; x4 Š ˆ ‰r; r_; u; u_ Š with r the ball position in a similar way as for the inverted pendulum system. The
and u the beam angle and B ˆ M= Jb =R2b 1 M† where M, Jb, continuous time state space description (43) is linearized
Rb are the mass, moment of inertia and radius of the ball, around the origin, for which a LQR controller is designed
respectively. For the control input one has t ˆ 2Mrr_u_ 1 with Q ˆ I; R ˆ 1 in the LQR cost function. Then the
MGr cos u 1 Mr2 1 J 1 Jb †u where t; G; J denote the continuous time model is discretized by using a fourth
torque applied to the beam, the acceleration of gravity and order Runge±Kutta integration rule with constant step h ˆ
the moment of inertia of the beam, respectively. As control 0:3 into a discrete time model similar to Eq. (37). The ®rst
32 J.A.K. Suykens et al. / Neural Networks 14 (2001) 23±35

Fig. 3. (Continued) (Top) state variables with respect to time of the closed-loop simulation model with LS-SVM controller: x1(t) (±); x2(t) (± ±); x3 (±.); x4(t)
(:). The state vector data before the vertical line were used in the training process as support vector data with an RBF kernel; (Bottom) control signal uk.

component of the reference state vector xr1;k is derived from the autonomous closed-loop system) and
d(t) according to this step size. The control law is taken as
" # 2usvm
k
xk LD ˆ : 46†
uk ˆ Llqr 2 LD † r 1 usvm
k 44† 2xk xk ˆ0
x1;k
Note that Eq. (46) also depends on the reference xr1;k : A zero
with
" # " #! initial state has been taken. SQP was applied for constrained
X
N xl xk nonlinear optimization of Eq. (30) with a similarly chosen
usvm
k ˆ al K ; 45† initial unknown parameter vector as for the inverted pendu-
lˆ1 xr1;l xr1;k
lum example. Simulation results of the closed-loop simula-
where Llqr is the resulting feedback matrix from LQR (for tion model are shown in Fig. 5.
 J.A.K. Suykens et al. / Neural Networks 14 (2001) 23±35 33

stage optimal control problem has been formulated for a

nonlinear state feedback controller consisting of support
vector machines. Therefore a least squares version of
SVM's has been considered, leading to equality
constraints instead of inequality constraints that would
result for example from Vapnik's epsilon-insensitive
loss function. Sparseness is lost in the least squares
case but other SVM features are still applicable, such
as Mercer's condition and the fact that no centers have
to be determined for RBF kernels. However, in order to
keep the optimal control problem formulation tractable,
an equality constraint based formulation of SVM's has
been preferred, motivating the use of LS-SVM's. The
cost function for the control problem and for the least
squares SVM have been formulated within one objective
function. The solution is characterized then by a set of
nonlinear equations. A drawback of this approach is that
the problem contains many unknowns, even in the equal-
ity constraint case of LS-SVM's. Therefore an alterna-
tive formulation of LS-SVM optimal control has been
given in less unknowns. Imposing local stability for the
control law has been discussed. A main difference of LS-
SVM control with other neural and more speci®cally
RBF controllers is that in the former case the state vector
sequence of the considered time horizon is supporting
the control signal and has to be obtained as solution to
the constrained nonlinear optimization problem, while
for the latter one often solves a parametric optimization
problem in the unknown interconnection weights. The
number of hidden units of the LS-SVM with RBF kernel
in the control law equals the considered number of
points in the discrete time evolution, while in primal
weight space the number of parameters can be in®nite.
Special attention has to be paid for the closed-loop simu-
lation models of the LS-SVM controllers, whose result
could differ from the solution to the constrained optimi-
zation problem, especially in the control of unstable
systems. Examples of LS-SVM control are given on
swinging up an inverted pendulum with local stabiliza-
tion in its upright position and tracking for a ball and
beam system.

Acknowledgements

This research work was carried out at the ESAT

Fig. 4. (Continued) (Top) (x1(t), x2(t)); (Middle) (x1(t), x3(t)); (Bottom) laboratory and the Interdisciplinary Center of Neural
(x3(t), x4(t)). The initial state is marked by a square and the target state
by W.
Networks ICNN of the Katholieke Universiteit Leuven,
in the framework of the FWO project G.0262.97 Learn-
ing and Optimization: an Interdisciplinary Approach,
the Belgian Programme on Interuniversity Poles of
Attraction, initiated by the Belgian State, Prime Minis-
7. Conclusions ter's Of®ce for Science, Technology and Culture (IUAP
P4-02 and IUAP P4-24) and the Concerted Action
In this paper we introduced the use of support vector Project MIPS and MEFISTO-666 of the Flemish
machines for solving optimal control problems. The N- Community. Johan Suykens is a postdoctoral researcher
34 J.A.K. Suykens et al. / Neural Networks 14 (2001) 23±35

Fig. 5. Tracking control of a ball and beam system by a LS-SVM controller: (Top) ball and beam system; (Bottom) reference input (±) and position of the ball
(± ±). The state vector data before the vertical line were used in the training process as support vector data with an RBF kernel. Shown is the closed-loop
simulation result.

with the National Fund for Scienti®c Research FWO Ð The conditions for optimality are given by
Flanders. 8    
>
> 2L0 2h 2f T 2f 2uk T
>
> ˆ 1 lk21 2 lk 2 lk ˆ 0; k ˆ 2; ¼; N
>
> 2xk 2xk 2xk 2uk 2xk
>
>
>
> 2L0 2r
Appendix A. Formulation without error variables >
> ˆ 1 lN ˆ 0
>
< 2xN11 2xN11
>
> 2L0 X
N
2f
A variation on the problem Eqs. (17) and (18) without the >
>
> ˆ w2 lTk w xk † ˆ 0
>
> 2w 2uk
use of the variables ek is >
>
>
kˆ1
>
> 2L0
>
: ˆ xk11 2 f xk ; wT w xk †† ˆ 0; k ˆ 1; ¼; N:
minJ0 xk ; w† ˆ JN xk ; uk w; xk †† 1 1
2 wT w A1† 2l k
A3†
subject to
( Note that the support values are directly related here to the
xk11 ˆ f xk ; uk w; xk ††; k ˆ 1; ¼; N x1 given† Lagrange multipliers lk : The set of nonlinear Eq. (A3) is of
uk ˆ w T w xk † k ˆ 1; ¼; N: the form

This gives the Lagrangian F3 xk ; xN11 ; w; lk † ˆ 0: A4†

X
N Mercer condition can be applied as to Eqs. (A3) and (A4),
L0 xk ; w; lk † ˆ JN xk ; uk w; xk †† 1 1
2 wT w 1 lTk ‰xk11 yielding a set of equations of the form
kˆ1
F4 xk ; xN11 ; lk † ˆ 0 A5†
2 f xk ; wT w xk ††Š:
A2† by elimination of w.
 J.A.K. Suykens et al. / Neural Networks 14 (2001) 23±35 35

References SchoÈlkopf, B., Sung, K. -K., Burges, C., Girosi, F., Niyogi, P., Poggio, T., &
Vapnik, V. (1997). Comparing support vector machines with Gaussian
Bishop, C. M. (1995). Neural networks for pattern recognition, Oxford: kernels to radial basis function classi®ers. IEEE Transactions on Signal
Oxford University Press. Processing, 45 (11), 2758±2765.
Boyd, S., & Barratt, C. (1991). Linear controller design, limits of perfor- SchoÈlkopf, B., Burges, C. J. C., & Smola, A. J. (1999). Advances in kernel
mance, Englewood Cliffs, NJ: Prentice-Hall. methods Ð Support vector learning, Cambridge, MA: MIT Press.
Bryson, A. E., & Ho, Y. C. (1969). Applied optimal control, Waltham, MA: Sezer, M. E., & Siljak, D. D. (1988). Robust stability of discrete systems.
Blaisdel. International Journal of Control, 48 (5), 2055±2063.
Cherkassky, V., & Mulier, F. (1998). Learning from data: concepts, theory Smola, A. (1999). Learning with Kernels. PhD thesis, published by Birlin-
and methods, New York: Wiley. ghoven: GMD.
Fletcher, R. (1987). Practical methods of optimization, New York: Wiley. Smola, A., & SchoÈlkopf, B. (1998). On a kernel-based method for pattern
Franklin, G. F., Powell, J. D., & Workman, M. L. (1990). Digital control of recognition, regression, approximation and operator inversion. Algor-
dynamic systems, Reading, MA: Addison-Wesley. ithmica, 22, 211±231.
Gill, P. E., Murray, W., & Wright, M. H. (1981). Practical optimization, Smola, A., SchoÈlkopf, B., & MuÈller, K. -R. (1998). The connection between
London: Academic Press. regularization operators and support vector kernels. Neural Networks,
Golub, G. H., & Van Loan, C. F. (1989). Matrix computations, Baltimore, 11 (4), 637±649.
MD: Johns Hopkins University Press. Suykens, J. A. K., & Vandewalle, J. (1999a). Training multilayer percep-
Hauser, J., Sastry, S., & Kokotovic, P. (1992). Nonlinear control via tron classi®ers based on a modi®ed support vector method. IEEE Trans-
approximate input±output linearization: the ball and beam example. actions on Neural Networks, 10 (4), 907±912.
IEEE Transactions on Automatic Control, 37 (3), 392±398. Suykens, J. A. K., & Vandewalle, J. (1999b). Least squares support vector
Haykin, S. (1994). Neural Networks: a Comprehensive Foundation, (2nd machine classi®ers. Neural Processing Letters, 9 (3), 293±300.
ed.). Englewood Cliffs: Macmillan. Suykens, J. A. K., De Moor, B., & Vandewalle, J. (1994). Static and
Ljung, L. (1987). System Identi®cation: Theory for the User, New York: dynamic stabilizing neural controllers, applicable to transition between
Prentice-Hall. equilibrium points. Neural Networks, 7 (5), 819±831.
MuÈller, K. R., Smola, A. J., RaÈtsch, G., SchoÈlkopf, B., Kohlmorgen, J., & Suykens, J. A. K., De Moor, B., & Vandewalle, J. (1997). NLq theory: a
Vapnik, V. (1997). Predicting time series with support vector machines. neural control framework with global asymptotic stability criteria.
In W. Gerstner, A. Germond, M. Hasler & J. -D. Nicoud, Proceedings Neural Networks, 10 (4), 615±637.
of the International Conference on Arti®cial Neural Networks Suykens, J. A. K., Lukas, L., & Vandewalle, J. (2000). Sparse approxima-
ICANN'97LNCS 1327 (pp. 999±1004). Berlin: Spinger. tion using least squares support vector machines. IEEE International
Narendra, K. S., & Mukhopadhyay, S. (1997). Adaptive control using Symposium on Circuits and Systems ISCAS 2000, Geneva, Switzerland,
neural networks and approximate models. IEEE Transactions on Neural May 28±31 (pp. II-757±760).
Networks, 8 (3), 475±485. Suykens, J. A. K., Lukas, L., & Vandewalle, J. (2000). Sparse Least
Narendra, K. S., & Parthasarathy, K. (1991). Gradient methods for the Squares Support Vector Machine Classi®ers. 8th European Sympo-
optimization of dynamical systems containing neural networks. IEEE sium on Arti®cial Neural Networks ESANN 2000, Bruges Belgium,
Transactions on Neural Networks, 2 (2), 252±262. (pp. 37±42).
Nguyen, D., & Widrow, B. (1990). Neural networks for self-learning Suykens, J. A. K., Vandewalle, J., & De Moor, B. (1996). Arti®cial neural
control systems. IEEE Control Systems Magazine, 10 (3), 18±23. networks for modelling and control of non-linear systems, Boston:
Parisini, T., & Zoppoli, R. (1994). Neural networks for feedback feedfor- Kluwer Academic.
ward nonlinear control systems. IEEE Transactions on Neural Vapnik, V. (1995). The nature of statistical learning theory, New York:
Networks, 5 (3), 436±449. Springer.
Plumer, E. S. (1996). Optimal control of terminal processes using neural Vapnik, K. (1998a). Statistical learning theory, New York: Wiley.
networks. IEEE Transactions on Neural Networks, 7 (2), 408±418. Vapnik, V. (1998b). The support vector method of function estimation. In J.
Poggio, T., & Girosi, F. (1990). Networks for approximation and learning. A. K. Suykens & J. Vandewalle, Nonlinear modeling: advanced black-
Proceedings of the IEEE, 78 (9), 1481±1497. box techniques (pp. 55±85). Boston: Kluwer Academic.
Saerens, M., Renders, J. -M., & Bersini, H. (1993). Neural controllers based on Vapnik, V., Golowich, S., & Smola, A. (1997). Support vector method for
backpropagation algorithm. In M. M. Gupta & N. K. Sinha, IEEE Press function approximation, regression estimation and signal processing.
book on intelligent control: theory and practic New York: IEEE Press. Advances in Neural Information Processing Systems, Vol. 9.
Sanner, R. M., & Slotine, J. -J. E. (1992). Gaussian networks for direct Cambridge, MA: MIT Press.
adaptive control. IEEE Transactions on Neural Networks, 3 (6), 837± Verrelst, H., Van Acker, K., Suykens, J., Motmans, B., De Moor, B., &
863. Vandewalle, J. (1998). Application of NLq neural control theory to a
Saunders, C., Gammerman, A., & Vovk, V. (1998). Ridge regression learning ball and beam system. European Journal of Control, 4, 148±157.
algorithm in dual variables. Proceedings of the 15th International Confer- Werbos, P. (1990). Backpropagation through time: what it does and how to
ence on Machine Learning ICML-98, Madison, WI (pp. 515±521). do it. Proceedings of the IEEE, 78 (10), 1150±1560.