Neurocontrol of Nonlinear Dynamic Systems Subject to

Unmeasured Disturbance Inputs

Ressom Habtom and Lothar Litz

Institute for Process Automation, University of Kaiserslautem

PO Box 3049, 67653 Kaiserslautern, Germany
[habtom,litz] @e-technik.uni-kl.de

Abstract. This paper presents a neural network-based approach for estimation of

unmeasured disturbance inputs, modeling and control of nonlinear dynamic
systems. Some inputs of a dynamic system are assumed to be measurable all the
time, while others can only deliver training data for a certain period of time. The
unmeasured disturbance inputs are estimated on-line based on a recurrent neural
network model of the system and using the extended Kalman filter (EKF).
Furthermore, the training of a recurrent neurocontroller is carried out on the basis
of the neural model of the system. In addition to the measurable input and output
variables of the system, the neurocontroller makes use of the estimated
unmeasured disturbance inputs to calculate its control signals. A mathematical
model of a drying drum is employed to demonstrate the proposed approach.

1. Introduction
Due to their strong representational and mapping capabilities, recurrent neural
networks have been widely employed for the modeling and control of nonlinear
dynamic systems. Recently, there has been another dimension of interest in exploring
recurrent networks for state estimation. This is due to the feature that appropriately
structured recurrent networks have the potential to estimate not only the output of the
plant but also the internal states of the plant.
The Extended Kalman Filter (EKF), which is an optimal state and parameter
estimation tool, has been applied for the training of neural networks. The use of EKF
as a recurrent network learning algorithm is elaborated in [1] and its application for
the training of a neural controller is discussed in [2]. In our work [3], a method is
proposed, where the EKF and the recurrent neural model of the unknown system are
employed for the estimation of unmeasured inputs. In this paper, we apply this
approach to estimate unmeasured disturbance inputs in a closed-loop. An off-line
trained neurocontroller employs the estimated disturbance inputs and other measured
variables to control a dynamic system subject to unmeasured disturbance inputs.

2. Modeling of Dynamic Systems with a Recurrent Neural Network

A nonlinear dynamic system can be modeled by using a neural network containing
local recurrence as well as feedback connections among the units of a layer. Such
layers of units arranged in a feedforward fashion form an architecture described as
recurrent multilayer perceptron (RMLP) network in [4]. These kinds of networks have
the effect of modeling plants, in which the plant's nonlinear model is assumed to be
described by a set of difference equations, where the plant's states are coupled non-
linearly with the imposed controls. An RMLP having a single hidden layer as parallel
 856

identification model for a nonlinear dynamic system is depicted in Fig. 1. The RMLP
network in this figure can be represented mathematically as
x(k + 1) = f(WYy(k) + WUu(k) + Whx(k) + b u) (1)

y(k + 1) = g(WXx(k + I) + W°y(k) + b x) (2)

where x(k) and y(k) denote the output vector of the hidden units and the output vector
of the network at time k, respectively; W y and W e represent the weight matrices
connecting the hidden units to the previous output of the network and the input vector
u, respectively; W x is the weight between the hidden units and the output units; W h
and W ° are weight matrices representing the local and lateral connections for the
hidden units and the output units, respectively; and b e, b x are the input and output
biases, respectively. The vectors of activation functions for the hidden units and the
output units are denoted as f(.) and g(.), respectively.

u(k)

Fig. 1 Parallel identification model with an RMLP network.

A dynamic backpropagation (DBP) algorithm based on the framework established
in [5] and [6] can be applied to train the RMLP network. Both the internal and
external recurrences have to be considered in the formulation of the sensitivity model.
Depending on the application, batch or pattern learning approach could be used for
training. A batch learning can be effected by using the Levenberg-Marquardt
algorithm after having established the Jacobian matrix, which represents the partial
derivative of each output with respect to each weight for each pattern using the DBP
algorithm. This approach was employed in [7] to train a recurrent network. The EKF
algorithm, which is well suited for on-line applications, can be efficiently adopted for
pattern learning. Using the EKF algorithm, we can train the network by updating
either only the weights of the network or both the weights and the states of the
network. The latter is referred to as parallel EKF training, [2]. To use this training
algorithm, we ought to formulate a state space representation of the identification
network and augment the resulting state space model by the weight vector w, which
denotes the network's weight. Rewriting eqs. (1) and (2) and augmenting by w, we get
x ( k + 1) = f ( W YxY(k) + W e u ( k ) + W h x ( k ) + b u ) (3)
x Y ( k + l ) = g ( W X f ( . ) + W ° x Y ( k ) + b x) (4)
w(k + 1) = w ( k ) (5)
y ( k ) = xY(k) (6)
 857

where x y represents the output vector of the output units at time k. To use the discrete
EKF algorithm, the linearization of the states should be carried out on every step of
training and we have to associate plant and measurement noises.

3. Estimation of Unmeasured Disturbance Inputs Using the EKF

Assuming that we have already obtained a neural model of our dynamic system but we
can not measure some of the our inputs, which we consider as disturbance inputs, we
apply the EKF algorithm to estimate the unmeasured inputs on-line, [3]. An
augmented state space representation of the neural network with the unmeasured
disturbance inputs as additional states needs to be constructed, so that the EKF
algorithm could be employed. A common method to incorporate additional states is to
assume the states as constant plus noise.
Considering the recurrent network stated in Section 2, we formulate the augmented
state model in the following. In order to distinguish the unmeasured disturbance inputs
from the measurable inputs, we split the vector u into two vectors u m and z, which
represent measurable and unmeasured inputs, respectively. Let W am and W z be the
weights (extracted from the matrix W u) connecting the hidden units with Um and z,
respectively. Now, including the vector z as an additional state in the state model of
eqs. 3, 4 and 6, we get an augmented state model of the following form,
x(k + 1) = f(WYxY(k) + Wumum (k) + WZz(k) + W h x ( k ) + b U ) + w l ( k ) (7)

X y ( k q- 1) = g(W xf(.) + W O x y ( k ) + b x) + w a (k) (8)

z ( k + 1) = z ( k ) + w 3 ( k ) (9)

y(k) = xY(k) + v ( k ) (10)

where wl(k), w2(k), w3(k) are vectors of plant noise and v(k) is that of measurement
noise, at time k. Now, the discrete EKF can be applied to estimate the unmeasured
input vector z together with the states of the network (x and xY). The estimation is
based on the error between the output of the plant and the identification network with
the weights of the identification being kept unchanged.

4.Neurocontroller for a Plant with Unmeasured Disturbance Inputs

In general, the purpose of the neurocontroller is to drive the unknown dynamic system
in such a way that the error between the output of the system and the desired out is
minimized. Assuming that a neural model of the unknown system is already obtained
using the method stated in Section 2, a neural controller can be trained based on this
model. As shown in Fig. 2, the training of a neural controller (NC) can be carried out
by backpropagating the error between the output of identification network (y) and that
of the reference model (Yd) through the identification network (NI) such that the
cascade of NC and NI mimic the reference model. In Fig. 2, we assume that both NC
and NI use the same RMLP architecture with a single hidden layer. The previous
output of NI and the disturbance input vector ud are fed to both NI and NC. While the
reference input vector (r) is applied to the controller, the identification network is
driven by the current control signal uc, which is generated from the neurocontroller.
 858

For the purpose of training the neurocontroller, the reference vector and r and the
disturbance input vector Ud are generated randomly. The dynamic backpropagation
(DBP) algorithm involves a sensitivity model considering both the internal and
external recurrences. An off-line training can be carried out by using an efficient
training algorithm like the Levenberg-Marquardt method. If an on-line adaptation of
the controller is demanded, then the EKF algorithm can be preferably applied [2].

Fig. 2 Training a neural controller.

In the closed operation, we implement the off-line trained neurocontroller to drive

the unknown dynamic system. However, we assume that some of the elements of the
input disturbance vector ua can not be measured on-line. As a result, we can not use
our controller unless we have a mechanism to compensate the effect of the
unmeasured disturbance inputs. One approach is, of course, to ignore the unmeasured
disturbance inputs during the training of the controller and adapt the parameters of the
controller to overcome the disturbance caused by these inputs. An approach that we
propose here, is to incorporate the method stated in Section 3 for the estimation of the
unmeasured disturbance inputs. We feed the estimated signal to the neurocontroller.
As illustrated in Fig. 3, the disturbance input vector is estimated using the EKF
algorithm based on the error between the output of the identification model y and that
of the plant yp. If only some of the disturbance inputs could not be measured on-line,
then only those unmeasured inputs will be estimated while the measured inputs will be
directly applied to the controller and identification network (this case is not indicated
in the Fig. 3). The error between the plant output yp and that of the reference model Ya
can be used for an on-line adaptation of the parameters of the neurocontroller.

rlk> ,I ,--,XZ"~V,-O/t \~-31 u~<~l j :x,~t ~ I l yik+I)

~1 +

- < _ ~ -'--~e(k+l)

"I'a ReferenceModel I YdCk+')

Fig. 3 The overall structure of the neurocontroller for a plant subject to unmeasured
disturbance inputs.
 859

5. S i m u l a t i o n Results : A Drying Process

In the course of extracting sugar from roots of sugar beet, there will be scraps of beet
and molasses obtained as byproducts. The purpose of the drying process is to increase
the percentage of the dry substance contained in the byproducts, so that they could be
conserved until they are taken up by the market. We consider a mathematical model of
a drum, which involves high temperature drying process, as our plant. A distinct
feature of this model is that it contains nonlinear functions, high order linear dynamic
transfer functions, transport delays and that it is non minimum phase.
The process variables, which are of interest to us are: the input oil quantity (ul),
which is fed to a burner at the inlet of the drum; the input feed flow rate (u2); the dry
substance content (u3) at the inlet of the drum; and the dry substance content at the
outlet of the drum (y), which is controlled through the oil quantity to achieve a dry
substance content of about 91%. Hence, under closed loop operation, ul is the control
signal, while u2 and u3 act as disturbances. Furthermore, u2 is assumed to deliver
training data for a certain period of time but can not be measured on-line. Hence, it
acts as unmeasured disturbance input during the closed-loop operation.
Based on training data generated from the mathematical model, an RMLP with a
single hidden layer having 10 units was trained to predict the output y from the inputs
ul, u2, and u3. The result of training for test data is shown in Fig. 4. After having
obtained the RMLP model of the drum, an RMLP with a single hidden layer of 10
units was trained as a controller using the structure depicted in Fig. 2. In this figure,
the disturbance input vector ua denotes the inputs u2 and u3 of the drum, and uc
represents ul of the drum. The reference input and the disturbance inputs (u2 and u3)
were generated randomly during training. After the completion of the training, the
performance of the neurocontroller is tested using a reference input different form the
training reference input. This result is depicted in Fig. 5.
Ioo

94 I

95
93 1

90
9t

90
85

89
RM LP (y) ~ plant's output (yp)

80 20; 40;
~0; SO; looo s s ,oo ,ooo l,oo
[timin.] [ t / m in.]
Fig. 4 Measured and predicted values of y. Fig. 5 Performance of the neurocontroller.
Finally, the neurocontroller is implemented in the overall control structure shown
in Fig. 3. Now, the unmeasured disturbance input (u2) of the drum is estimated using
the EKF algorithm, while the other disturbance input (Ul) is applied to both the
identification and controller networks. In Fig. 6, the input u2, which is applied to the
mathematical model, and the estimated unmeasured input z are shown. Fig. 7 shows
the tracking ability of the neurocontroller, when the reference input changes.
 860

48 95

% %
94
46

44

92 i
42
91

4,
90

38 ~ ~ reference(r)
-- measured(u 2) 89
plant's output (yp)
estimated (z)
36 I i i
20'° ,0; ,.o 800 1000 ~8 20o 400 600 800 lOO~l

[timin.] [t/rain.]
Fig. 6 Measured and estimated values of u2. Fig. 7 The controller tracking the reference.

6. C o n c l u s i o n s
The simulation results have demonstrated the effectiveness of the proposed
approach not only in estimating the unmeasured disturbance input of the drying drum
but also in driving the unknown system to the desired output. The fact that the inherent
characteristics of the mathematical model is complex and that we employed two
strongly varying disturbance inputs of which one is estimated, makes the control
aspect rather difficult. In view of this, we believe that the performance of the neural
controller is quite satisfactory. Even with the assumption that we could measure all the
disturbance inputs, the control aspect is not easy. As already pointed out in Section 4,
the neurocontroller can be theoretically adapted on-line with the aim of minimizing
the error between the output of the plant and that of the reference model by applying
the identification model as a channel for backpropagation. On this regard, a further
study will be carried out on the perspective of combining our approach to an adaptive
control law to improve the overall performance.

References
1. R.J. Williams. Training recurrent networks using the extended Kalman filter, in Proc. of
the International Joint Conference on Neural Networks, Baltimore 1992, vol. IV, pp. 241-
246.
2. G. V. Puskorius and L. A. Feldkamp. Neurocontrol of nonlinear dynamical systems with
Kalman filter trained recurrent networks, IEEE Trans. on Neural Networks, vol. 5, pp. 279-
297, 1994.
3. R. Habtom and L. Litz. Estimation of unmeasured inputs using recurrent neural networks
and the extended Kalman filter, in Proc. of International Conference on Neural Networks,
Houston 1997, vol. IV, pp. 2067-2071.
4. B. Fernandez, A. G. Parlos, and W. K. Tsai, Nonlinear dynamic system identification using
artificial neural networks (ANNs), in Proc. of International Joint Conference on Neural
Networks, San Diego 1990, vol. II, pp. 131-141.
5. K.S. Narendra and K. Parthasarathy. Identification and control of dynamical systems using
neural networks, IEEE Trans. on Neural Networks, vol. 1, pp. 4-27, 1990.
6. R.J. Williams and D. Zipser. A learning algorithm for continually running l)ally recurrent
neural networks, Neural Computation, vol. 1, pp. 270-280, 1989.
7. R. Habtom and L. Litz. Neural network-based control of nonlinear dynamic systems with
unmeasured inputs, in Proc. of Twelfth International Conference on Systems Engineering,
Coventry, UK, 1997.