Engineering Applications of Artificial Intelligence 101 (2021) 104195

Contents lists available at ScienceDirect

Engineering Applications of Artificial Intelligence

journal homepage: www.elsevier.com/locate/engappai

State–space modeling for control based on physics-informed neural networks

Florian Arnold ∗, Rudibert King
Technische Universität Berlin, Department of Process Engineering, Chair of Measurement and Control, Strasse des 17. Juni 135, 10623 Berlin, Germany

ARTICLE INFO ABSTRACT

Keywords: Dynamic system models, based on partial differential equations (PDEs), are often unsuitable for direct use in
Machine learning control or state estimation purposes, due to the high computational cost arising from the necessity to apply
State–space sophisticated numerical methods for a solution, such as semi-discretization, also known as spatial discretization.
Neural networks
Hence, there is often an inevitable trade-off between accuracy and computational efficiency during the model
Model predictive control
reduction step to ensure real-time applicability.
State estimation
In this contribution, we propose a state–space model formulation, using so-called physics-informed neural
networks. This modeling approach enables a highly efficient inclusion of complex physical system descriptions
within the design of control or state estimation setups. The resulting state–space model does not require any
numerical solution techniques during the state propagation, as each time step is based on the evaluation of
a reasonably sized neural net that approximates the solution of the PDE. Thus, this approach is suitable for
real-time applications of various complex dynamic systems that can be described by one or a set of PDEs.
Besides the modeling approach itself, the contribution also provides an illustrative example of the state–
space modeling method in the context of model predictive control, as well as state estimation with an extended
Kalman filter. These methods will be applied to a system based on a numerical solution of the Burgers equation.

1. Introduction a high numerical burden. Especially recurrent neural networks (Connor

et al., 1994) were considered for time series prediction early. Variations
With accurate process models, control approaches like model pre- of this network topology, like the so-called Long Short Term Memory
dictive control (MPC) or iterative learning control provide superb (LSTM) recurrent networks are now widely used for different applica-
control performance. Likewise, precise modeling enables model-based tions, as described in Sagheer and Kotb (2019), or Jiang et al. (2020).
estimation methods like Kalman filtering. Usually, modeling complex
The potential that machine learning approaches can have for control
dynamic systems leads to a set of ordinary or partial differential equa-
purposes has been investigated (Hunt et al., 1992), even though the
tions. The solution to the latter can be especially challenging as it
might require costly numerical methods. Even though optimal control computational cost of such methods limited the applicability likewise.
approaches for complex PDEs like the Navier–Stokes equation (Desai A set of data-driven approaches primarily focused on flow control is
and Ito, 1994; Gunzburger and Manservisi, 2000) exist, the computa- given in Brunton et al. (2020). However, with increasing computational
tional burden limits its applicability for real processes. For example, resources, applications like an MPC setup with a machine-learned
the computational cost to calculate a numerical PDE solution over model became feasible (Gillespie et al., 2018). Also, a combination of
the prediction horizon of an MPC, or within the time update of a data-driven methods and conventional state–space modeling has been
Kalman filter can prohibit a real-time application of such methods even investigated, for example, in Rangapuram et al. (2018) and Hauser
for slow processes. Therefore, a trade-off between model complexity et al. (2019). The main concern for these approaches, especially re-
and accuracy is necessary to ensuring the applicability of the control garding control and state estimation problems, is the dependency the
or state estimation setup for the considered physical system. Classi-
neural nets have on the used training data. Small or insufficiently
cally, model reduction techniques are applied to the semi-discretized
sampled training data might result in a trained neural network with
system, such as moment matching approximation and balanced trun-
cation for linear systems, or proper orthogonal decomposition, see Bai poor generalization capability, which prevents predictable behavior
(2002), Mehrmann and Stykel (2005), Gräßle et al. (2019), Noack et al. when used, for example, in a control setup.
(2003), and references therein. In this contribution, we present a new approach to generate state–
Alternative modeling approaches from the field of machine learning space models for control and state estimation purposes of dynamic
have been investigated in the past decades, to overcome the problem of systems that are described by PDEs. The method utilizes so-called

∗ Corresponding author.
E-mail addresses: [email protected] (F. Arnold), [email protected] (R. King).

https://doi.org/10.1016/j.engappai.2021.104195
Received 21 July 2020; Received in revised form 1 February 2021; Accepted 6 February 2021
Available online 4 March 2021
0952-1976/© 2021 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license
(http://creativecommons.org/licenses/by-nc-nd/4.0/).
F. Arnold and R. King Engineering Applications of Artificial Intelligence 101 (2021) 104195

physics-informed neural networks (PINNs) as introduced in Raissi et al.

(2017a,b). PINNs can be used to approximate the solution of PDEs
by including the PDE itself in the training process of a feed-forward
neural network and, by that, forcing a similar behavior for the partial
derivatives of the trained neural network and the PDE solution. The
results obtained by Raissi et al. indicate that the approximation of
the PDE solution with the PINN approach is highly accurate, even
with small sets of training data and reasonably sized neural networks.
However, the PINN method, as presented in the literature, is only Fig. 1. Space–time grid points of the input data ℎ and 𝑔 for a small set of samples.
suitable for autonomous PDEs in a bounded space–time domain. Thus,
the main focus of this work is the formulation of a state–space model
based on the PINN approach that enables control synthesis and state ( )
which defines a grid of collocation points 𝒑𝑔,𝑖 , 𝑡𝑔,𝑖 where the PDE
estimation for dynamic systems that are modeled initially by PDEs. approximation from Eq. (3) should be valid:
The organization of this contribution is as follows: We start with ( ) [ ( )]
a brief review of the concept of PINN in Section 2. In Section 3, 𝛹 𝑡 𝒑𝑔,𝑖 , 𝑡𝑔,𝑖 +  𝛹 𝒑𝑔,𝑖 , 𝑡𝑔,𝑖 ≈ 0 (7)
we introduce a method to construct a PINN-based state–space model ( )
For the collocation points 𝒑𝑔,𝑖 , 𝑡𝑔,𝑖 , additional labels similar to Eq. (5)
that is suitable for control and state estimation applications. Also, we
are redundant, as the desired behavior of the neural network 𝛹 is
assess the properties and benefits of the presented state–space model.
defined by the PDE that relates the partial derivatives of the net, for
In Section 4, an MPC example is presented for the Burgers equation.
which appropriate methods of determination are necessary, see below.
Additionally, we discuss a corresponding state estimation example,
A specific loss function can be assigned to each group of training
using an Extended Kalman filter. Finally, conclusions are drawn in
data. For the first group, the fit with respect to input data ℎ and
Section 5.
corresponding labels ℎ can be evaluated by the mean squared error
2. Physics informed neural networks (MSE)
𝑛ℎ
( ) 1 ∑[ ( ) ( )]2
The concept of PINN has been introduced in Raissi et al. (2017a,b). 𝐿ℎ ℎ , ℎ , 𝛹 = ℎ 𝒑ℎ,𝑖 , 𝑡ℎ,𝑖 − 𝛹 𝒑ℎ,𝑖 , 𝑡ℎ,𝑖 . (8)
𝑛ℎ 𝑖=1
Further investigations for different applications, including systems with
vortices and shock waves, are provided in Raissi (2018), Raissi et al. Accordingly, the loss
(2019a,b). This section will briefly review the idea of PINN and 𝑛
( ) 1 ∑[ 𝑡 ( ) [ ( )]]2
𝑔
introduce the notation used in this contribution. The basic concept 𝐿𝑔 𝑔 , 𝛹 = 𝛹 𝒑𝑔,𝑖 , 𝑡𝑔,𝑖 +  𝛹 𝒑𝑔,𝑖 , 𝑡𝑔,𝑖 (9)
𝑛𝑔 𝑖=1
of physics-informed neural networks is the approximation of a PDE
solution ℎ at position 𝒑 and time 𝑡 with a neural network 𝛹 , so ( )
quantifies the fit for the PDE collocation points 𝒑𝑔,𝑖 , 𝑡𝑔,𝑖 , that is, when
that 𝐿𝑔 = 0, the PDE is fulfilled exactly at all collocation points. As the
ℎ (𝒑, 𝑡) ≈ 𝛹 (𝒑, 𝑡) (1) neural network 𝛹 is part of both loss functions in Eqs. (8) and (9), the
sum
is valid for 0 ≤ 𝑡 ≤ 𝑡ub and, for example, a box-constraint for the ( ) ( ) ( )
space domain, 𝒑lb ≤ 𝒑 ≤ 𝒑ub . We consider a scalar problem here for 𝐿 ℎ , ℎ , 𝑔 , 𝛹 = 𝐿ℎ ℎ , ℎ , 𝛹 + 𝐿𝑔 𝑔 , 𝛹 (10)
simplicity, that is, ℎ ∈ R. Vectors, in contrast to scalars, are printed in can be used within the training of the neural network 𝛹 to ensure a
bold letters, for example, 𝒑 ∈ R𝑛𝑝 , where 𝑛𝑝 is the spatial dimension.
fit with respect to Eq. (1), and Eq. (3). By that, the training process
We assume that a PDE
preserves the dynamic behavior defined by the PDE instead of merely
ℎ𝑡 +  [ℎ] = 0 (2) forcing the best possible fit for plain data points. Moreover, when ℎ
consists of only initial and boundary conditions, no solution of the PDE
can describe the system dynamics. In this notation, ℎ𝑡
indicates the is necessary for training the net.
partial time derivative, and  is an arbitrary differential operator. As The definition and training of all neural nets used here is done with
the neural network 𝛹 approximates the PDE solution as described in
TensorFlow (Abadi et al., 2015). TensorFlow features automatic differen-
Eq. (1),
tiation, that is, the net 𝛹 can be easily differentiated with respect to all
𝛹 𝑡 +  [𝛹 ] ≈ 0 (3) inputs 𝑡 and 𝒑 to determine the expressions in Eq. (9). For training, the
L-BFGS-B algorithm is exploited. We used the default parameter setting
should hold for the neural network, as well. Since Eqs. (1) and (3) of the SciPy implementation of the L-BFGS-B algorithm, see Virtanen
define two separate requirements for the neural network 𝛹 , the data et al. (2020).
for the training of 𝛹 can be divided into two groups accordingly. The
first group contains the training input data with the space–time grid 2.1. Example: Burgers equation
{( ) ( ) ( )}
ℎ = 𝒑ℎ,1 , 𝑡ℎ,1 , … , 𝒑ℎ,𝑖 , 𝑡ℎ,𝑖 , … , 𝒑ℎ,𝑛ℎ , 𝑡ℎ,𝑛ℎ (4)
Below, we will shortly walk through the setup of a PINN for the one-
and the corresponding labels dimensional viscous Burgers equation, similar to Raissi et al. (2019a).
{ ( ) ( ) ( )} We extend this example throughout this contribution to illustrate the
ℎ = ℎ 𝒑ℎ,1 , 𝑡ℎ,1 , … , ℎ 𝒑ℎ,𝑖 , 𝑡ℎ,𝑖 , … , ℎ 𝒑ℎ,𝑛ℎ , 𝑡ℎ,𝑛ℎ . (5)
application of PINN for control and state estimation purposes. For
( )
These 𝑛ℎ training sets define a more intuitive reading, we will switch the notation to the scalar
( points ) 𝒑ℎ,𝑖 , 𝑡ℎ,𝑖 in the space–time domain,
at which the solution ℎ 𝒑ℎ,𝑖 , 𝑡ℎ,𝑖 of the PDE is known. We assume position 𝑝 instead of 𝒑.
that at least the PDE solution at the bounds of the space–time domain The viscous Burgers equation
is (available,
) that
( is, initial
) condition ℎ (𝒑, 0) and boundary conditions
ℎ𝑡 + ℎ ⋅ ℎ𝑝 − 𝜆ℎ𝑝𝑝 = 0 (11)
ℎ 𝒑lb , 𝑡 and ℎ 𝒑ub , 𝑡 are predefined. Fig. 1 exemplifies the situation
with a small set of points for a one-dimensional problem. describes a one-dimensional flow with friction in the 𝑝-direction, where
The second group of training data relates to the PDE formulation of ℎ𝑝𝑝 = 𝜕 2 ℎ∕𝜕𝑝2 . The solution ℎ (𝑝, 𝑡) presents the flow velocity at position
the neural network 𝛹 in Eq. (3) and only consists of input data 𝑝 and time 𝑡. In this setup, we consider an initial condition
{( ) ( ) ( )}
𝑔 = 𝒑𝑔,1 , 𝑡𝑔,1 , … , 𝒑𝑔,𝑖 , 𝑡𝑔,𝑖 , … , 𝒑𝑔,𝑛𝑔 , 𝑡𝑔,𝑛𝑔 , (6) ℎ (𝑝, 0) = − sin (𝜋𝑝) (12)

2
F. Arnold and R. King Engineering Applications of Artificial Intelligence 101 (2021) 104195

and boundary conditions system, it is desirable to assign physical meaning to the state 𝒙𝑘 , as
( ) well. As the PDE solution describes the course of a physical quantity
ℎ 𝑝lb , 𝑡 = ℎ (−1, 𝑡) = 0 (13)
in the space–time plane, we propose to define the state based on the
( )
ℎ 𝑝ub , 𝑡 = ℎ (1, 𝑡) = 0. (14) PDE solution at specific spatial points as a function of time. This has
to be true already for 𝑡 = 0. Thus, the initial state 𝒙0 has to represent
These settings are similar to the example presented in Raissi et al.
the initial condition ℎ (𝒑, 0), but with a limited number of states and
(2019a). However, since we compare the results in this work with
not a full discretization of the space domain. Moreover, a network
numerical simulations instead of the analytical solution, we choose a
representing 𝒙0 must allow for the calculation of the state at a future
larger kinematic viscosity 𝜆 = 0.02 to obtain a less stiff system. The
time, that is, it would be desirable to find a net 𝛹0 = 𝛹0 (𝒑, 𝑡) with
viscosity is only adapted to avoid numerical problems with the finite
𝛹0 (𝒑, 0) = 𝒙0 that could be interrogated at a future time 𝑡1 to calculate
volume code chosen for solving the PDE. The results in Raissi et al.
𝒙1 = 𝛹0 (𝒑, 𝑡1 ). For this, a state of fixed size 𝑛𝑥
(2019a, 2017a) indicate that the PINN approach is also suitable for
[ ( ) ( ) ( )]T
very stiff systems with shock-like phenomena. Nevertheless, it should 𝒙0 = ℎ 𝒑𝑥,1 , 0 , … , ℎ 𝒑𝑥,𝑖 , 0 , … , ℎ 𝒑𝑥,𝑛𝑥 , 0 (17)
be noted that a variation in parameters like 𝜆 always requires a new
( )
training of the net. that contains the initial conditions ℎ 𝒑𝑥,𝑖 , 0 at positions
For the training of the PINN, we used 1203 values in ℎ and ℎ { }
which define initial and boundary conditions. The initial condition  = 𝒑𝑥,1 , … , 𝒑𝑥,𝑖 , … , 𝒑𝑥,𝑛𝑥 (18)
is incorporated with 201 values at equidistant positions 𝑝, and each
is considered. The actual continuous initial condition can then be
boundary condition is represented by 501 values at equidistant times
approximated with an interpolation 𝐼 so that
𝑡, for 0 ≤ 𝑡 ≤ 𝑡𝑢𝑏 , with 𝑡𝑢𝑏 = 1 chosen such that a substantial part
( )
of the expected dynamic evolution is captured. The training data 𝑔 ℎ (𝒑, 0) ≈ 𝐼 , 𝒙0 , 𝒑 (19)
contains 5000 equally distributed random points (𝑝, 𝑡) in the interior
for 𝒑𝑙𝑏 ≤ 𝒑 ≤ 𝒑𝑢𝑏 . Based on this approximation and an appropriate
of the space–time domain at which the PDE should hold. However, we
inclusion of the dynamical description of the process, see below, a PINN
additionally extended 𝑔 with the points of the initial and boundary
𝛹0 could be trained for the initial condition represented by the state 𝒙0 .
conditions to ensure that the PDE also holds at the boundary of the ( )
For that, all 𝑛0 training inputs 𝒑ℎ,𝑖 , 𝑡ℎ,𝑖 in ℎ , with 𝑡ℎ,𝑖 = 0 are moved
space–time domain, compare with Fig. 1. Consequently, a total of 7406
to a separate set of training data
training samples are used. Similar to Raissi et al. (2017a), the PINN
{( ) ( ) ( )}
𝛹 consists of 8 hidden layers of 20 neurons, an additional input layer 0 = 𝒑0,1 , 0 , … , 𝒑0,𝑖 , 0 , … , 𝒑0,𝑛0 , 0 , (20)
of two neurons corresponding to the inputs 𝑝 and 𝑡, and an output ( )
layer with only one neuron. The input and output training sets are and all corresponding labels ℎ 𝒑ℎ,𝑖 , 0 are removed from ℎ . For the
normalized to a range [−1; 1], and the net’s weights are set using Xavier new training input data 0 defined in Eq. (20), a separate part of the
initialization. A detailed analysis of the influence of the net’s depth and loss function
width can be found in the aforementioned literature. The hyperbolic 𝑛0
( ) 1 ∑[ ( ) ( )]2
tangent defines the activation function of the neurons. Training for this 𝐿0 0 , , 𝒙0 , 𝛹0 = 𝐼 , 𝒙0 , 𝒑0,𝑖 − 𝛹0 𝒑0,𝑖 , 0 (21)
𝑛0 𝑖=1
setting takes about one minute on an appropriate graphics processing
unit (GPU) like an Nvidia Titan. These settings for the training gave can be formulated that only reflects the interpolated initial condition.
( )
similar good results for various examples with different initial and In Eq. (21), the interpolation 𝐼 , 𝒙0 , 𝒑 or the real initial condition
boundary conditions. ℎ (𝒑, 0) can be used. To respect the dynamics of the system as well, the
The left-hand plot in Fig. 2 shows an approximation of the described loss function is extended by the terms 𝐿ℎ and 𝐿𝑔 as in the last chapter
Burgers equation, with the PINN 𝛹 in the space–time plane. For the ( ) ( )
𝐿 ℎ , ℎ , 𝑔 , , 𝒙0 , 0 , 𝛹0 =𝐿0 0 , , 𝒙0 , 𝛹0
visualization, the network is evaluated on a space–time grid with dis- ( )
cretization steps 𝛥𝑝 = 0.01 and 𝛥𝑡 = 0.002. For comparison, we calculate +𝐿ℎ ℎ , ℎ , 𝛹0
a numerical solution on the same space–time grid using a finite volume ( )
+𝐿𝑔 𝑔 , 𝛹0 . (22)
approach. The results show a similar quality to the results presented
in Raissi et al. (2019a, 2017a). To illustrate this, a comparison with The loss function in Eq. (22) can now be applied for training with
the numerical solution is shown in Fig. 3 for 𝑡 = 0, 𝑡 = 0.5, and 𝑡 = 1. respect to an initial condition represented by the state 𝒙0 and the
In these plots, the numerical solution and the PINN approximation are remaining boundary conditions as well as the collocation points similar
almost indistinguishable, which indicates a superb fit of the trained to Eq. (10). If the time step 𝛥𝑡 = 𝑡1 −𝑡0 = 𝑡1 is smaller than the maximum
network, especially since the inner points of the space–time domain time 𝛥𝑡 ≤ 𝑡ub for which the trained PINN 𝛹0 is valid, the state
were only trained with the PDE information. [ ( ) ( ) ( )]T
𝒙1 = 𝛹0 𝒑𝑥,1 , 𝛥𝑡 , … , 𝛹0 𝒑𝑥,𝑖 , 𝛥𝑡 , … , 𝛹0 𝒑𝑥,𝑛𝑥 , 𝛥𝑡 (23)
3. State–space modeling
with 𝒑𝑥,𝑖 ∈  could be calculated. From this state, a new neural network
𝛹1 could be trained again using the loss function from Eq. (22) with 𝒙1
The approach reviewed in the previous section is designed solely for
instead of 𝒙0 . Even though this formulation defines a state propagation,
autonomous PDEs in a limited space–time domain. As discussed in the
the setup is not suitable for real-time application, since the training of
Introduction, control synthesis and state estimation methods require a
a neural network 𝛹𝑘 would be required in each time step 𝑘.
state–space model, including a representation of the actuation and of
Therefore, a different approach has to be taken. A single neural net
the measurement. In this section, we propose an example of how to
is sought that can be used for different initial conditions to predict the
derive a discrete-time state–space model
future evolution of the system. For this, the network has to include the
( )
𝒙𝑘+1 = 𝒇 𝒙𝑘 , 𝒖𝑘 (15) state as an additional input variable. A single PINN
( ) ( )
𝒚 𝑘 = 𝒛 𝒙𝑘 , 𝒖𝑘 (16) ℎ (𝒑, 𝑡)𝒙𝑘 ≈ 𝛹 𝒑, 𝑡, 𝒙𝑘 (24)

based on the PINN formulation shown above. has then to be trained for arbitrary initial conditions, that is, states
The main challenge using the PINN concept within a state–space 𝒙𝑘 . In Eq. (24), ℎ (𝒑, 𝑡)𝒙𝑘 represents the PDE solution obtained with an
( )
setup is the definition of an appropriate state 𝒙𝑘 . Since the PINN 𝛹 initial condition that is given by an interpolation 𝐼 , 𝒙𝑘 , 𝒑 , which is
approximates the solution of a PDE to describe an actual physical based on this state 𝒙𝑘 . As now the state is an input to the network 𝛹 ,

3
F. Arnold and R. King Engineering Applications of Artificial Intelligence 101 (2021) 104195

Fig. 2. Space–time plot for a PINN with initial and boundary conditions according to Eq. (12), Eq. (13), and Eq. (14). Left: The PINN is trained with the exact initial and boundary
conditions. Right: The PINN is trained with the exact boundary conditions and 100 random initial conditions represented by the state vector 𝒙.

Fig. 3. Comparison of numerical solution, PINN without state input (SI), and with state input at 𝑡 = 0 s, 𝑡 = 0.5 s, and 𝑡 = 1 s.

the state has to be part of the training data. We consider 𝑁𝑥 different with corresponding labels
states ( )
ℎ = {ℎ 𝒑ℎ,1 , 𝑡ℎ,1 𝒙 , … ,
𝒙1 , … , 𝒙𝑗 , … , 𝒙𝑁𝑥 (25) ( ) 1
ℎ 𝒑ℎ,𝑖 , 𝑡ℎ,𝑖 𝒙 , … ,
within the training. These can be interpreted as 𝑁𝑥 different initial ( 1
)
conditions, which respect the spatial boundary condition as well, from ℎ 𝒑ℎ,𝑛ℎ , 𝑡ℎ,𝑛ℎ ,…,
𝒙1
which the net must be able to predict the future evolution. The points ( )
ℎ 𝒑ℎ,1 , 𝑡ℎ,1 𝒙 , … ,
of the initial condition are separated into the set ( )
𝑗

( ) ( ) ( ) ℎ 𝒑ℎ,𝑖 , 𝑡ℎ,𝑖 𝒙 , … ,
𝑗
0 = { 𝒑0,1 , 0, 𝒙1 , … , 𝒑0,𝑖 , 0, 𝒙1 , … , 𝒑0,𝑛0 , 0, 𝒙1 , ( )
( ) ( ) ( ) ℎ 𝒑ℎ,𝑛ℎ , 𝑡ℎ,𝑛ℎ ,…,
… , 𝒑0,1 , 0, 𝒙𝑗 , … , 𝒑0,𝑖 , 0, 𝒙𝑗 , … , 𝒑0,𝑛0 , 0, 𝒙𝑗 , 𝒙𝑗
( ) ( ) ( ) ( )
ℎ 𝒑ℎ,1 , 𝑡ℎ,1 𝒙 , … ,
… , 𝒑0,1 , 0, 𝒙𝑁𝑥 , … , 𝒑0,𝑖 , 0, 𝒙𝑁𝑥 , … , 𝒑0,𝑛0 , 0, 𝒙𝑁𝑥 }. (26) ( )
𝑁𝑥

ℎ 𝒑ℎ,𝑖 , 𝑡ℎ,𝑖 𝒙 , … ,
The training input data of the second part of the loss function (22) (
𝑁𝑥
)
extends to ℎ 𝒑ℎ,𝑛ℎ , 𝑡ℎ,𝑛ℎ }. (28)
( ) 𝒙𝑁𝑥
ℎ = { 𝒑ℎ,1 , 𝑡ℎ,1 , 𝒙1 , … ,
( ) For the last training data group that contains the collocation points, the
𝒑ℎ,𝑖 , 𝑡ℎ,𝑖 , 𝒙1 , … , training input data is extended as well:
( )
𝒑ℎ,𝑛ℎ , 𝑡ℎ,𝑛ℎ , 𝒙1 , … , ( )
( ) 𝑔 = { 𝒑𝑔,1 , 𝑡𝑔,1 , 𝒙1 , … ,
𝒑ℎ,1 , 𝑡ℎ,1 , 𝒙𝑗 , … , ( )
( ) 𝒑𝑔,𝑖 , 𝑡𝑔,𝑖 , 𝒙1 , … ,
𝒑ℎ,𝑖 , 𝑡ℎ,𝑖 , 𝒙𝑗 , … , ( )
( ) 𝒑𝑔,𝑛𝑔 , 𝑡𝑔,𝑛𝑔 , 𝒙1 , … ,
𝒑ℎ,𝑛ℎ , 𝑡ℎ,𝑛ℎ , 𝒙𝑗 , … , ( )
( ) 𝒑𝑔,1 , 𝑡𝑔,1 , 𝒙𝑗 , … ,
( )
𝒑ℎ,1 , 𝑡ℎ,1 , 𝒙𝑁𝑥 , … , 𝒑𝑔,𝑖 , 𝑡𝑔,𝑖 , 𝒙𝑗 , … ,
( ) ( )
𝒑ℎ,𝑖 , 𝑡ℎ,𝑖 , 𝒙𝑁𝑥 , … , 𝒑𝑔,𝑛𝑔 , 𝑡𝑔,𝑛𝑔 , 𝒙𝑗 , … ,
( ) ( )
𝒑ℎ,𝑛ℎ , 𝑡ℎ,𝑛ℎ , 𝒙𝑁𝑥 } (27) 𝒑𝑔,1 , 𝑡𝑔,1 , 𝒙𝑁𝑥 , … ,

4
F. Arnold and R. King Engineering Applications of Artificial Intelligence 101 (2021) 104195
( )
𝒑𝑔,𝑖 , 𝑡𝑔,𝑖 , 𝒙𝑁𝑥 , … , 4. Application: Burgers equation
( )
𝒑𝑔,𝑛𝑔 , 𝑡𝑔,𝑛𝑔 , 𝒙𝑁𝑥 } (29) We again consider the setup of the Burgers equation from Sec-
With this data, the loss tion 2.1, with the initial condition
( function
) from Eq. (22) can be applied to train
the neural network 𝛹 𝒑, 𝑡, 𝒙𝑘 . Note that this net is only trained once ℎ(𝑝, 0) = − sin(𝜋𝑝), (37)
before the actual application in a control or state estimation setup. As
the net is trained with random states, additional training between time which is now reflected in the initial state
steps is not required anymore. [ ]T [ ]T
𝒙0 = ℎ(𝑝1 , 0), ℎ(𝑝2 , 0), … , ℎ(𝑝10 , 0) = 𝑥0,1 , 𝑥0,2 … , 𝑥0,10 (38)
For the actuation, we assume a system manipulation through an
adjustment of the boundary conditions. A description based on a source at 10 equidistant positions
term might be useful for some applications but is not considered here.
Since a boundary condition bounds the space–time plane similar to the 𝑝1 , 𝑝2 , … , 𝑝10 (39)
initial condition, the procedure for transferring the initial condition
from 𝑝1 = −1 to 𝑝10 = 1. Between these positions, we assume a linear
to a state description could be applied equally well for the actuation.
interpolation 𝐼.
However, for an inclusion of an arbitrary actuation as well as arbitrary
In contrast to the autonomous example shown above, the additional
states, that is, initial conditions, a significantly larger network would
input layer now contains 12 neurons corresponding to the inputs 𝑝, 𝑡,
be necessary, which again would require more excessive training for
and a state 𝒙𝑘 of size 𝑛𝑥 = 10. Arbitrary states are incorporated by
a sufficient fit. Besides that, evaluating an increased network results
building a set of training data that contains 100 random states chosen
in higher computational costs within the applications discussed below.
from the domain −1 ≤ 𝑥𝑖 ≤ 1. For that, each of these random states
Alternatively, we train a set of 𝑛𝑢 networks with a fixed actuation, that
is combined with the training data described in Section 2.1. Thus, the
is, boundary condition 𝒖𝑗
{ amount of training data increases by a factor of 100. With regard to
( ) ( ) ( )}
𝛹 = 𝛹𝒖1 𝒑, 𝑡, 𝒙𝑘 , … , 𝛹𝒖𝑗 𝒑, 𝑡, 𝒙𝑘 , … , 𝛹𝒖𝑛 𝒑, 𝑡, 𝒙𝑘 , (30) the now 740 600 training samples, we extended the net size to 8 layers
𝑢
with 50 neurons. In our investigations with various net topologies,
which will be advantageous when it comes to model predictive control. these dimensions gave the best results for this example. We again used
We assume a bounded actuation normalized training sets, and the net’s weights are set using Xavier
𝒖lb ≤ 𝒖 ≤ 𝒖ub , (31) initialization. The larger number of training samples, as well as the
increased net dimension, enlarge the training task massively. Without
discretized into 𝑛𝑢 actuation values incorporating the state as an input, training takes about one minute for
{ } this example on an appropriate GPU. In contrast, it takes six to seven
𝒖 − 𝒖lb 𝒖 − 𝒖lb
 = 𝒖lb , 𝒖lb + ub , 𝒖lb + 2 ub , … , 𝒖ub hours to train the neural network with the additional state as input.
𝑛𝑢 − 1 𝑛𝑢 − 1
{ } Furthermore, we assume an actuation through the boundary con-
= 𝒖1 , … , 𝒖𝑗 , … , 𝒖𝑛𝑢 . (32)
dition ℎ(−1, 𝑡), which is discretized in steps of 0.1 between −1 and 1.
( ) Accordingly, 21 neural networks have to be trained for each actuation
A network 𝛹𝒖𝑗 𝒑, 𝑡, 𝒙𝑘 can then be trained using the training input data
according to Eq. (27), Eq. (26), and Eq. (29), together with the labels in
similar to Eq. (28). The labels have to be adapted for each actuation  = {−1, −0.9, … , 0.9, 1} . (40)
input 𝒖𝑗 so that, for example,
( ) ( ) A finer sampling of the actuation would be possible, however, for this
ℎ 𝒑lb , 𝑡 𝒙 = 𝒖𝑗 or ℎ 𝒑ub , 𝑡 𝒙 = 𝒖𝑗 (33) example our investigations indicate that a finer grid does not provide
𝑗 𝑗
noticeable improvements. The trained parameters, i.e., the weights of
holds, depending on the boundary condition that matches the actuation
the neural nets, are in a range of approximately [−1.5; 2.5], slightly
for the system. Here, it is assumed that actuation occurs either through
varying between different boundary conditions.
the lower or upper spatial boundary condition. For an actuation 𝒖𝑘 not
included in  , an interpolation rule for location 𝒑𝑥,𝑖 The right-hand side of Fig. 2 illustrates the fit of the PINN with
( an additional state as input. The actuation is set to 𝑢(𝑡) = 0 so that
( ) ( ))
𝐼𝑢 𝛹𝒖 𝒑𝑥,𝑖 , 𝑡, 𝒙𝑘 , 𝛹𝒖̄ 𝑘 𝒑𝑥,𝑖 , 𝑡, 𝒙𝑘 (34) the result is comparable with the fit obtained by a PINN that is only
𝑘
trained for one specific initial condition, as described in Section 2.1.
can be defined, where 𝒖𝑘 and 𝒖̄ 𝑘 are the two closest actuation candi- In this visualization, a difference between the two results is hardly
dates in  . recognizable, which indicates a sufficient generalization capability of
Finally, the discrete-time state–space model reads the PINN with state input. For better comparability, the PINN solution
( )
⎡ 𝐼 𝛹 (𝒑 , 𝛥𝑡, 𝒙 ) , 𝛹 (𝒑 , 𝛥𝑡, 𝒙 ) ⎤ with state input is shown in Fig. 3 at 𝑡 = 0 s, 𝑡 = 0.5 s, and 𝑡 = 1 s together
⎢ 𝑢 𝒖𝑘 𝑥,1 𝑘 𝒖̄ 𝑘 𝑥,1 𝑘
⎥ with the PINN solution without state input from Section 2.1 and the
⎢ ⋮ ⎥
⎢ ( ( ) ( )) ⎥ numerical solution. In contrast to the PINN solution without state input,
𝒙𝑘+1 = ⎢ 𝐼𝑢 𝛹𝒖 𝒑𝑥,𝑖 , 𝛥𝑡, 𝒙𝑘 , 𝛹𝒖̄ 𝑘 𝒑𝑥,𝑖 , 𝛥𝑡, 𝒙𝑘 ⎥ (35) which is almost indistinguishable from the numerical solution, the
𝑘
⎢ ⋮ ⎥
⎢ ( ( ) ( )) ⎥ PINN solution with state input deviates noticeably from the numerical
⎢ 𝐼𝑢 𝛹𝒖 𝒑𝑥,𝑛 , 𝛥𝑡, 𝒙𝑘 , 𝛹𝒖̄ 𝒑𝑥,𝑛 , 𝛥𝑡, 𝒙𝑘 ⎥ solution, particularly at the peaks of the profile. Nevertheless, the fit
⎣ 𝑘 𝑥 𝑘 𝑥 ⎦
( ) is still sufficient, especially if we keep in mind that this model, which
𝒚 𝑘 = 𝑧 𝒙𝑘 , 𝒖𝑘 (36) is based on a PINN solution, is dedicated to control or state estimation
with the fixed parameters 𝒑𝑥,𝑖 and 𝛥𝑡. Thus, a simple evaluation of applications where model errors can be compensated for.
the neural nets contained in Eq. (35) allows for a one-step prediction For the shown example, the CPU time to numerically calculate the
without numerical integration. Moreover, as all nets are formulated in solution at 𝑡 = 1 s with the finite volume code is about ten times higher
TensorFlow, derivatives of the nets can be readily evaluated, which will compared to a similar calculation using the PINN model.1 However, the
be beneficial for the extended Kalman filter study shown below. difference in computational costs heavily depends on the required time
We do not specify the measurement equation in greater detail here,
as it has to be chosen for a concrete system with respect to the defined 1
The actual computation time depends on the used hardware and imple-
state 𝒙𝑘 . For the MPC and state estimation example of the Burgers mentation. On a seventh generation Intel i7 CPU it takes about 0.14 s for the
equation, we define the measurement below. numerical solution and 0.013 s for the prediction based on the PINN model.

5
F. Arnold and R. King Engineering Applications of Artificial Intelligence 101 (2021) 104195

discretization to obtain a sufficiently accurate numerical solution. As is chosen, which defines the state 𝑥𝑘,3 as the scalar system output 𝑦𝑘 .
the focus of this contribution is a state–space representation that can Fig. 4 shows the measurement axis in the space–time plane. Without
be used in a real-time application, the inference computation time, that control, due to the viscosity described in the Burgers equation, the field
is, the time to evaluate the PINN model, is the relevant measure. ℎ(𝑝, 𝑡) would converge to 0 everywhere, as well as for the measurement
We now use the state–space model for the Burgers equation within position 𝑦𝑘 . Instead, the MPC aims to track a sinusoidal reference,
a control and state estimation example. For that, we consider a system
that is based on the numerical solution of the Burgers equation. 𝑟(𝑡) = 𝐴 ⋅ sin(𝑓𝑟 ⋅ 2𝜋 ⋅ 𝑡) + 𝑏 (54)

with a frequency 𝑓𝑟 = 0.02 Hz, an amplitude 𝐴 = 0.4, and an offset

4.1. MPC application 𝑏 = 0.4. Based on the reference for the upcoming 𝑛𝐻 time steps
[ ]T
In this section, we demonstrate the capability of the presented 𝑹𝑘 = 𝑟((𝑘 + 1) ⋅ 𝛥𝑡), 𝑟((𝑘 + 2) ⋅ 𝛥𝑡), … , 𝑟((𝑘 + 𝑛𝐻 ) ⋅ 𝛥𝑡) , (55)
approach in a control context. MPC is a universal control approach used the cost function for time step 𝑘 is defined as
for a wide range of applications like disturbance rejection, reference ( ( ) ( ) )
tracking, or general cost function optimization. We will discuss only 𝐽 𝒙𝑘 𝒖𝑘 , 𝒚 𝑘 𝒖𝑘 , 𝒖𝑘 = ‖𝑹𝑘 − 𝒀 𝑘 ‖22 . (56)
the idea of MPC, shortly. A survey of the theory and application of
The prediction horizon is set to 𝑛𝐻 = 5, and the actuation horizon is
MPC can be found, for example, in Qin and Badgwell (2003) or Lee
𝑛𝑈 = 3 time steps long. The time step size for this example is 𝛥𝑡 = 0.5 s.
(2011). For more recent developments in the field of MPC, we refer as
Similar to the previously shown examples, the initial condition is set
well to Mayne (2014).
according to Eq. (12). Fig. 4 illustrates the resulting system behavior
We assume a generic dynamic system described by the state–space
in the space–time plane. On the left-hand side, the initial condition
model
is visible for the first few seconds before the enforced time-periodic
( )
𝒙𝑘+1 = 𝒇 𝒙𝑘 , 𝒖𝑘 (41) behavior becomes dominant.
( ) For a better evaluation of the actual tracking performance of the
𝒚 𝑘 = 𝒛 𝒙𝑘 , 𝒖𝑘 , (42)
MPC, Fig. 5 shows the system output 𝑦(𝑡) and the defined reference
consisting of the state equation (41) and measurement equation (42), a 𝑟(𝑡). The MPC is capable of sufficiently tracking the reference after a
known state 𝒙𝑘 at time step 𝑘, and an already determined input 𝒖𝑘 for short period of a few seconds. From that point on, the system output
the next sampling period. With the state–space model, the states adequately follows the reference for most of the time. Exceptions
[ ]T are the minima and maxima of the reference. We presume that the
𝑿 𝑘 = 𝒙T𝑘+1 , 𝒙T𝑘+2 , … , 𝒙T𝑘+𝑛 (43) observed error at these times is a result of the model error and the
𝐻
limited dynamic of the assumed system. Overall, the tracking shows
and the system outputs
a comparable quality to the results in Arbabi et al. (2018) and Atwell
[ ]T
𝒀 𝑘 = 𝒚 T𝑘+1 , 𝒚 T𝑘+2 , … , 𝒚 T𝑘+𝑛 (44) et al. (2001), where a linear model derived from a Koopman operator or
𝐻
a linearized model is used, respectively. Fig. 6 shows the corresponding
can be predicted for the next 𝑛𝐻 time steps based on a known actuation actuation 𝑢(𝑡).
The solution of the optimization problem in every time step, in-
[ ]T cluding predictions with the PINN, takes about 0.4 s on average on
𝑼 𝑘 = 𝒖T𝑘 , 𝒖T𝑘+1 , … , 𝒖T𝑘+𝑛 . (45)
𝑈 an Intel I7 processor with 4 cores using MATLABs fmincon solver. This
for the next 𝑛𝑈 time steps. If 𝑛𝑈 < 𝑛𝐻 , the actuation 𝑼 𝑘 has to be highlights the advantage of this approach with respect to real-time
extended to obtain a control trajectory, an example being capability.

𝑼 ∗𝑘 = [𝒖𝑘+1 , 𝒖𝑘+2 , … , 𝒖𝑘+𝑛𝑈 , 𝒖𝑘+𝑛𝑈 , … , 𝒖𝑘+𝑛𝑈 ]T (46) 4.2. Extended Kalman filter application
⏟⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏟⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏟ ⏟⏞⏞⏞⏞⏞⏞⏞⏞⏟⏞⏞⏞⏞⏞⏞⏞⏞⏟
𝑼𝑘 constant 𝒖𝑘+𝑛
𝑈
Kalman filtering is a very well-known method for estimating the
that determines a system input for all 𝑛𝐻 future time steps. Depending state of uncertain systems based on measurement and model infor-
on the predicted states 𝑿 𝑘 , system outputs 𝒀 𝑘 , and the chosen actuation mation. There are various approaches based on the original idea of a
𝑼 𝑘 , a cost function Kalman filter customized for particular uses. In this work, we will focus
( ( ) ( ) ) on the so-called Extended Kalman filter (EKF) that is an extension of the
𝐽𝑘 = 𝐽 𝑿 𝑘 𝑼 𝑘 , 𝒀 𝑘 𝑼 𝑘 , 𝑼 𝑘 (47)
original Kalman filter approach, which is only suitable for linear model
can be formulated that quantifies the control performance of the up- descriptions.
coming 𝑛𝐻 time steps. The solution to the optimization problem We will assume a scalar input 𝑢(𝑡) for the considered example. Nev-
( ( ) ( ) ) ertheless, the procedure for multi-input multi-output systems functions
𝑼̃ 𝑘 = arg min 𝐽 𝑿 𝑘 𝑼 𝑘 , 𝒀 𝑘 𝑼 𝑘 , 𝑼 𝑘 , (48) accordingly. We assume a discrete-time uncertain system
𝑼𝑘
( )
subject to 𝒙𝑘+1 = 𝒇 𝒙𝑘 , 𝑢𝑘 + 𝒘𝑘 (57)
( )
∀𝑖 ∈ {1, 2, … , 𝑛𝐻 } (49) 𝒚 𝑘 = 𝒛 𝒙𝑘 , 𝑢𝑘 + 𝒗𝑘 (58)
( )
𝒙𝑘+𝑖 = 𝒇 𝒙𝑘+𝑖−1 , 𝒖𝑘+𝑖−1 (50) with the additive independent and identically distributed disturbances
( ) 𝒘𝑘 and 𝒗𝑘 . Both disturbances, system noise 𝒘𝑘 and measurement noise
𝒚 𝑘+𝑖−1 = 𝒛 𝒙𝑘+𝑖−1 , 𝒖𝑘+𝑖−1 (51) 𝒗𝑘 , are expected to be white noise Gaussian processes. Consequently,
𝒖lb ≤ 𝒖𝑘+𝑖−1 ≤ 𝒖ub , (52)
𝐸{𝒘𝑘 } = 𝟎, 𝐸{𝒘𝑘 𝒘T𝑙 } = 𝑸𝑘 𝛿𝑘,𝑙 (59)
gives an optimal control trajectory 𝑼̃ 𝑘 regarding the cost function from
𝐸{𝒗𝑘 } = 𝟎, 𝐸{𝒗𝑘 𝒗T𝑙 } = 𝑹𝑘 𝛿𝑘,𝑙 (60)
Eq. (47). In an MPC, for the upcoming time step 𝑘 + 1, the first element
of the optimal control trajectory, 𝒖̃ 𝑘+1 , is applied to the system and a applies for their first and second moments. The matrix 𝑸𝑘 and the scalar
new optimization with the described steps starts all over again. 𝑹𝑘 quantify the covariance of the disturbances. 𝛿𝑘,𝑙 is the Kronecker
For this MPC example, an arbitrary measurement equation delta. Using the model in Eqs. (57) and (58), an estimation
( ) ( )
𝑦𝑘 = 𝑧 𝒙𝑘 , 𝒖𝑘 = 𝑥𝑘,3 (53) 𝒙̂ 𝑘+1|𝑘 = 𝒇 𝒙̂ 𝑘|𝑘 , 𝑢𝑘 (61)

6
F. Arnold and R. King Engineering Applications of Artificial Intelligence 101 (2021) 104195

Fig. 4. Space–time plot of the numerical Burgers system with an MPC actuation at 𝑝 = −1. The broken line shows the measurement axis of the scalar output 𝑦𝑘 = 𝑥𝑘,3 .

Fig. 5. Tracking of a sinusoidal reference with an MPC, based on a PINN state–space model.

Fig. 6. Actuation of the MPC corresponding to the reference tracking example in Fig. 5.

( ) ([ ( )]
𝒚̂ 𝑘+1|𝑘 = 𝒛 𝒙̂ 𝑘+1|𝑘 , 𝑢𝑘+1 (62) = tr 𝒙𝑘+1 − 𝒙̂ 𝑘+1|𝑘 − 𝑲 𝑘+1 𝒚 𝑘+1 − 𝒚̂ 𝑘+1|𝑘
can be calculated based on the estimated state 𝒙̂ 𝑘|𝑘 from time step 𝑘. [ ( )]T )
𝒙𝑘+1 − 𝒙̂ 𝑘+1|𝑘 − 𝑲 𝑘+1 𝒚 𝑘+1 − 𝒚̂ 𝑘+1|𝑘 . (66)
The subscript 𝑘|𝑘 indicates that a variable is given at time step 𝑘 under
the condition that all measurements up to the time step 𝑘 have been As a result, see for example Simon (2007) or Simon (2010),
processed as
𝑷 𝑘+1|𝑘 = 𝑭 𝑷 𝑘|𝑘 𝑭 T + 𝑸𝑘 (67)
𝒙̂ 𝑘|𝑘 = 𝐸{𝒙̂ 𝑘 |𝒚 1 , 𝒚 2 , … , 𝒚 𝑘 }. (63)
and
When a new measurement 𝒚 𝑘+1 is obtained in time step 𝑘+1, the model- [ ]−1
𝑲 𝑘+1 = 𝑷 𝑘+1|𝑘 𝑯 T 𝑯𝑷 𝑘+1|𝑘 𝑯 T + 𝑹𝑘+1 . (68)
based estimation 𝒙̂ 𝑘+1|𝑘 can be adapted using a proportional approach
with
( ) 𝜕𝒇 |
𝒙̂ 𝑘+1|𝑘+1 = 𝒙̂ 𝑘+1|𝑘 + 𝑲 𝑘+1 𝒚 𝑘+1 − 𝒚̂ 𝑘+1|𝑘 . (64) 𝑭 = | (69)
𝜕𝒙 |𝒙=𝒙̂ 𝑘|𝑘
The quality of the estimation can be quantified by the second moment
and
of the estimation error 𝒙𝑒𝑘
𝜕𝒛 |
[ ][ ]T 𝑯= | . (70)
𝑷 𝑘|𝑘 = 𝒙𝑘 − 𝒙̂ 𝑘|𝑘 𝒙𝑘 − 𝒙̂ 𝑘|𝑘 . (65) 𝜕𝒙 |𝒙=𝒙̂ 𝑘+1|𝑘
⏟⏞⏞⏞⏟⏞⏞⏞⏟ ⏟⏞⏞⏞⏞⏞⏟⏞⏞⏞⏞⏞⏟ In this context, the option to apply automatic differentiation to calcu-
𝒙𝑒𝑘 𝒙𝑒𝑘 T
late 𝑭 and 𝑯 is the main advantage of a PINN-based state–space model
The idea of a Kalman filter is an optimal adaption of the filter gain formulated in TensorFlow and other machine learning frameworks. With
𝑲 𝑘+1 , with respect to the estimation error 𝒙𝑒𝑘+1 by means of a mini- the optimal filter 𝑲 𝑘+1 in Eq. (68), the second moment of the estimation
mization of the trace of its second moment error for time step 𝑘 + 1 reads
( ) ([ ][ ]T )
tr 𝑷 𝑘+1|𝑘+1 = tr 𝒙𝑘+1 − 𝒙̂ 𝑘+1|𝑘+1 𝒙𝑘+1 − 𝒙̂ 𝑘+1|𝑘+1 𝑷 𝑘+1|𝑘+1 = 𝑷 𝑘+1|𝑘 − 𝑲 𝑘+1 𝑯𝑷 𝑘+1|𝑘 . (71)

7
F. Arnold and R. King Engineering Applications of Artificial Intelligence 101 (2021) 104195

Algorithm 1 Extended Kalman Filter An additional example concerning state estimation is illustrated in
Fig. 9. In contrast to the previous EKF example of an initial estimation
1: 𝑷 𝑘|𝑘 = 𝑷 0|0
according to Eq. (73), the initial estimation is now set to
2: 𝑸𝑘 = 𝑸0
3: 𝑹𝑘+1 = 𝑹0 𝒙̂ 0 = [0, 0, 1, 0, 0, 0, 0, −1, 0, 0]T . (76)
4: 𝒙̂ 𝑘|𝑘 = 𝒙̂ 0|0
5: while TRUE do Fig. 9 shows the numerical solution of the Burgers equation, that is, the
6: ⊳ Time Update ’’real’’ system behavior, and the EKF-based estimation using the same
7: 𝑷 𝑘+1|𝑘 = 𝑭 𝑷 𝑘|𝑘 𝑭 T + 𝑸𝑘 measurement information as in the previous EKF example. Addition-
( ) ally, the model prediction of the PINN, without a state correction by
8: 𝒙̂ 𝑘+1|𝑘 = 𝒇 𝒙̂ 𝑘|𝑘 , 𝑢𝑘
( ) the Kalman filter, highlights the advantage of the EKF-based estimation.
9: 𝒚̂ 𝑘+1|𝑘 = 𝒛 𝒙̂ 𝑘+1|𝑘 , 𝑢𝑘+1
The PINN model itself is applied to interpolate between the estimated
10: ⊳ Measurement Update state to obtain the displayed spatial profiles. The results indicate that
[ ]−1 the falsely estimated initial condition can be compensated for by the
11: 𝑲 𝑘+1 = 𝑷 𝑘+1|𝑘 𝑯 T 𝑯𝑷 𝑘+1|𝑘 𝑯 T + 𝑅𝑘+1
( ) EKF, while the pure model prediction with Eq. (35) yields unreliable
12: 𝒙̂ 𝑘+1|𝑘+1 = 𝒙̂ 𝑘+1|𝑘 + 𝑲 𝑘+1 𝒚 𝑘+1 − 𝒚̂ 𝑘+1|𝑘
information for all the times. In Fig. 9, we used the trained PINN to
13: 𝑷 𝑘+1|𝑘+1 = 𝑷 𝑘+1|𝑘 − 𝑲 𝑘+1 𝑯𝑷 𝑘+1|𝑘
interpolate between the actual states
14: end while
Although this state estimation setup yields good results, it should
be noted that the performance heavily depends on the chosen mea-
surement equation, that is, the number and position of the measurable
By applying the described procedure in each time step, the state es- states. With fewer or other measurement positions, the observability
timation 𝒙̂ 𝑘 will improve over time and, as long as the state is fully of the complete state vector might get lost. Nevertheless, the com-
observable from the measurement, it converges to the real state 𝒙𝑘 . putational burden for solving the EKF equations allowed again for a
Algorithm 1 summarizes the steps of one iteration of the Extended real-time application.
Kalman filter consisting of time and measurement update. Without
loss of generality, we will assume a constant actuation of 𝑢(𝑡) = 0 in 5. Conclusion
what fallows. In contrast to the MPC setup, the sampling time for this
example is set to 𝛥𝑡 = 0.1 s. Similar to the MPC example, the state The presented results indicate that the introduced state–space mod-
vector 𝒙𝑘 consists of the velocity at 10 equidistant positions. We assume eling based on the PINN formulation is a promising approach for highly
that four of the 10 states, namely 𝑥𝑘,2 , 𝑥𝑘,5 , 𝑥𝑘,6 , 𝑥𝑘,9 , are measured. complex physical systems that can be described by PDEs. In contrast
Accordingly, the measurement equation reads to state–space formulation based on a numerical approximation of a
⎡ 0 1 0 0 0 0 0 0 0 0 ⎤ PDE, the proposed PINN-based approach avoids computational costly
( ) ⎢ 0 0 0 0 1 0 0 0 0 0
⎥ time integration. This is especially advantageous for very stiff PDE
𝒚 𝑘 = 𝐳 𝒙𝑘 , 𝑢𝑘 = ⎢ ⎥ ⋅ 𝒙𝑘 . systems that require a fine time discretization within the numerical
⎢ 0 0 0 0 0 1 0 0 0 0 ⎥
⎢ 0 0 0 0 0 0 0 0 1 0 ⎥ solution. The Burgers equation example shows that the model fit with
⎣ ⎦
respect to a numerical solution is sufficient for control purposes and
For an extreme case, to show the convergence of the filter, the initial state estimation applications. The positive results in the tracking per-
condition of the real system formance of the MPC setup and also in the realization of an Extended
Kalman filter could be obtained with a relatively small neural network
ℎ(𝑥, 0) = − sin(𝜋 𝑥) (72)
of just over 400 neurons. This is a promising result with regard to
is assumed to be completely unknown despite the measurements at four a real-time application. After the initial training prior to the actual
positions. Therefore, the estimated state is initialized with application, the computational cost for the model prediction with the
PINN state–space model is substantially lower in comparison to the
𝒙̂ 0 = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]T . (73)
numerical solution. This enables the application of sophisticated model-
Fig. 7 shows a color map of the velocity ℎ(𝑥, 𝑡) in the space–time plane based control and state estimation methods. Besides the small size of
for 𝑡 ∈ [0, 1]. The measurement times and positions are identified with the neural network, the potential for parallelization in the evaluation
black circles. of the model enables a highly efficient implementation using embedded
The tuning parameters, that is, the initial covariance matrices of the architectures with hardware acceleration, such as FPGAs or GPUs. An
estimation error and the system noise, are set to additional advantage is the simple structure of the neural networks
applied in this contribution. Feed-forward neural networks allow for
𝑷 0|0 = 0.5 ⋅ 𝑰 10 𝑸0 = 0.5 ⋅ 𝑰 10 . (74)
an efficient calculation of state or actuation derivatives using automatic
10 differentiation. This enables, for example, an easy linearization for an
In Eq. (74), 𝑰 denotes the identity matrix of size 10. The covariance
of the measurement noise is initialized with Extended Kalman filter application or for applying other methods of
linear control theory.
𝑹𝑘 = 𝑰 4 (75)
Even though the presented work provides a generic modeling
A first impression of the behavior of the Kalman filter is given on the method for all kinds of systems that can be described by PDEs, we think
left-hand side of Fig. 8, which shows the evolution of the measurable that our approach might be especially useful for flow and combustion
states. All estimations converge to the real states after 0.5 s, which is an control and the observation of such processes. The application of a
expectable result for states directly corresponding to a measurement. A state–space model within control that is based on a neural network
better initial guess, and one that exploits the measurement information, but also includes physical information in the form of PDEs might prove
would result in a faster convergence. More significant for evaluating to be a promising alternative to other purely data driven approaches.
the state estimation performance, however, is the observation of the Including physical information might not only result in more robust
hidden states. The right side of Fig. 8 shows the course of the hidden models. In contrast to purely data driven approaches, upscaling or mod-
states 𝑥3 (𝑡), 𝑥4 (𝑡), 𝑥7 (𝑡), and 𝑥8 (𝑡) and their estimates. The states 𝑥1 (𝑡) ification of a process can be tackled on a theoretical/numerical basis.
and 𝑥10 (𝑡) are not shown here since they mainly depend on boundary This can be done without resorting to numerous experimental tests first
conditions that are constant and equal to the initial estimation in this to gain new data, as the capability of prediction is maintained by using
example. the original PDE. Aside from that, the state–space formulation allows

8
F. Arnold and R. King Engineering Applications of Artificial Intelligence 101 (2021) 104195

Fig. 7. Space–time plot of the numerical system that should be observed by the EKF. Measurement times and positions are marked with black circles.

Fig. 8. State evolution. Left: Evolution of measurable states and the corresponding estimation over time Right: Evolution of hidden states and the corresponding estimation over
time.

Fig. 9. Numerical solution of Burgers equation compared with EKF-based estimation and plain model prediction at 𝑡 = 0, 𝑡 = 0.2, 𝑡 = 0.4, and 𝑡 = 0.6.

us to use it together with a broad range of control and state estimation In this contribution, the results presented for the Burgers equa-
approaches for which in-depth theoretical analysis is available. This can tion are meant to be a proof of concept. Though the consideration
be especially useful for convergence and stability analysis, which are of stronger nonlinear phenomena like vortices and shock waves is
more difficult, if possible at all, with purely data driven models. more difficult, the results in the literature show that an approximation

9
F. Arnold and R. King Engineering Applications of Artificial Intelligence 101 (2021) 104195

of systems that can be described by the Navier–Stokes equations is Gräßle, C., Hinze, M., Volkwein, S., 2019. Model order reduction by proper orthogonal
possible with the PINN approach. However, a transformation of the decomposition. arXiv preprint arXiv:1906.05188.
presented state–space modeling method will be challenging. In future Gunzburger, M.D., Manservisi, S., 2000. Tracking problem for Navier – Stokes flows.
SIAM J. Numer. Anal. 37 (5), 1481–1512.
investigations, we will apply the method to a more realistic scenario
Hauser, M., Gunn, S., Saab Jr., S., Ray, A., 2019. State-space representations of deep
with a flow field described by the Euler equations. A setup for the neural networks. Neural Comput. 31, 1–17.
Euler equations would show the applicability of the approach for Hunt, K.J., Sbarbaro, D., Zbikowski, R., Gawthrop, P.J., 1992. Neural networks for
real flow applications. Furthermore, an investigation of stability and control systems - A survey. Automatica 28 (6), 1083–1112.
convergence properties of setups based on a PINN state–space model is Jiang, W., Zhang, N., Xue, X., Xu, Y., Zhou, J., Wang, X., 2020. Intelligent deep learning
method for forecasting the health evolution trend of aero-engine with dispersion
necessary to ensure safe and predictable behavior when applied on a
entropy-based multi-scale series aggregation and LSTM neural network. IEEE Access
real experiment. 8, 34350–34361.
Lee, J.H., 2011. Model predictive control: Review of the three decades of development.
CRediT authorship contribution statement Int. J. Control Autom. Syst. 9 (3), 415–424.
Mayne, D.Q., 2014. Model predictive control: Recent developments and future promise.
Florian Arnold: Conceptualization, Methodology, Software, Valida- Automatica 50 (12), 2967–2986.
Mehrmann, V., Stykel, T., 2005. Balanced truncation model reduction for large-scale
tion, Writing - original draft, Visualization. Rudibert King: Conceptual-
systems in descriptor form. In: Dimension Reduction of Large-Scale Systems.
ization, Writing - review & editing, Supervision, Project administration, Springer, pp. 83–115.
Funding acquisition. Noack, B.R., Afanasiev, K., Morzyński, M., Tadmor, G., Thiele, F., 2003. A hierarchy
of low-dimensional models for the transient and post-transient cylinder wake. J.
Declaration of competing interest Fluid Mech. 497, 335–363.
Qin, S.J., Badgwell, T.A., 2003. A survey of industrial model predictive control
technology. Control Eng. Pract. 11, 733–764.
The authors declare that they have no known competing finan- Raissi, M., 2018. Deep hidden physics models: Deep learning of nonlinear partial
cial interests or personal relationships that could have appeared to differential equations. J. Mach. Learn. Res. 19, 1–24.
influence the work reported in this paper. Raissi, M., Perdikaris, P., Karniadakis, G.E., 2017a. Physics informed deep learning
(Part I): Data-driven discovery of nonlinear partial differential equations. arXiv:
Acknowledgment 1711.10566.
Raissi, M., Perdikaris, P., Karniadakis, G.E., 2017b. Physics informed deep learning
(Part II): Data-driven discovery of nonlinear partial differential equations. arXiv:
The authors gratefully acknowledge support by the 1711.10566.
Deutsche Forschungsgemeinschaft (DFG), Germany as part of collabo- Raissi, M., Perdikaris, P., Karniadakis, G.E., 2019a. Physics-informed neural networks:
rative research center SFB 1029 ‘‘Substantial efficiency increase in gas A deep learning framework for solving forward and inverse problems involving
turbines through direct use of coupled unsteady combustion and flow nonlinear partial differential equations. J. Comput. Phys. 378, 686–707.
Raissi, M., Wang, Z., Triantafyllou, M.S., Karniadakis, G.E., 2019b. Deep learning of
dynamics’’ in project A05.
vortex-induced vibrations. J. Fluid Mech. 861, 119–137.
Rangapuram, S.S., Seeger, M.W., Gasthaus, J., Stella, L., Wang, Y., Januschowski, T.,
References 2018. Deep state space models for time series forecasting. In: Bengio, S., Wal-
lach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Garnett, R. (Eds.), Advances
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., 2015. TensorFlow: Large-scale machine in Neural Information Processing Systems, Vol. 31. Curran Associates, Inc., pp.
learning on heterogeneous systems. Software available from tensorflow.org. https: 7785–7794.
//www.tensorflow.org/. Sagheer, A., Kotb, M., 2019. Time series forecasting of petroleum production using
Arbabi, H., Korda, M., Mezić, I., 2018. A data-driven Koopman model predictive deep LSTM recurrent networks. Neurocomputing 323, 203–213.
control framework for nonlinear flows. In: Proceedings of 2018 IEEE Conference Simon, D., 2007. Optimal state estimation: Kalman, H-infinity, and nonlinear
on Decision and Control (CDC) Miami Beach, FL, USA, pp. 6409–6414. approaches. Wiley-Interscience.
Atwell, J., Borggaard, J., King, B., 2001. Reduced order controllers for Burgers’ equation Simon, D., 2010. Kalman filtering with state constraints: A survey of linear and
with a nonlinear observer. Appl. Math. Comput. Sci. 11 (6), 1311–1330. nonlinear algorithms. IET Control Theory Appl. 4 (8), 1303–1318.
Bai, Z., 2002. Krylov subspace techniques for reduced-order modeling of large-scale
Virtanen, P., Gommers, R., Oliphant, T.E., Haberland, M., Reddy, T., Cournapeau, D.,
dynamical systems. Appl. Numer. Math. 43 (1–2), 9–44.
Burovski, E., Peterson, P., Weckesser, W., Bright, J., van der Walt, S.J., Brett, M.,
Brunton, S.L., Noack, B.R., Koumoutsakos, P., 2020. Machine learning for fluid
Wilson, J., Millman, K.J., Mayorov, N., Nelson, A.R.J., Jones, E., Kern, R.,
mechanics. Annu. Rev. Fluid Mech. 52 (1), 477–508.
Larson, E., Carey, C.J., Polat, İ., Feng, Y., Moore, E.W., VanderPlas, J., Lax-
Connor, J.T., Martin, R.D., Atlas, L.E., 1994. Recurrent neural networks and robust
alde, D., Perktold, J., Cimrman, R., Henriksen, I., Quintero, E.A., Harris, C.R.,
time series prediction. IEEE Trans. Neural Netw. 5 (2), 240–254.
Archibald, A.M., Ribeiro, A.H., Pedregosa, F., van Mulbregt, P., SciPy 1.0 Contrib-
Desai, M., Ito, K., 1994. Optimal controls of Navier-Stokes equations. SIAM J. Control
utors, 2020. SciPy 1.0: Fundamental algorithms for scientific computing in python.
Optim. 32 (5), 1428–1446.
Gillespie, M.T., Best, C.M., Townsend, E.C., Wingate, D., Killpack, M.D., 2018. Learning Nature Methods 17, 261–272.
nonlinear dynamic models of soft robots for model predictive control with neural
networks. In: 2018 IEEE International Conference on Soft Robotics, RoboSoft 2018.
IEEE, pp. 39–45.

10