Mahesh2022 Chapter PhysicsInformedNeuralNetworkFo
Mahesh2022 Chapter PhysicsInformedNeuralNetworkFo
net/publication/356389462
CITATIONS READS
18 2,711
3 authors:
Qing Lin
University of Giessen
10 PUBLICATIONS 118 CITATIONS
SEE PROFILE
All content following this page was uploaded by J. Leandro on 08 March 2022.
1 Introduction
Flood forecasting plays a prominent role today in flood management. There has been
a transition in flood mitigation measures from a structural approach to non-structural
approaches such as flood forecasting and early warning system; this transition is
because no single mitigation or defense measure is entirely adequate [12]. The flood
forecasting techniques help in flood management preparedness and early warning
systems by providing an extended lead time for the decision-makers [29].
Conventionally, flood forecasting is performed by physically based models or
conceptual models [5]. The conceptual hydrological models are usually based on
the continuity equation where the dynamic and kinematic wave is applied, while the
hydraulic routing is physically based and the fully dynamic Saint Venant Equations is
numerically solved [12, 22]. A physically based model provides accurate results but
is computationally expensive, scales with domain size, requires large geometry data,
and does not meet real-time applications due to the large number of computations
performed.
The data-driven model, such as Artificial Neural Network (ANN) for flood fore-
casting, has been very promising [17]. Artificial Neural Networks are algorithms
used to map input and output through a mathematical relation. It is made of a neuron
structure organized in layers. Stacking multiple layers of this structure allows deter-
mining complex non-linear relationships [10]. Rumelhart et al. [26] were the first to
train such a neural network to learn with backpropagation training. LeCun et al. [18]
pioneered deep learning where multilayer perceptron stacked with a dense number
of hidden layers to form a deep neural network. Thus, such a network that determines
complex non-linearities finds its place in a hydrological application. However, the
© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2022 77
S. Kolathayar et al. (eds.), Climate Change and Water Security, Lecture Notes
in Civil Engineering 178, https://doi.org/10.1007/978-981-16-5501-2_7
78 R. B. Mahesh et al.
ANNs are still inferior to the classical method of solving the Saint Venant Equa-
tions in terms of accuracy. Therefore, it has always resulted in tradeoff between the
model performance and computational efforts while selecting an appropriate model.
Several architectures that are available under the ANN structure have been applied
to flood forecasting. Lin et al. [19] used ANN to determine the flood inundation
maps. Kasiviswanathan et al. [15] applied ANN for flood forecasting, where they
compared Wavelet-Based Neural network (WNN) with ANN in terms of the forecast
accuracy and precision at various lead-times. Elsafi [7] did an ANN study for flood
prediction where ANN was applied at specific locations to determine the prediction
using the upstream boundary condition. Kratzert et al. [17] tackle the loss of the
input information’s sequential order in a neural network using the Recurrent Neural
Network (RNN), where Long Short-Term Memory (LSTM) models are state of the
art. Another similar test was performed by Sit and Demir [31] for the applicability of
neural networks for flood forecasting where the gradient vanishing problem of RNN
was addressed, and the Gated Recurrent Unit (GRU) was used as the network archi-
tecture. Bhola et al. [3] have used Computer Vision to determine flood inundation
forecast. Due to the complex nature of flood forecasting, researchers have always
implemented leading and promising architecture.
Research in deep learning has progressed to utilize physics information to either
solve PDE or enhance a PDE solver. The latter uses neural networks to boost the
PDE solver; such a field is called Geometric Deep Learning. Haehnel et al. [11]
used the domain decomposition technique from Computational Fluid Dynamics,
where the deep learning models are trained at subdomain levels under the boundary
condition constraint to monitor and forecast air pollution. In a CFD-driven machine
learning model, Zhao et al. [35] used RANS to model turbulence and a machine
learning model to correct the error between the turbulence and direct numerical
simulation. Dockhorn [6], through his work, discussed solving PDE such as Navier
Stokes Equation using combined loss function and BFGS optimizer. Baymani et al.
[2] also utilized ANN to solve the Navier Stokes Equation, where a stream function is
used in the error function to drive it to a minimum. Wave propagation with the neural
network has been explored with different architecture on image datasets [8, 32]. The
most popular research to solve the PDE using a neural network is by Raissi et al. [25].
They introduced PINN that is capable of solving any physical law governing the data.
PINNs are deep learning techniques used to solve the Partial Differential Equation
(PDE) by approximating a solution to the PDE. The hypothesis for PINNs is to bridge
the gap between classical methods and the data-driven neural networks by utilizing
available physics information. The Navier Stokes Equation has been encoded in
the PINNs in such an application case [24]. PINN implementations further include
seismic wave propagation [14] and flood forecasting with image datasets [23]. The
PINNs could be an efficient solution to tackle the tradeoff between the performance
and the computational efforts. They are very recent advancements; therefore, there
is currently every less or no research where the PINNs are constrained with Saint
Venant Equations for flood forecasting using numerical data.
Physics Informed Neural Network for Spatial-Temporal Flood Forecasting 79
This work focuses on PINNs for spatial-temporal scale flood forecasting with
priori information of the Saint Venant Equations. It comprises an encoded structure
of physics law with the inclusion of PDE loss function, which introduces the priori
information into the system of neural network explicitly. Numerical simulations of
the Saint Venant Equations are performed to generate synthetic datasets to establish
and evaluate the performance PINNs.
2 Methodology
Rumelhart et al. [26] were the first to apply the backpropagation algorithm to a
multilayer neural network. Backpropagation is the backward propagation of error for
the training of a feed-forward neural network. At the end of feed-forward propagation,
the error between predicted and target output is determined with a loss function.
The objective of training with backpropagation is to reduce the loss function to a
minimum. It is achieved by gradually adjusting the weights in a backward direction.
A chain rule is used to approximate the derivative of the loss function with respect
L is the loss function, wi, j
to every weight in the network as given by Eq. (2), where
are the weights for node (i, j), yi is the output, and wi xi is the weighted sum of
input node and weights.
∂L ∂L ∂ yi ∂ wi xi
= . . (2)
∂wi, j ∂ yi ∂ wi xi ∂wi, j
80 R. B. Mahesh et al.
∂ Lt
wi,t+1
j = wi, j − η
t
(3)
∂wi,t j
νt = β1 νt−1 − (1 − β1 ) ∗ gt (4)
vt
wt = −η √ ∗ gt (6)
st + ∈
In this work, the implemented model is a Keras (2.2.4 version) sequential model
that allows for stacking multiple layers in sequential order. A fully connected network
is created with one input layer, three hidden layers, and one output layer. The shape of
the input is five unless specified and output neurons are two. The model is compiled
after defining the structure and the hyperparameter setting. The hyperparameter
configuration is based on (i) hyperparameter tunning through grid search of neurons
(5, 15, 25), (ii) industry expertise, (iii) published research [19, 28], or (iv) default
values of the library. An overview of the parameter and hyperparameter setting for
this work is tabulated in Appendix 1. The parameters in the Saint Venant Equations
(Eq. (8)), along with boundary discharge, were used as the input information. The
flow of information is from five input neurons with data labels as boundary discharge
(q, m3 /s), manning’s friction coefficient (n), slope (S), time (t, s), and distance (x,
m). The vector of these input data is passed through the input layer, hidden layers,
and finally to the output layer with two outputs: water depth (h, m) and velocity (v,
m/s), creating an encoded structure ANN. Apart from this encoded architecture, a
basic architecture is created to evaluate the significance of the encoded structure.
This basic architecture has one input neuron, i.e., boundary discharge (q), and two
output neurons: h and u.
Physics Informed Neural Network for Spatial-Temporal Flood Forecasting 81
Physics Informed Neural Network (PINN) is a deep learning technique for solving
PDE. A neural network is built that minimizes the loss function with the priori infor-
mation encoded. Priori information acts as a regularizing agent by narrowing the true
solution space, and also, the encoded structure quickly steers toward the right solu-
tion. As a priori information, the PDE is introduced in the neural network by imple-
menting a loss function leveraging the Automatic Differentiation (AD) [1] technique
to determine its derivatives. AD works on the principle of augmenting the standard
computation with derivative computation. This leads to a numerical computation
consisting of operators whose derivatives are known. An overall derivative is deter-
mined by combining the operators with the chain rule. The gradients are calculated
using the Kgradients from the Keras library.
The total loss function of the PINNs includes regression loss and a PDE loss.
The PDE loss is implemented such that the equation reduces to zero when the error
is zero. The Saint Venant Equations with source terms given in Eq. (8) [4, 20, 27,
34] is implemented as the PDE loss, where S0 is the bed slope, S f is the friction
term, and n is the manning’s coefficient. The constructed loss function is given by
∂u i ∂u i ∂h i
+ ui +g − gS0 + gS f = 0 (8)
∂t ∂x ∂x
n 2 u i2
S0 = −S Sf = (9)
4/3
hi
1 2
N
ui − ui
MSEu = (11)
N i=1
1 2
N
MSE f = h i − h i (12)
N i=1
1 ∂u i
N
∂u i ∂h i
MSEPDE = | + ui +g − gS0 + gS f |2 (13)
N i=1 ∂t ∂x ∂x
82 R. B. Mahesh et al.
Fig. 1 A schematic representation of physics informed neural networks (PINNs) for solving Saint
Venant equations with combined loss function
The synthetic dataset used to train, validate, and test the neural networks are gener-
ated with FullSWOF-2D (Full Shallow Water Equation for Overland Flows) [4].
Firstly, a study area of 1000 m is set up in FullSWOF-2D with the input boundary
condition synthesized from the SCS Hydrograph method [30]. Only 14 Direct Runoff
Hydrographs resulting from the best combination of effective rainfall duration (1–
6 m) and rainfall depth (1–8 cm) are selected. The selection is made such that the
peak discharges (q p ) are between the range of 2–18 m3 /s, and it is spread over low
(2–5 m3 /s), medium (6–14 m3 /s), and high (15–18 m3 /s) flow. The data is generated
by running the simulations in FullSWOF-2D for 34 flood scenarios designed by
creating a combination of the three stream characteristics: slope, friction coefficient,
and peak discharge. Only the necessary features as that of the Saint Venant Equations
are extracted and split into training, validation, and test dataset at 1 s timestep interval
and 500 grid points. The validation and test dataset remains the same for all model
except for the dataset with varied slope.
The datasets are designed to meet the requirement of different model analyses.
The original training dataset (Appendix 3) has 8 flood scenarios respecting the split
rule of ~70% of the entire dataset while the rest ~30% made up validation dataset
with 3 flood scenarios. The test dataset is carefully designed with 6 different flood
scenarios. In the original training dataset, the 8 flood scenarios are designed by
varying 3 slope values (1/100, 1/500, 1/1000) with 3 friction coefficients (0.025,
0.035, 0.045), resulting in 8 combinations (except one scenario with Fr > 1) for
Physics Informed Neural Network for Spatial-Temporal Flood Forecasting 83
which low, medium, and high peak discharges are assigned. Larger timesteps (3 s,
5 s) in addition to the original 1 s timestep were explored to find a feasible larger
timestep for the PINN in order to reduce the training time.
The model’s performance is measured through accuracy. The metrics used in this
work are Mean Squared Error (MSE), Coefficient of Determination (R2 ), and Nash
Sutcliffe Efficiency (NSE) [13]. MSE is the squared absolute error between the output
and the target values. It is given by Eq. (14), where N is the number of training
samples, di are the target values, and yi is the output values. R2 is used to evaluate
the linear correlation between target and output values. It signifies the goodness
of fit of the model. It is given by Eq. (15), where d is the mean of target values
and y is the mean of output values. NSE metric is used for hydrological prediction
evaluation; it takes into consideration different catchment dynamics. It is determined
using Eq. (16).
1
N
MSE = (di − yi )2 (14)
N i=1
⎡ ⎤2
N
⎢ i=1 di − d yi − y ⎥
R2 = ⎢
⎣
⎥ (15)
N
2 N 2 ⎦
i=1 di − d i=1 yi − y
N
i=1 (di − yi )2
NSE = 1 − (16)
N 2
i=1 di − d
The model architectures in this work consist of two outputs (h, u). For each
output, an average metric (MSE, NSE, R2 ) is computed by taking the average of 6
flood scenarios in the test dataset. The model’s overall performance is evaluated as
an average of averaged metric values, i.e., average MSE = 0.5 * (average MSE of u
+ average MSE of h). Similarly, the average R2 and average NSE is determined.
84 R. B. Mahesh et al.
Table 1 ANN 1 and ANN 2 model performance on evaluation metrics for test dataset
Model Avg. MSE Avg. h MSE Avg. u MSE Avg. R2 Avg. NSE
ANN 1 0.3771 0.4824 0.2719 0.70 0.44
ANN 2 0.0554 0.0785 0.0323 0.98 0.93
PINN model was implemented to constrain the neural network in the solution search
space of the Saint Venant Equations. The ANN 2 was extended to PINN with the
implementation of a custom loss function of PDE loss + MSE loss, along with the
best parameters from grid search (Appendix 2). The results are shown in Table 2.
The average MSE of PINN is 0.0535. PINN has h average MSE lower than u average
Physics Informed Neural Network for Spatial-Temporal Flood Forecasting 85
MSE, and the model has very good NSE (0.92) and accuracy (R2 = 0.97). Figure 2
shows good predictions for water height level and velocity for the best performing
test flood scenario with the PINN model.
On average MSE, PINNs perform better than ANN 2. The h average MSE is better
than u average MSE, contrary to the result in ANN 2. This shift might be attributed
to the PDE loss as it contains depth-averaged terms. Since these terms are a part of
the loss function, the model is well trained to predict water depths, and this is also
an indication of the constraints successfully imposed by the PDE loss. A significant
amount of improvement was seen using ANN 2, where the network’s encoded input-
output structure had considerable influence on improving the results. In comparison,
the physics loss function could introduce a minimal improvement (0.2%) over ANN
2. A 28% increase is seen in the case for predictions for h, and a 58% decrease in
velocity predictions was observed. The physics constraint works well for predicting
h; therefore, PINNs could be potentially used to predict water depths as the main
output.
Fig. 2 Comparison of predicted versus actual water height level and velocity for a test flood scenario
(S = 1/1000, n = 0.040, qp = 6 m3 /s) at distance x = 1, x = 501, and x = 999 for PINN
86 R. B. Mahesh et al.
Table 3 Different PINN model performance on evaluation metrics for test dataset
Model Avg. MSE Avg. h MSE Avg. u MSE Avg. R2 Avg. NSE
PINN 3s 0.0913 0.1041 0.0784 0.93 0.86
PINN 5s 0.1219 0.1317 0.1122 0.90 0.78
Timestep sensitivity analysis was performed to check the stability of the PINN at
larger timesteps. The original training dataset was truncated for every value at 3 s
and 5 s interval to develop PINN 3s and PINN 5s models. The model’s performance
is tabulated in Table 3. PINN 3s model had a 71% decrease, and PINN 5s model had
>100% performance decreased compared to PINN. The results validate that the 1 s
timestep was ideal for this work. Compared to timesteps in the numerical solvers,
the 1 s timestep is still relatively large and contributes to less computational effort in
PINNs.
The different model results are presented in the above sections, and to be able to
compare the models’ overall performance, the average MSE of the six test scenarios
is considered. The average MSE for each model is represented in the form of box
plots. Figure 3a represents the model performance on velocity forecasting, and Fig. 3b
represents the model performance on water height forecasting. Overall model perfor-
mance for different models is shown in Fig. 3c. The model comparison was made
to determine the best performing model, including the timestep sensitivity analysis.
The best performing model on average MSE metric is PINN, and it can be observed
there is performance improvement due to the reduction of error from water height
predictions, as shown in Fig. 3b.
4 Conclusion
In this work, Physics Informed Neural Network (PINN) was established to spatially
and temporally forecast floods on a 1D channel. It consists of neural network archi-
tecture, which is encoded with physics or a priori information, in this case, the Saint
Venant Equations in the form of a PDE loss function. The automatic differentiation
technique is used to calculate the derivatives for the PDE loss to facilitate the training
process.
Physics Informed Neural Network for Spatial-Temporal Flood Forecasting 87
Fig. 3 Box plot for different established models for a average u MSE metric, b average h MSE
metric, and c average MSE; average of u + h
Synthetic datasets containing physics information were generated using the SCS
Hydrograph Method and FullSWOF-2D to train data driven models. The workflow in
this work was modeling from a simple case to a more complex scenario. Foremost,
ANN 1 with just q as input was modeled with MSE loss, and later ANN 2 with
inputs (q, x, t, S, n) were modeled with outputs h and u. It was observed that ANN
2 has much lower MSE loss than ANN 1, indicating that the neural network learns
better with encoded structure, which consisted of the discharge as an input boundary
conditions.
The physics constraint was added to the encoded structure through the PINNs
modeled using the combined MSE and PDE loss. The performance of PINN was
better than the earlier established ANN 2. The majority of performance improvement
was introduced by the encoded structure in comparison to the combined loss function.
Better performance was seen for water depth forecasting in PINN than ANN 2 due
to the depth-averaged terms in the PDE Loss. This confirms that PDE loss was being
imposed positively to constrain the solution search space respecting the governing
equations.
PINN with 1 s timestep performed better in a timestep analysis while the perfor-
mance decayed with the increase in timestep (PINN 3s & PINN 5s). However, the 1 s
timestep is larger than the timesteps generally used in the numerical solver, which
help reduce the computational effort for PINN.
88 R. B. Mahesh et al.
Through this work, it is concluded that flood forecasting can be performed with
neural networks constrained with Saint Venant Equations through a physics informed
neural network architecture resulting in improved performance.
Appendix
Attribute Value
Activation function ReLU
Number of layers 3 [19]
Number of neurons Gird search; (n1, n2, n3)
Number of epochs Early stopping
Learning rate 0.0001
Early stopping patience 1
Batch size 128
Optimizer Adam
Loss function MSE
References
1. Baydin AG, Pearlmutter BA, Radul AA, Siskind JM (2017) Automatic differentiation in
machine learning: a survey. J Mach Learn Res 18(1):5595–5637
2. Baymani M, Effati S, Niazmand H, Kerayechian A (2015) Artificial neural network method
for solving the Navier–Stokes equations. Neural Comput Appl 26(4):765–773
3. Bhola PK, Nair BB, Leandro J, Rao SN, Disse M (2019) Flood inundation forecasts using
validation data generated with the assistance of computer vision. J Hydroinf 21(2):240–256
4. Delestre O, Darboux F, James F, Lucas C, Laguerre C, Cordier S (2017) FullSWOF: full
shallow-water equations for overland flow. J Open Source Softw 2(20):448. https://doi.org/10.
21105/joss.00448
5. Devia GK, Ganasri BP, Dwarakish GS (2015) A review on hydrological models. Aquat Procedia
4:1001–1007
6. Dockhorn, T. (2019) A discussion on solving partial differential equations using neural
networks. ArXiv Preprint arXiv:1904.07200
7. Elsafi SH (2014) Artificial neural networks (ANNs) for flood forecasting at Dongola station in
the River Nile Sudan. Alex Eng J 53(3):655–662
8. Fotiadis S, Pignatelli E, Valencia ML, Cantwell C, Storkey A, Bharath AA (2020) Comparing
recurrent and convolutional neural networks for predicting wave propagation. ArXiv Preprint
arXiv:2002.08981
90 R. B. Mahesh et al.
9. Géron A (2019) Hands-on machine learning with Scikit-Learn, Keras, and TensorFlow:
concepts, tools, and techniques to build intelligent systems. O’Reilly Media, Sebastopol
10. Goodfellow I, Bengio Y, Courville A, Bengio Y (2016) Deep learning, vol 1. MIT Press,
Cambridge
11. Haehnel P, Mareček J, Monteil J, O’Donncha F (2020) Using deep learning to extend the range
of air pollution monitoring and forecasting. J Comput Phys 408:109278
12. Jain SK, Mani P, Jain SK, Prakash P, Singh VP, Tullos D, Kumar S, Agarwal SP, Dimri AP
(2018) A brief review of flood forecasting techniques and their applications. Int J River Basin
Manag 16(3):329–344
13. Kao I-F, Zhou Y, Chang L-C, Chang F-J (2020) Exploring a long short-term memory based
encoder-decoder framework for multi-step-ahead flood forecasting. J Hydrol 124631
14. Karimpouli S, Tahmasebi P (2020) Physics informed machine learning: seismic wave equation.
Geosci Front 11(6):1993–2001
15. Kasiviswanathan KS, He J, Sudheer KP, Tay J-H (2016) Potential application of wavelet neural
network ensemble to forecast streamflow for flood management. J Hydrol 536:161–173
16. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. ArXiv Preprint arXiv:
1412.6980
17. Kratzert F, Klotz D, Brenner C, Schulz K, Herrnegger M (2018) Rainfall–runoff modelling
using long short-term memory (LSTM) networks. Hydrol Earth Syst Sci 22(11):6005–6022
18. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444
19. Lin Q, Leandro J, Wu W, Bhola P, Disse M (2020) Prediction of maximum flood inundation
extents with resilient backpropagation neural network: case study of Kulmbach. Front Earth
Sci 8:332
20. Martins R, Leandro J, Djordjević S (2018) Wetting and drying numerical treatments for the
Roe Riemann scheme. J Hydraul Res 56(2):256–267
21. Moriasi DN, Arnold JG, Van Liew MW, Bingner RL, Harmel RD, Veith TL (2007) Model
evaluation guidelines for systematic quantification of accuracy in watershed simulations. Trans
ASABE 50(3):885–900
22. Mujumdar PP (2001) Flood wave propagation. Resonance 6(5):66–73
23. Qian K, Mohamed A, Claudel C (2019) Physics informed data driven model for flood prediction:
application of deep learning in prediction of urban flood development. ArXiv Preprint arXiv:
1908.10312
24. Raissi M, Karniadakis GE (2018) Hidden physics models: machine learning of nonlinear partial
differential equations. J Comput Phys 357:125–141
25. Raissi M, Perdikaris P, Karniadakis GE (2019) Physics-informed neural networks: a deep
learning framework for solving forward and inverse problems involving nonlinear partial
differential equations. J Comput Phys 378:686–707
26. Rumelhart DE, Hinton GE, Williams RJ (1985) Learning internal representations by error
propagation
27. de Saint-Venant AJC (1871) Theorie du mouvement non permanent des eaux, avec application
aux crues des rivieres et a l’introduction de marees dans leurs lits. Comptes Rendus Des Seances
de l’Academie Des Sciences 36:154–174
28. Scarselli F, Tsoi AC (1998) Universal approximation using feedforward neural networks: a
survey of some existing methods, and some new results. Neural Netw 11(1):15–37
29. Sene K (2012) Flash floods: forecasting and warning. Springer Science & Business Media,
Berlin
30. Singh PK, Mishra SK, Jain MK (2014) A review of the synthetic unit hydrograph: from the
empirical UH to advanced geomorphological methods. Hydrol Sci J 59(2):239–261
31. Sit M, Demir I (2019) Decentralized flood forecasting using deep neural networks. ArXiv
Preprint arXiv:1902.02308
32. Sorteberg WE, Garasto S, Cantwell CC, Bharath AA (2019) Approximating the solution
of surface wave propagation using deep neural networks. INNS big data and deep learning
conference, pp 246–256
33. Stephen I (1990) Perceptron-based learning algorithms. IEEE Trans Neural Netw 50(2):179
Physics Informed Neural Network for Spatial-Temporal Flood Forecasting 91
34. Zhang R, Zen R, Xing J, Arsa DMS, Saha A, Bressan S (2020) Hydrological process surrogate
modelling and simulation with neural networks. In: Pacific-Asia conference on knowledge
discovery and data mining, pp 449–461
35. Zhao Y, Akolekar HD, Weatheritt J, Michelassi V, Sandberg RD (2020) RANS turbulence
model development using CFD-driven machine learning. J Comput Phys 109413