See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/350146453

Physics-Informed Neural Networks (PINNs) for Heat Transfer Problems

Article in Journal of Heat Transfer · March 2021

DOI: 10.1115/1.4050542

CITATIONS READS
221 12,286

5 authors, including:

Shengze Cai Zhicheng Wang

Zhejiang University Dalian University of Technology
30 PUBLICATIONS 1,453 CITATIONS 34 PUBLICATIONS 1,399 CITATIONS

SEE PROFILE SEE PROFILE

Sifan Wang Paris Perdikaris

University of Pennsylvania University of Pennsylvania
23 PUBLICATIONS 2,503 CITATIONS 113 PUBLICATIONS 12,866 CITATIONS

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Machine Learning, AFOSR and DARPA View project

Numerical Simulation of Long Slender Offshore Structures View project

All content following this page was uploaded by Zhicheng Wang on 26 March 2021.

The user has requested enhancement of the downloaded file.

Journal of Heat Transfer. Received February 08, 2021;
Accepted manuscript posted March 17, 2021. doi:10.1115/1.4050542
Copyright (c) 2021 by ASME

Physics-Informed Neural Networks (PINNs) for

Heat Transfer Problems

Shengze Cai∗ Zhicheng Wang∗

Division of Applied Mathematics Division of Applied Mathematics

Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/doi/10.1115/1.4050542/6665384/ht-21-1083.pdf by Massachusetts Inst Of Tech. user on 26 March 2021

Brown University Brown University

d
Providence, Rhode Island 02912 Providence, Rhode Island 02912
Email: shengze [email protected] Email: zhicheng [email protected]

ite
Paris Perdikaris†

ed
Sifan Wang
Graduate Group in Applied Department of Mechanichal Engineering
Mathematics and Computational Science and Applied Mechanics

py
University of Pennsylvania University of Pennsylvania
Philadelphia, Pennsylvania 19104 Philadelphia, Pennsylvania 19104
Email: [email protected] Email: [email protected]

Co
George Em Karniadakis‡
Division of Applied Mathematics and School of Engineering

ot
Brown University
Providence, Rhode Island 02912
tN
Email: george [email protected]
ipr

Physics-informed neural networks (PINNs) have gained tive use of neural networks in solving general heat transfer
sc

popularity across different engineering fields due to their problems of industrial complexity. Taken together, the results
effectiveness in solving realistic problems with noisy data presented herein demonstrate that PINNs not only can solve
nu

and often partially missing physics. In PINNs, automatic ill-posed problems, which are beyond the reach of traditional
differentiation is leveraged to evaluate differential operators computational methods, but they can also bridge the gap be-
without discretization errors, and a multi-task learning prob- tween computational and experimental heat transfer.
Ma

lem is defined in order to simultaneously fit observed data

while respecting the underlying governing laws of physics.
Here, we present applications of PINNs to various proto- Nomenclature
type heat transfer problems, targeting in particular realistic ANN Artificial neural network
ed

conditions not readily tackled with traditional computational CFD Computational fluid mechanics
methods. To this end, we first consider forced and mixed con- CNN Convolutional neural network
pt

vection with unknown thermal boundary conditions on the DPIT Digital particle image thermometry
heated surfaces and aim to obtain the temperature and ve- PINN Physics-informed neural network
locity fields everywhere in the domain, including the bound-
ce

PDE Partial differential equation

aries, given some sparse temperature measurements. We also TSP Temperature-sensitive paint
consider the prototype Stefan problem for two-phase flow,
Ac

aiming to infer the moving interface, the velocity and tem- Physics:
perature fields everywhere as well as the different conductiv- x Spatial coordinate (x, y)
ities of a solid and a liquid phase, given a few temperature t Temporal coordinate
measurements inside the domain. Finally, we present some u Velocity in x-direction
realistic industrial applications related to power electronics v Velocity in y-direction
to highlight the practicality of PINNs as well as the effec- p Pressure
θ Temperature
Re Reynolds number
∗ Contributed equally.
Pe Péclet number
† Address all correspondence for other issues to this author. Ri Richardson number
‡ Address all correspondence for other issues to this author.
Journal of Heat Transfer. Received February 08, 2021;
Accepted manuscript posted March 17, 2021. doi:10.1115/1.4050542
CopyrightRa(c) 2021 by ASME
Rayleigh number 1 Introduction
k Thermal diffusivity parameter The application of machine learning techniques to heat
s Moving surface transfer problems can be dated back to 1990s, when artifi-
cial neural networks (ANN) were used to learn the convec-
Neural network:
tive heat transfer coefficients [1] from data. In recent years,
xi Neuron input
more advanced learning-based methods have been developed
y j Neuron output
also aided by the improvement of the appropriate hardware,
α Neuron amplitude for for adaptive activation function
e.g., GPU technology, and have been beneficial to various
b Neuron bias
heat transfer problems [2–4]. In particular, the convolutional
w Neuron weight
neural network (CNN) - a widely-used deep learning tech-

Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/doi/10.1115/1.4050542/6665384/ht-21-1083.pdf by Massachusetts Inst Of Tech. user on 26 March 2021

β All parameters in the first neural network (Section 4)
nique - has been successfully employed to make predictions
θ All parameters in the second neural network (Section 4)

d
from image-like data in complex and high-dimensional prob-
σ Activation function
lems [5–8]. For example, a CNN proposed in [8] was used

ite
L Loss function
to predict the local heat flux of turbulent channel flows by
N Number of training points
feeding it with the wall-shear stress and the wall pressure.
λ Weighting coefficient in loss function

ed
In other work, a CNN based on the U-net [9] architecture,
employed a 2D slice of time-averaged temperature field as
input to predict a ridge pattern, which was used to quantify
CONTENTS

py
the fraction and time variability of turbulent heat transfer [5].
1 Introduction 2 The aforementioned methods are mostly based on the super-
vised learning strategy, where a database with labels (i.e.,

Co
2 Overview of Physics-informed Neural Networks 3 ground-truth) is required in order to train the model. The
advantage of such data-driven methods is that once trained,
3 Convection Heat Transfer: Unknown Thermal the prediction procedure will be vary fast. However, the data

ot
Boundary Conditions 4 generation is a big issue and the generalization (from one ex-
3.1 Forced Convection . . . . . . . . . . . . . . 4 perimental condition to another) of the model is not guaran-
tN
3.1.1 Forced convection in an enclosure . . 4 teed. In addition to the supervised learning, other approaches
3.1.2 Flow past cylinder . . . . . . . . . . 5 have been taken into account. For instance, an unsupervised
3.1.3 Active sensor placement . . . . . . . 6 learning strategy was applied for a conjugate thermal opti-
ip

3.1.4 Results . . . . . . . . . . . . . . . . 6 mization problem [10]. More recently, deep reinforcement

3.2 Mixed Convection . . . . . . . . . . . . . . . 9 learning (RL) has also been applied to control thermal sys-
r

tems; for example, [11] showed that RL-based control is

sc

4 Two-Phase Stefan Problems 11 able to stabilize the conductive regime and bring the onset
of convection up to a Rayleigh number Ra = 3 × 104 , outper-
5 Applications to Power Electronics 13 forming the state-of-the-art linear controllers. Also, [12] ap-
nu

5.1 Heat transfer in electronic chips . . . . . . . 13 plied RL to several natural and forced convection problems,
5.2 SimNet for heat sink design . . . . . . . . . . 15 demonstrating that their RL algorithms can alleviate the heat
Ma

transfer enhancement related to the onset of convection.

6 Summary and Outlook 15
The aforementioned efforts do not directly take into ac-
count the underlying physics of heat transfer problems. To
tackle this problem, a multi-task learning approach is re-
ed

quired, such as the framework of physics-informed neural

networks (PINNs). This approach was first proposed for
solving both forward and inverse problems described by a
pt

combination of some data and of partial differential equa-

tions (PDEs), and subsequently it was applied to various
ce

fluid mechanics problems [13–16] as well as heat transfer

problems [16–20]. For instance, motivated by the limited
Ac

understanding of the physical mechanisms responsible for

the heat transfer enhancement in rough turbulent Rayleigh-
Bénard convection, the authors in [19] applied the PINN
framework to predict turbulent transport at Ra = 2 · 107 in
a sub-domain of a Rayleigh-Bénard cavity filled with water
and bearing two square-based roughness elements placed on
the hot plate. PINNs training benefited from a large direct
numerical simulations database. The influence of the choice
of the data acquisition and residuals sampling, in relation to
the problem geometry and initial/boundary conditions, was
Journal of Heat Transfer. Received February 08, 2021;
Accepted manuscript posted March 17, 2021. doi:10.1115/1.4050542
Copyrightalso
(c) reported
2021 by [19].
ASME Nvidia also has developed the code Sim- tions of partial differential equations (PDEs), expressed as:
Net based on PINNs and applied it to solve multiphysics
problems involving heat transfer of a FPGA heat sink inside ut + Nx [u] = 0, x ∈ Ω, t ∈ [0, T ],
a channel [16]; see also section 5.2 below. Moreover, PINNs
u(x, 0) = h(x), x ∈ Ω, (1)
can be employed to quantify the flow fields of natural con-
vection from data measurements, as demonstrated in [18]. u(x,t) = g(x,t), x ∈ ∂Ω, t ∈ [0, T ],
In this paper, the authors inferred the flow fields based on
a number of temperature as well as velocity measurements, where Nx is a general linear or nonlinear differential oper-
and investigated the influence of the data selection. The flexi- ator; x ∈ Rd and t are the spatial and temporal coordinates,
bility offered by PINNs has also been leveraged to tackle free respectively; Ω and ∂Ω denote the computational domain and

Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/doi/10.1115/1.4050542/6665384/ht-21-1083.pdf by Massachusetts Inst Of Tech. user on 26 March 2021

boundary and Stefan problems; a general class of problems the boundary; u(x,t) is the solution of the PDEs with initial
condition h(x) and boundary condition g(x,t). We remark

d
for which the evolution of unknown boundaries and moving
interfaces has to be estimated concurrently with solving the that this formulation can be easily generalized to higher or-

ite
underlying PDE. For example, Wang et al. [21] have demon- der PDEs since they can be written as systems of first order
strated the effectiveness of PINNs in solving both forward PDEs.
Following the framework of PINNs proposed in [23],

ed
and inverse problems Stefan problems involving multi-phase
interfaces ; see also section 4 below. More recently, the u(x,t) is approximated by a fully-connected network, which
PINNs algorithm was applied to quantify the velocity and takes the coordinates (x,t) as inputs and outputs uNN (x,t).

py
pressure fields of natural convection over an espresso cup The neural network is composed of multiple hidden layers,
from temperature data obtained by a background oriented where the inputs of each hidden layer (X = [x1 , x2 , . . . , xi ])
schlieren experiment [20], indicating that the method can and outputs (Y = [y1 , y2 , . . . , y j ]) are propagated through the

Co
be successfully used to deal with real experimental data. In network as
addition to PINNs, an unsupervised learning approach with
auto-encoder and image gradient was proposed in [22] to y j = σ(wi, j xi + b j ), (2)

ot
solve the heat equations on a chip. The principle of this
method is similar to PINNs. However, by training the net- where wi, j and b j are trainable weights and biases, respec-
tN
work with a set of data, the proposed framework can also tively; σ(·) is the activation function representing a simple
generalize the trained network for predicting solutions for nonlinear transformation. The parameters of the network can
heat equations with unseen source terms. These algorithms, be trained by minimizing a composite loss function taking
ip

which aim to solve the heat equations, introduce the physical the form
models to the neural networks, and train the model by min-
r

imizing a loss function involving the residuals of the gov-

L = Lr + Lb + L0 , (3)
sc

erning equations. Therefore, the solutions can be determined

with limited data or without any data except for the boundary
and initial conditions. where
nu

In this paper, we mainly review the applications of 1 Nr 2

PINNs on inverse heat transfer problems in forced and mixed Lr = ∑ ut (xi ,t i ) + Nx [u(xi ,t i )] (4a)
Ma

Nr i=1
convection and on the two-phase Stefan problem with a mov-
ing interface. We also include two examples from industry 1 Nb 2
on how to use PINNs in the thermal design of power elec-
Lb = ∑ u(xi ,t i ) − gi
Nb i=1
(4b)
ed

tronics. The paper is organized as follows. In the next sec-

1 N0 2
tion we provide an overview of PINNs, and in section 3 we L0 = ∑ u(xi ,t i ) − hi .
N0 i=1
(4c)
present two prototype problems of forced and mixed convec-
pt

tion. In section 4 we present the formulation of PINNs for the

two-phase Stefan problem with some illustrative results, and Here Lr , Lb and L0 penalize the residuals of governing equa-
ce

in section 5 we present two industrial applications of PINNs tions, the boundary conditions and the initial conditions, re-
to power electronics by Ansys and Nvidia. We conclude in spectively; Nr , Nb and N0 are the numbers of data points for
different terms. Notice that all loss terms are a function of
Ac

section 6 with a brief summary and outlook.

the network weight and bias parameters, wi, j and b j , respec-
tively, the dependence on which has been omitted to favor
notation simplicity. In addition, if some data are available in-
side the domain, an extra loss term indicating the mismatch
between the predictions and the data can be taken into ac-
count:
2 Overview of Physics-informed Neural Networks
Nd
In the context of physics-informed neural networks 1 2
Ldata = ∑ u(xi ,t i ) − uidata . (5)
(PINNs), a neural network is used to approximate the solu- Nd i=1
Journal of Heat Transfer. Received February 08, 2021;
Accepted manuscript posted March 17, 2021. doi:10.1115/1.4050542
CopyrightIn(c) 2021toby
order ASME the residuals for L , derivatives of the here by asking the simple question: what if we have some
compute r
outputs with respect to the inputs (i.e., ut and Nx [u]) are re- temperature measurements at a few points, e.g., using sim-
quired. Such computation is achieved in the PINN frame- ple thermocouples, in convenient locations that may or may
work by using automatic differentiation in the deep learn- not include the heated surface with the unknown boundary
ing code. Automatic differentiation relies on the fact that condition. Can we then solve an inverse problem by utilizing
combining the derivatives of the constituent operations by both the measurements and the governing equations of flow
the chain rule gives the derivative of the overall composi- and heat transfer. In a typical CFD set up, this would require
tion. This technique is a key enabler for the development a tedious data assimilation method [31, 32] blended in with
of PINNs, and is the key element that differentiates PINNs flow and heat transfer solvers and may take extremely long
from similar efforts in the early 90s [24, 25], which relied on times to converge, if it could converge at all. Here, inspired

Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/doi/10.1115/1.4050542/6665384/ht-21-1083.pdf by Massachusetts Inst Of Tech. user on 26 March 2021

manual derivation of back-propagation rules. Nowadays, au- by the development of PINNs, we exploit the expressivity

d
tomatic differentiation capabilities are well-implemented in of deep neural networks to formulate this ill-posed problem,
most deep learning frameworks such as TensorFlow [26] and blending seamlessly data and mathematical models while si-

ite
PyTorch [27], and it allows us to avoid tedious derivations or multaneously inferring both the flow and temperature fields.
numerical discretization while computing derivatives of all

ed
orders in space-time. In the following sections, the problem-
dependent governing equations, loss functions and the train- 3.1 Forced Convection
ing configurations of PINNs are described case by case. We first consider PINN applications to forced convec-

py
A schematic of the PINN framework is demonstrated in tion problems, which are commonly encountered in various
Figure 1, in which a simple heat equation ut = αuxx is used industrial systems. The governing equations of this problem
as an example to show how to set up a PINN for heat transfer are the incompressible Navier-Stokes equations and the cor-

Co
problems. As shown in Figure 1(a), the fully-connected neu- responding temperature equation:
ral network is used to approximate the solution u(x,t), which
is then applied to construct the residual loss Lr , boundary ∂θ 1
+ (u · ∇)θ = ∇2 θ,

ot
conditions loss Lb and initial conditions loss L0 . The pa- ∂t Pe
rameters of the fully-connected network are trained by using 1
∂u (6)
+ (u · ∇)u = −∇p + ∇2 u + Riθ,
tN
gradient-descent methods based on the back-propagation of
∂t Re
the loss function. We also demonstrate the data points used
∇ · u = 0,
for different terms of the loss function, as shown in Figure
ip

1(b). Note that the residual points for computing the residual
loss Lr can be randomly selected in the space-time domain where θ, u = (u, v)T and p are the dimensionless tempera-
r

and the numbers of points can be defined by the users. ture, velocity and pressure fields, respectively. Pe, Re and
sc

Ri denote the Peclet, Reynolds and Richardson number, re-

spectively. Note that for forced convection discussed in
3 Convection Heat Transfer: Unknown Thermal
nu

this section, Ri = 0. Given the boundary and initial condi-

Boundary Conditions tions, simulating the forced convection problems with stan-
In standard heat transfer handbooks and textbooks, one dard CFD techniques is relatively simple. However, here we
Ma

can find explicit expressions of the Nusselt number as a func- aim to demonstrate a new proof-of-concept of inferring the
tion of the Reynolds and Prandtl numbers, for several proto- entire temperature field from very sparse temperature mea-
typical heat transfer problems, e.g., inside and around pipes, surements. In particular, unlike the well-posed simulation
boundary layers, and other surfaces given that the thermal problem, which explicitly defines the thermal boundary con-
ed

boundary condition is precisely specified, i.e., constant tem- ditions on all boundaries of the domain, the problem we fo-
perature or constant heat flux, e.g., see [28–30]. Moreover, cus on is an ill-posed problem, where the boundary condi-
at the present time, standard CFD methods can simulate ar-
pt

tions are not fully known and have to be discovered from

bitrary thermal boundary conditions, e.g., the mixed Robin very few measurements while simultaneously infer the en-
boundary condition, or arbitrary temperature distributions tire temperature field everywhere in the domain. Moreover,
ce

imposed on the heated surface with very good accuracy and we can also infer the velocity and pressure fields everywhere
on industrial complexity geometric domains. However, in in the domain. This can be considered as an inverse problem,
Ac

real heat transfer applications, e.g., in power electronics or which can be addressed by PINNs. To demonstrate the PINN
in a nuclear reactor, unlike the velocity boundary conditions, effectiveness, we perform the forced convection simulations
the thermal boundary conditions are never precisely known in two different scenarios: heat transfer inside an enclosure,
as this would require an enormous and complex instrumenta- and heat transfer over a stationary cylinder, where the flows
tion, which is not feasible in industrial applications but only are both steady.
in a few cases in research experimental setups.
The lack of thermal boundary conditions leads to an
ill-posed boundary value problem for the energy equation, 3.1.1 Forced convection in an enclosure
which cannot be solved no matter how sophisticated the CFD We consider 2D steady forced convection heat transfer
method is. To this end, we address this ill-posed problem in an enclosure as shown in Figure 2; the two hot objects with
Journal of Heat Transfer. Received February 08, 2021;
Accepted manuscript posted March 17, 2021. doi:10.1115/1.4050542
Copyright (c) 2021 by ASME

Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/doi/10.1115/1.4050542/6665384/ht-21-1083.pdf by Massachusetts Inst Of Tech. user on 26 March 2021

d
ite
ed
Fig. 1: Overview of physics-informed neural networks (PINNs). (a) Schematic of PINN framework. A fully-connected
neural network is used to approximate the solution u(x,t), which is then applied to construct the residual loss Lr , boundary
conditions loss Lb and initial conditions loss L0 . The derivatives of u are computed by automatic differentiation in Ten-

py
sorFlow [26]. The parameters of the fully-connected network are trained by using gradient-descent methods based on the
back-propagation of the loss function. (b) Schematic of point selection for PINN in the computational domain. The points
with different colors correspond to different terms in the loss function. Note that the residual points for computing the

Co
residual loss can be randomly selected in the space-time domain.

ot
tN
r ip
sc
nu

Fig. 2: Forced convection in an enclosure. The boundaries in red color are at higher temperature, and the cooling inflow
enters from the left-bottom and exits at the right-top. The blue arrows show the velocity vectors. The temperature at the
Ma

boundaries EF and GH are not known. The green triangles represent the temperature probes while the red circles denote the
velocity probes.
ed

red boundaries in the enclosure are at temperature θ = 1; uni- data and the reference solution is obtained numerically us-
form cooling flow of u = 1, v = 0 enters from the left-bottom ing the spectral/hp element method [33]. The simulation
pt

and exits from the right-top boundary. The temperature on domain size is [−7.5 D, 22.5 D] × [−10 D, 10 D], consisting
the rest of the walls and inflow boundary is θ = 0. On the of 2094 quadrilateral elements, where D is the diameter of
ce

outflow boundary, ∂θn = 0. The governing equations are given the cylinder. The cylinder center is located at (0, 0). On
by (6) with Re = 50, Pe = 36 and Ri = 0.0. Note that here the fluid flow part, the cylinder surface is assumed to be no-
Ac

the height of AB is used as the characteristic length for the slip, no-penetration wall. Uniform velocity (u = U∞ , v = 0)
Reynolds number. The simulation started from initial condi- is imposed on the inflow boundary where x/D = −7.5, pe-
tions: u = 0, v = 0 and θ = 0 and stopped when both the flow riodic boundary condition is used on the lateral boundaries
and temperature reach steady state. where y/D = ±10, and zero-pressure boundary is prescribed
on the outflow boundary where x/D = 22.5. On the heat
transfer part, constant temperature θ = θ∞ = 0 is imposed
3.1.2 Flow past cylinder on the inflow boundary, periodic boundary is assumed on the
Here we consider the classical two-dimensional heat lateral boundaries and zero-gradient is employed on the out-
transfer problem of forced heat convection around a circu- flow boundary. For the cylinder surface, constant wall tem-
lar cylinder in steady state; we assume that we have some
Journal of Heat Transfer. Received February 08, 2021;
Accepted manuscript posted March 17, 2021. doi:10.1115/1.4050542
Copyrightperature
(c) 2021isby ASME as a simple example but any arbitrary where
considered
temperature distribution can be readily modeled too. The
governing equations of this problem are given in (6), with 1 4 Nr
the Reynolds number Re = 20 and Peclet number Pe = 200. Lr = ∑ ∑ |ek (xi , yi )|2
Nr k=1
(9a)
i=1
The simulation has been carried out until both flow and tem-
perature reach steady state. 1 Nub
Lub = ∑ [u(xi , yi ) − uib ]2
Nub i=1
(9b)

1 Nθb
Lθb = ∑ |θ(xi , yi ) − θb |2
Nθb i=1
(9c)
3.1.3 Active sensor placement

Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/doi/10.1115/1.4050542/6665384/ht-21-1083.pdf by Massachusetts Inst Of Tech. user on 26 March 2021

1 Nθ
In data-driven problems, it is often the case that the sen- Lθ = ∑ |θ(xi , yi ) − θidata |2 . (9d)

d
sor locations, where data are collected, may affect the in- Nθ i=1

ite
ferred results. To obtain the best sensor locations, it generally
requires trial-and-error, which is a costly process. Therefore, The first term Lr penalizes the governing equations (6), in-
we also propose a method to adaptively select the location cluding the heat equation, the momentum equations and the

ed
of sensors in order to minimize the number of temperature continuity equation. Here, Nr is the batch size of residual
measurements. This method allows us to start with a small points, which are randomly selected in the spatial domain.

py
number of fixed sensors (e.g., 4 or 5) and iteratively add more The second and third terms are the boundary conditions for
sensors at the essential locations. velocity and temperature fields, respectively. It should be
noted that the residual points for velocity boundary and tem-

Co
The criterion of adding sensors we propose is based on perature boundary can be different, thus two independent
the residual of the temperature equation in (6), i.e., terms are applied in the loss function. θb denotes the en-
vironmental temperature, which is known in practice. The
last term of the loss function Lθ is the mismatch between the

ot
e = uθx + vθy − Pe−1 (θxx + θyy ). (7) inferred temperatures and the in-situ measured values, e.g. at
the thermocouple locations. The value of Nθ depends on the
tN
sensor placement in different scenarios.
The parameters of the neural networks are randomly
The reasons of using this residual include: (a) this residual, initialized using the Glorot scheme [34] and trained by the
ip

which represents the error of the temperature equation, can Adam optimizer [35] with a decreasing learning rate sched-
be directly computed by the neural network and it does not ule. The results are obtained after 80, 000 iterations with de-
r

require any prior knowledge; (b) as we will show in the fol- creasing learning rates of 1 × 10−3 , 1 × 10−4 , 1 × 10−5 and
sc

lowing result, there is a correlation between this residual and 1 × 10−6 (20k iterations for each).
the posterior temperature error; (c) this can be considered as To investigate the performance of the proposed neural
nu

a sensitivity metric of the temperature with respect to the lo- network on inferring the temperature, velocity and pressure
cation (x, y). Taking the forced convection of flow past cylin- fields in the domain, the results obtained by PINNs are quan-
der as an example, the proposed method for adaptive sensor titatively evaluated against the reference solutions (provided
Ma

placement can be summarized as follows: by the high-order CFD method). We use the relative L2 er-
rors as the metric:

εV =k V −V ∗ k2 / k V ∗ k2 ,
ed

Step 1 Provide an initial sensor configuration. (10)

Step 2 Train the neural network (PINN).
Step 3 Compute the residual (7) on the cylinder boundary where V represents one of the predicted quantities (θ, u, v, p)
pt

inferred by PINN and find the position with maximum and V ∗ is the corresponding reference.
value of (7). Enclosure: We employ a fully-connected neural net-
ce

Step 4 Update the sensor placement and training data, and work with ten hidden layers and 120 neurons per layer. The
go back to Step 2. number of residual points for the three terms in loss func-
Ac

tion 8 are: Nr = 10736, Nub = 757 and Nθb = 548. Note that
the points for Lub include all the boundaries of the enclo-
sure since the velocity boundary conditions are always well
3.1.4 Results defined. As shown in Figure 2, the points of Nθb are uni-
For the steady forced convection problem, the PINN loss formly distributed in all other boundaries except at EF and
function is expressed as: GH. Moreover, in order to mimic the scenario in real life that
limited measurement data are available, here we assume that
the temperature at the positions of the 4 green triangles and
velocity at the positions of the 5 circles are known, see Fig-
L = Lr + Lub + Lθb + Lθ , (8) ure 2. The comparison between PINN inferred result and the
Journal of Heat Transfer. Received February 08, 2021;
Accepted manuscript posted March 17, 2021. doi:10.1115/1.4050542
Copyright (c) 2021 by ASME For the first case shown in the figure, there are only five
sensors: two on the cylinder surface and three in the wake.
Although the neural network can predict accurate solutions
of velocity and pressure, it fails to predict the temperature
field very accurately (εθ > 10%). As seen from Case 2,
adding one more sensor on the surface can dramatically im-
prove the results, showing that the information at the front
stagnation point of the cylinder is significant in this problem.
Case 3 is used to investigate another sensor placement in the
wake behind the cylinder, which differs from Case 2 and we

Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/doi/10.1115/1.4050542/6665384/ht-21-1083.pdf by Massachusetts Inst Of Tech. user on 26 March 2021

can find that it can further increase the accuracy. The second

d
group in Figure 4, including Case 4, 5 and 6, shows the fact
that we can obtain relatively good results when there is no

ite
temperature measurement on the cylinder surface. Increasing
the number of sensors around the cylinder can improve the

ed
inference performance. Moreover, compared with the third
group (Case 7, 8 and 9), we find that placing one more sensor
at the front stagnation point on the cylinder surface can help
Fig. 3: Forced convection in an enclosure: comparison of

py
the method to find a much more accurate solution. For ex-
θ, u, v and p inferred by PINN and the reference simulation
ample, the temperature error in Case 7 is only 1.25% while
results by the spectral element method (SEM).
that in Case 4 is 15.09%. It is worth noting that in all dif-

Co
ferent sensor configurations, the velocity and pressure fields
are accurately inferred by the PINN algorithm. The reason is
reference simulation result is shown in Figure 3. It could be that the boundary conditions of the velocity are well-defined,

ot
observed that PINN can predict the flow and temperature in thus the NS equations (which are independent to the heat
the whole domain well qualitatively. The L2 error of the tem- equation) can be well-resolved by PINNs [15].
tN
perature on the two unknown walls EF and GH is 10.82%. A typical example of the inferred results is demonstrated
Flow past cylinder: A fully-connected neural network in Figure 5, where the fields are inferred by PINN with sen-
with ten hidden layers and 200 neurons per layer is employed sor configuration Case 3. The point-wise absolute errors
ip

in PINNs to infer the solutions. The numbers of residual between CFD simulation and PINN results are also given,
points for different terms in loss function (8) are given as where we can see the magnitudes of the errors are very small
r

Nr = 20000, Nub = 900 and Nθb = 500. In particular, the compared to the magnitudes of inferring quantities. Figure
sc

points for computing Lub include the inlet, upper and lower 6 demonstrates the corresponding temperature and Nusselt
bounds, as well as the cylinder boundary. We should point number profiles on the cylinder surface. We observe that
out again that the boundary condition of circular cylinder for the temperature profile is approximately constant (equal to 1)
nu

velocity is known (no-slip) and that for temperature is un- and the Nusselt number looks consistent with the CFD solu-
known. The number of measurements (Nθ ) depends on case tion. The mean Nusselt numbers along the cylinder surface
Ma

by case and is specified in the following. In the following, of exact and predicted solutions are 5.90 and 5.89, respec-
we consider first the case of (unknown) prescribed tempera- tively. That results in an accurate Nusselt number prediction
ture and subsequently the case of prescribed heat flux on the with error smaller than 1%. Note that there are only six sen-
cylinder surface. sors used in Case 3. Using such limited measurements for
ed

Constant temperature surface: Nine different sensor prediction is an inverse problem, which is generally difficult
configurations are considered for the case with constant tem- to solve. In addition to the unknown boundary condition,
perature surface, which are illustrated in Figure 4. The num- the proposed neural network can simultaneously infer the
pt

ber of temperature measurements Nθ varies from 5 to 11. As velocity, pressure and temperature fields everywhere in the
shown in the figure, we only have a couple of sensors on the domain.
ce

cylinder and a couple more in the wake. Based on these ob- Constant flux surface: To discover the boundary con-
servations, the thermal boundary condition on the cylinder dition with constant heat flux (dθ/d~n = 0), we apply an ex-
Ac

surface is to be inferred. The relative L2 errors of temper- perimental setup with seven sensors (three on the boundary
ature, velocity and pressure fields computed over the whole and four in the wake domain), which is illustrated at the top-
domain are also provided in Figure 4, along with the sensor left corner in Figure 7. The corresponding temperature and
placements. We note that the training of the network param- Nusselt number profiles are also demonstrated in the figure,
eters is a non-convex optimization problem (where a global where we can observe the consistency between CFD simula-
minima is not guaranteed), thus that the result of each PINN tion and PINN inference. The mean Nusselt numbers along
simulation may be affected by the randomness of network the cylinder surface of CFD and PINN solutions are 7.11 and
initialization. Therefore, we perform ten independent train- 6.96, respectively. In this case, the L2 errors of (θ, u, v, p)
ing processes for each configuration and demonstrate the best inferred by PINN are 3.18%, 0.18%, 0.24% and 0.46%, re-
results in Figure 4. spectively.
Journal of Heat Transfer. Received February 08, 2021;
Accepted manuscript posted March 17, 2021. doi:10.1115/1.4050542
Copyright (c) 2021 by ASME

Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/doi/10.1115/1.4050542/6665384/ht-21-1083.pdf by Massachusetts Inst Of Tech. user on 26 March 2021

d
ite
ed
py
Co
ot
Fig. 4: Forced convection of flow past cylinder: results of different sensor configurations for inferring constant temperature
surface.
tN
Active sensor placement: We have introduced the
ip

method for active sensor placement in a previous section, and

here we present an example of inferring the constant tem-
r

perature surface in the flow past a cylinder to demonstrate

sc

the effectiveness of this method. As mentioned, the resid-

ual of heat transfer equation is considered as the criterion
nu

since there exists a correlation between the residual and the

error of Nusselt number (which is a posterior assessment).
As an example for explanation, the normalized error of Nus-
Ma

selt number and the normalized residual of the network for

Case 1 in Figure 4 are illustrated in Figure 8. As we can see,
although the shapes of the curves are not exactly identical to
each other, the peak locations of the residual are close to the
ed

locations with maximum Nusselt number error. Therefore,

it is reasonable to expect better performance of the physics-
pt

informed neural network if we add new sensors on the posi-

tion with maximal residual.
ce

Based on this strategy, we carry out an experiment of

active sensor placement for forced convection of flow past
Ac

a cylinder. In this case, the initial sensor setup is Case 1

in Figure 4, which contains 5 measurements. As shown in
Figure 9(a), the algorithm automatically added a sensor near
Fig. 5: Forced convection of flow past a cylinder with con-
the front stagnation point, which results in the configuration
stant temperature surface: temperature, velocity and pres-
corresponding to Case 2 in Figure 4. That means the active
sure fields inferred by PINN with sensor configuration Case
sensor placement algorithm with PINNs can achieve a very
3 (left) and the point-wise absolute errors between CFD and
high accuracy only with one iteration in this case. From Fig-
PINN (right).
ure 9(b-c), we can observe that the profiles of temperature
and Nusselt number on the cylinder surface are consistent
with the CFD solutions. The L2 -norm error of temperature
Journal of Heat Transfer. Received February 08, 2021;
Accepted manuscript posted March 17, 2021. doi:10.1115/1.4050542
Copyright (c) 2021 by ASME

Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/doi/10.1115/1.4050542/6665384/ht-21-1083.pdf by Massachusetts Inst Of Tech. user on 26 March 2021

d
Fig. 8: Active sensor placement: the normalized error of

ite
Nusselt number and the normalized residual of the heat equa-
tion on the cylinder surface, inferred by PINN with sensor
configuration Case 1 in Figure 4. We can observe the corre-

ed
lation between the two profiles.

py
in the domain dramatically decreases to about 3% after the

Co
first iteration. We note that although the active sensor place-
Fig. 6: Forced convection of flow past a cylinder with con- ment algorithm can be performed with more iterations, here
stant temperature surface: (a) temperature and (b) Nusselt we only demonstrate one iteration since the accuracy is al-
number profiles on the cylinder boundary, inferred by PINN ready high enough to terminate the process. For complex

ot
with sensor configuration Case 3. problems, more iterations (i.e., more sensors) are expected.
However, such adaptive sensor placement algorithm allows
tN
us to avoid a trial-and-error procedure for selecting sensor
locations.
ip

3.2 Mixed Convection

r

In order to obtain data and the reference solution, we

sc

consider a two dimensional heat transfer problem of the

mixed convection around a circular cylinder and we numeri-
cally simulate it using the spectral/hp element method, with
nu

the computational domain identical to the one described in

Section 3.1.2. The cylinder surface is assumed to be no-slip,
Ma

no-penetration wall and constant wall temperature is con-

sidered. The governing equations of this problem are the
Boussinesq approximations of the incompressible Navier-
Stokes equations and the corresponding heat transfer equa-
ed

tion, as shown in Equations (6), where the Reynolds number

Re = 100, Péclet number Pe = 71 and Richardson number
Ri = 1.0. In this paper, we investigate the mixed convection
pt

where the force term is acted in the −y-direction. Compared

to the forced convection investigated in the previous section,
ce

the mixed convection problem in this section involves un-

steady flow. To assess the PINN method, the time span of in-
Ac

terest is about t ∈ [0, 15], covering more than three shedding

periods, and the time step is ∆t = 0.1. Moreover, there is a
Fig. 7: Forced convection of flow past a cylinder with con- force term in the NS equations, which couples the solutions
stant flux surface: (a) temperature and (b) Nusselt number of temperature and velocity fields. The mixed convection
profiles on the cylinder boundary. The sensor configuration problem we consider is still an ill-posed problem, where the
is given at the top-left corner. With this setup, the errors temperature boundary condition on the cylinder surface and
of (θ, u, v, p) inferred by PINN are 3.18%, 0.18%, 0.24% and the entire flow fields need to be inferred from a few measure-
0.46%, respectively. ments only. In addition to the thermocouples, in this section
we assume that it is available to measure a small patch of
temperature in the wake behind the cylinder, e.g., using a
Journal of Heat Transfer. Received February 08, 2021;
Accepted manuscript posted March 17, 2021. doi:10.1115/1.4050542
Copyright (c) 2021 by ASME

Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/doi/10.1115/1.4050542/6665384/ht-21-1083.pdf by Massachusetts Inst Of Tech. user on 26 March 2021

d
Fig. 9: Active sensor placement: (a) the sensor configurations of initial setup and after the first iteration, (b) temperature

ite
profile on the cylinder surface, (c) Nusselt number on the cylinder surface.

ed
technique like DPIT [36]. spatio-temporal domain, in the case we have the batch size

py
Results: For the unsteady case, PINN takes the space Nr = 10000 for each iteration. The second and third terms are
and time coordinates (t, x, y) as inputs and outputs (θ, u, v, p). the boundary conditions for velocity and temperature fields,
Different from the network used for forced convection prob- respectively. Note again that the temperature surface of the

Co
lem, here we employ the formulation of neuron-wise locally cylinder boundary is unknown, while other boundaries in-
adaptive activation function [37], where the relation between cluding the inlet, the upper and lower bounds of the compu-
the input X and outputs Y of the l-th each hidden layer is tational domain are given as θb .
expressed as:

ot
We aim to infer the entire flow fields from tempera-
ture measurements at a few locations (e.g., thermocouples)
y j = σ(αlj (wli, j xil + blj )), as well as in a small area (e.g., DPIT patch). An example
tN
(11)
of the configuration is demonstrated in Figure 10(a), where
DPIT measures the temperature in [1, 2] × [−0.5, 0.5], which
where α j is a neuron-level parameter that can be learned dur- is represented by a 8×8 grid. Together with three additional
ip

ing the training process and σ(·) = sin (·). The unknown sensors, we have Nθ = 67 for the PINN algorithm to infer the
parameters of the neural network model can be learned by entire fields. Compared to the loss function used for forced
r

minimizing the following loss function: convection problems, here we also add an extra term Lsym ,
sc

which is used to enforce the symmetric nature of the tem-

L = Lr + Lub + Lθb + Lθ + Lsym , (12) perature profile on the cylinder surface. Although the loss
nu

function without Lsym also works, we find that this a pri-

ori knowledge can help to improve the accuracy. Here, we
where
also note that it is flexible to encode any prior knowledge in
Ma

the PINN framework. We randomly select Nsym = 50 on the

1 4 Nr cylinder surface to enforce the symmetry condition.
Lr = ∑ ∑ |ek (xi , yi ,t i )|2
Nr k=1
(13a)
i=1 The neural network applied for mixed convection prob-
ed

1 Nt Nub lem is composed of ten hidden layers and 150 neurons per
Lub = ∑ |u(xi , yi ,t i ) − uib |2
Nt Nub i=1
(13b) layer. The training procedure is performed by applying
the Adam optimizer with decreasing learning rate 1 × 10−3 ,
pt

1 Nt Nθb 1 × 10−4 , 1 × 10−5 and 5 × 10−6 ; each involves 200k itera-

Lθb = ∑ |θ(xi , yi ,t i ) − θb |2
Nt Nθb i=1
(13c) tions. This can ensure that the neural network converges to a
ce

plateau after training.

1 Nt Nθ The relative L2 -norm errors of inference results over the
Lθ = ∑ |θ(xi , yi ,t i ) − θidata |2 . (13d)
Ac

Nt Nθ i=1 spatial domain are demonstrated in Figure 10(b), which indi-

Nt Nsym cates that the errors are almost constant in time. We observe
1
Lsym = ∑ |θ(xi , yi ,t i ) − θ(xi , −yi ,t i )|2 . (13e) that the temperature error for the entire domain is approx-
Nt Nsym i=1 imately 2%, while the maximum error on the cylinder sur-
face is roughly 5%, as shown in Figure 10(c). The 2D flow
Here Nr , Nt Nθb , Nt Nub , Nt Nθ and Nt Nsym represent the num- fields at t = 5.0 are also illustrated in Figure 10(d), along
ber of residual points corresponding to different terms, with with the point-wise absolute errors. Overall, we can see the
Nt the number of snapshots. The first term enforces the gov- magnitudes of the errors are very small. However, it is also
erning equations by minimizing the residuals of equation worth noting that the errors of the flow (velocity and pres-
(6). Here, Nr training points are randomly selected in the sure) mainly exist in the region far from the cylinder, while
Journal of Heat Transfer. Received February 08, 2021;
Accepted manuscript posted March 17, 2021. doi:10.1115/1.4050542
Copyright (c) 2021 by ASME

Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/doi/10.1115/1.4050542/6665384/ht-21-1083.pdf by Massachusetts Inst Of Tech. user on 26 March 2021

d
ite
ed
py
Co
ot
Fig. 10: Mixed convection of flow past a cylinder. (a) Temperature measurements. DPIT measures temperature in an area
represented by a red rectangle with 8×8 grids. (b) The relative L2 -norm errors of temperature, velocity and pressure fields
tN
along time inferred by PINN. (c) The mean temperature profile on the cylinder surface. (d) The 2D fields at t = 5.0 inferred
by PINN and the corresponding point-wise error maps.
r ip

the mismatch of the temperature is near the cylinder. The Since their initial conception nearly two centuries ago,
sc

reason is that the velocity boundary condition on the cylin- free boundary and Stefan problems now define a well-
der surface is known (no-slip) while that for temperature is studied area in applied mathematics both in terms of the-
unknown. We also expect that adding velocity sensors in the ory [50], numerical methods [51], and applications across
nu

downstream region will further improve the inference perfor- a wide range of problems in science and engineering [42].
mance of PINNs, and this is a straightforward extension for A unique characteristic of free boundary problems that in-
Ma

the interested readers to pursue. troduces great challenges in terms of computational model-
ing is that they necessitate the solution of partial differential
equations in domains with unknown boundaries and com-
4 Two-Phase Stefan Problems plex time-dependent interfaces. Different numerical methods
ed

A large class of problems in heat transfer involve dy- have been developed to solve various types of free bound-
namic interactions between different material phases, giving ary problems, giving rise to different approaches for resolv-
rise to moving boundaries or free interfaces. A classical ex- ing the evolution of moving boundaries and free dynamic
pt

ample is the so-called Stefan problem that traces back to the interfaces [52–57]. All these methods have their own ad-
ice solidification problem describing the joint evolution of a vantages and limitations, and have been proven effective for
ce

liquid and a solid phase related to heat transfer [38]. This certain classes of Stefan problems. However, these classi-
problem was also considered in 1831 by Lamé and Clapey- cal techniques are usually specialized to a specific type of
Ac

ron in relation to the problems of ice formation in the polar free boundary problem and cannot be easily adapted to build
seas [39]. Applications of free boundary and Stefan prob- a general framework for seamlessly synthesizing governing
lems are ubiquitous in science and engineering as they can physical laws and observational data.
naturally model continuum systems with phase transitions,
moving obstacles, multi-phase dynamics, competition for re- In this section, we will illustrate how PINN models can
sources, etc. Specific use cases in the context of heat trans- be adapted to handle free boundaries and time-dependent in-
fer include thermal convection [40], Marangoni convection terfaces using a pedagogical example in heat conduction to
in a liquid pool [41], chemical vapor deposition [42], crystal illustrate the main ideas. To this end, let us consider heat
growth and solidification [43–46], welding [47, 48], semi- transfer in a two-phase system, where the latent tempera-
conductor design [49], and beyond [42]. ture distributions within each of the two phases, u1 (x,t) and
Journal of Heat Transfer. Received February 08, 2021;
Accepted manuscript posted March 17, 2021. doi:10.1115/1.4050542
Copyrightu (c) 2021
(x,t), by ASME satisfies a heat equation
respectively, (1) (2)
2 activations) with two outputs uθ (x,t) and uθ (x,t), i.e,

∂ui ∂2 u i θ
[x,t] −→
u (1)
[uθ (x,t), uθ (x,t)].
(2)
(23)
= ki 2 , (x,t) ∈ Ωi , i = 1, 2, (14)
∂t ∂x
In this way, the final predicted solution in the whole domain
where k1 , k2 are thermal diffusivity parameters. For simplic-
Ω can be given by
ity, here we assume that Ω = {(x,t) : (0, L) × (0, T ]} is a rect-
angular domain which is subdivided by a latent moving in-
(1) (2)
terface s(t) that separates the two phases as uθ (x,t) = uθ (x,t)1(0,sβ (t)) (x) + uθ (x,t)1(sβ (t),1) (x), (24)

Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/doi/10.1115/1.4050542/6665384/ht-21-1083.pdf by Massachusetts Inst Of Tech. user on 26 March 2021

d
Ω1 = {(x,t) ∈ Ω : 0 < x < s(t),t ∈ (0, T ]}. (15) where 1(0,sβ (t)) (x) denotes an indicator function taking a

ite
Ω2 = {(x,t) ∈ Ω : s(t) < x < L,t ∈ (0, T ]}. (16) value of one if x ∈ [0, sβ (t)] and zero otherwise. Then, the
exact temperature solution in the whole domain Ω can be ex-
pressed by

ed
Energy balance implies the following initial and boundary
Stefan conditions
uexact (x,t) = u1 (x,t)1(0,s(t)) (x) + u2 (x,t)1(s(t),1) (x). (25)

py
∗
u1 (s(t),t) = u2 (s(t),t) = u , t ∈ [0, 1], (17)
∂u1 ∂u2 The parameters (θ, β) can be learned by minimizing the fol-
s0 (t) = α1

Co
(s(t),t) + α2 (s(t),t), t ∈ [0, 1], (18) lowing mean squared error loss
∂x ∂x
s(0) = s0 . (19)
2 h i
L= Lr(k) + Ls(k)
bc + LsNc + Ls0 + Ldata ,

ot
∑ (26)
To illustrate the main capabilities of PINNs under this k=1
multi-physics setting, here we consider a prototype inverse
tN
one-dimensional two-phase Stefan problem, in a compu- where
tational domain Ω = [0, 2] × [0, 1] with given coefficients
k1 = 2, k2 = 1, α1 = −2, α2 = 1, and s0 = 1/2. Under
ip

(k) (k)
this setting, one can fabricate a benchmark for which the ex- 1 N ∂uθ ∂2 u θ
Lr(k) = ∑ | (xi ,t i ) − λk (xi ,t i )|2 , k = 1, 2,
act solution for the temperature distribution and the moving N i=1 ∂t ∂x2
r

boundary can be analytically derived as (27)

sc

N
1
u1 (x,t) = 2(exp((t + 1/2 − x)/2) − 1), (20) Ls(k)
bc =
(k)
∑ |uθ (sβ (t i ),t i )|2 , k = 1, 2, (28)
nu

N i=1
u2 (x,t) = exp((t + 1/2 − x) − 1), (21) (1) (2)
1 N ∂uθ ∂u dsβ
s(t) = t + 1/2. (22) LsNc = ∑ |2 ∂x (sβ (t i ),t i ) − ∂xθ (sβ (t i ),t i ) + dt (t i )|2 ,
Ma

N i=1
(29)
Now, suppose that α1 , α2 are known. Given some
1 N
measurements of the temperature distribution inside the Ls0 = ∑ |sβ (0) − s0 |2 , (30)
ed

domain Ω, our goal is to predict the latent functions N i=1

{u1 (x,t), u2 (x,t), s(t)} satisfying the system of Equations 1 M
(14) - (19). Moreover, we would also like to infer the un- Ldata = i
∑ |uθ (xdata i
,tdata ) − ui |2 . (31)
pt

M i=1
known thermal diffusivities k1 and k2 .
A training data set for this benchmark can be generated
ce

by randomly sampling M = 200 measurement points inside Here N denotes the batch size, {(xi ,t i )}Ni=1 are collocation
the domain Ω and obtain corresponding data for ui using points that are randomly sampled at each iteration of the gra-
Ac

Equation (25), i.e. {(xdatai ,t i ), ui }M . It is worth empha- dient descent. In addition, λ1 , λ2 are two extra trainable pa-
data i=1
sizing that for any given data-point {(xdatai ,t i ), ui )}, we do rameters. Since we know that the thermal diffusivity param-
data
not know the corresponding equation that governs ui during eters cannot be negative, we initialize them at 0.1 and con-
training. strain them to remain positive during model training.
Results: As illustrated in Figure 11, we can employ a Figure 12 summarizes the predictions of the proposed
PINN model to represent the unknown interface s(t) by a PINN model, indicating an accurate reconstruction of both
deep fully-connected neural network (5 layers, 100 hidden the latent temperature field, as well as the dynamic interface
units, tanh activations) sβ (t). Similarly, we represent tem- s(t). Moreover, Figure 13(b) shows the identified param-
perature distributions u1 (x,t) and u2 (x,t) by another fully- eters k1 and k2 . Here, we can observe a discrepancy be-
connected neural network (5 layers, 100 hidden units, tanh tween the true and identified parameters indicating that the
Journal of Heat Transfer. Received February 08, 2021;
Accepted manuscript posted March 17, 2021. doi:10.1115/1.4050542
Copyright (c) 2021 by ASME
∂2 u1 2
Correct PDE ∂u1
∂t − 2 ∂x2 = 0
∂u1
∂t − ∂∂xu21 = 0
2 2
Identified PDE (Original) ∂u1
∂t − 1.712 ∂∂xu21 = 0 ∂u1
∂t − 1.137 ∂∂xu21 = 0
2 2
Identified PDE (Adaptive) ∂u1
∂t − 2.003 ∂∂xu21 = 0 ∂u1
∂t − 1.000 ∂∂xu21 = 0

Table 1: Inverse one-dimensional two-phase Stefan problem Type II with unknown parameters: Correct partial differential
equation along with the identified one obtained using physics-informed neural networks with or without the adaptive learning
rate annealing put forth by Wang et. al. [58].

Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/doi/10.1115/1.4050542/6665384/ht-21-1083.pdf by Massachusetts Inst Of Tech. user on 26 March 2021

(1) (2)

d
uθ (x, t) = uθ (x, t)1(0,sβ (t)) (x) + uθ (x, t)1(sβ (t),1) (x) over the parameter space θ. The weighting coefficients λ for
the next iteration are updated using a moving average of the

ite
Temperature network
∂x form
Lr (θ, β)
x u(1)
u ∂t

ed
t ˆ (k+1) ,
λ(k+1) = (1 − α)λ(k) + αλ (34)
u(2) ∗
θ
Lsbc (θ, β)
Parameters: θ
+

py
Interface network
s
Minimize with α = 0.1. As demonstrated in [58], this adaptive strategy
β ∗ can effectively mitigate pathologies arising in the training of
LsN c (θ, β)
t PINNs due to stiffness in their gradient flow dynamics.

Co
Figure 13 and Table 1 present comparisons of the iden-
Parameters: β Ls0 (β) tified parameters between the original PINN formulation of
Raissi et. al. [23], and the proposed PINN formulation with
Fig. 11: Two-phase Stefan problem: PINN architecture for adaptive weights [58, 59]. Notice that the PINN with adap-

ot
inferring the latent temperature fields u1 (x,t), u2 (x,t), and tive weights not only converges to the exact parameters much
phase-transition interface s(t), from scattered noisy observa- faster, but also yields a significantly improved identification
tN
tions of temperature. accuracy. In addition, we also investigate the accuracy of
the reconstructed temperature u(x,t) and inferred interface
s(t) with respect to these two methods. A comparison of
ip

PINN model described above fails to correctly identify the relative L2 error in u(x,t) and s(t) between these two mod-
els is presented in Table 1 from which we can see that the
r

unknown thermal diffusivity parameters, even after 200, 000

dynamic weights approach improves the relative prediction
sc

training iterations. In fact, we can observe that the two iden-

tified parameters do not change as training goes on. This ob- error by about one order of magnitude. These figures and
servation suggests that our model seems to get stuck in some tables suggest that the weights in the loss function play an
nu

local minimum, and, as a result, fails to correctly recover the important role, and choosing appropriate weight coefficients
target parameter values. This indicates that the observed in- can enhance the performance of PINNs by accelerating con-
accuracy in the model’s predictions is not due to insufficient vergence and avoiding poor local minima.
Ma

training iterations, but relates to some issue pertaining to the

model itself.
To resolve this issue, we proceed by applying the strat- 5 Applications to Power Electronics
Dealing with the extreme heat fluxes in power electron-
ed

egy of dynamic weights put forth by Wang et al. [58]. Specif-

ically, we reformulate the loss function as ics requires advanced cooling technologies. The target heat
density levels can be > 1kW/cm2 and > 1kW/cm3 with a
pt

typical chip temperature rise 30o C and maximum tempera-

2
ture difference across the chip footprint around 10o C [60].
h i
L= ∑ Lr(k) + Ls(k)
bc + LsNc ) + Ls0 + λLdata , (32)
ce

k=1 In the following, we present two recent works by Ansys and

Nvidia in modeling heat transfer in power electronics using
PINNs.
Ac

where the weight parameter λ is adaptively updated during

training by utilizing the back-propagated gradient statistics
[58]. Specifically, the estimates of λ are computed by 5.1 Heat transfer in electronic chips
The thermal management in semiconductors is ex-
tremely important in the AI chip-package-systems, 5G net-
ˆ (k+1) = max {|∇θ Lr |} / ∇θ λ(k) Ldata ,
λ (33)
θ works and automotive. To this end, solving heat transfer
problems on electronic chips is investigated by Central Ma-
chine Learning (ML) Team at Ansys. The heat transfer from
where maxθ {|∇θ Lr |} is the maximum value attained by
a chip involves heat conduction in solid phase, heat transfer
|∇θ Lr |, and ∇θ λ(k) Ldata denotes the mean of ∇θ λ(k) Ldata from the chip to the surrounding fluid (air) and radiation. To
Journal of Heat Transfer. Received February 08, 2021;
Accepted manuscript posted March 17, 2021. doi:10.1115/1.4050542
Copyright (c) 2021 by ASME
(a)

Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/doi/10.1115/1.4050542/6665384/ht-21-1083.pdf by Massachusetts Inst Of Tech. user on 26 March 2021

d
(b)

ite
ed
py
Co
Fig. 12: Inverse two-phase Stefan problem: (a) Exact temperature solution along with training data (×) versus the predicted
solution. The relative L2 error: 2.57 · 10−3 . (b) Left: Comparison of the and exact predicted moving boundary. Right: The

ot
absolute error between the exact and the predicted free boundary for t ∈ [0, 1]. The L2 error is 3.93 · 10−4 .
tN
test the effectiveness of using PINNs to solve heat equations, line passing through the hottest spot on the chip (PINN pre-
various canonical cases, such as 2D transient heat transfer diction in orange curve, numerical results in blue curve).
ip

with Dirichlet boundary condition, 2D transient heat transfer

with source, as well as some practical engineering problems,
r

Inspired by PINN, an Auto Encoder and Image Gradi-

sc

such as 2D and 3D heat transfer on a chip, chip tile tempera-

ture prediction etc., have been investigated. The experimen- ent (AEIG) based approach has also been proposed by the
tal results are shown in Fig. 14. Central ML Team at Ansys team for solving 2D and 3D
nu

chip thermal analysis [22]. Fig. 14(d-1) presents some ex-

In Fig. 14(a), a 2D transient heat transfer problem with
amples of the 2D power maps, their corresponding ground
Dirichlet boundary conditions (temperatures on top, bottom,
truth, network prediction and the prediction error. The pro-
Ma

left and right walls are 150, 300, 50 and 100K, respectively)
posed network shows acceptable agreement with numerical
is solved and the comparison between numerical simulation
simulation with mean absolute percentage error (MAPE) of
results and PINN prediction is given. A 2D transient heat
0.4%. Fig. 14(d-2) demonstrates some prediction examples
transfer case with heat source in the center is depicted in
by AEIG on 3D power maps and the MAPE is 0.14%.
ed

Fig. 14(b). Since the temperature profile along the line pass-
ing through the center of the square is of interest, temperature
calculations from a numerical solver and PINN are compared Moreover, a chip tile-based temperature prediction prob-
pt

along this line. It is worth noting that even though the resid- lem is solved and exhibited in Fig. 14(e). A chip consists of
ual points only cover a certain range in time axis (e.g., 0 – 20 arrays of tiles and the temperature distribution under the con-
ce

s), the trained model can make predictions beyond the train- dition that power is only applied on a single tile of the chip is
ing range for extrapolation (e.g., 40 s) with reasonable accu- concerned. Since there could be variations about parameters
Ac

racy (PINN prediction in orange curve, numerical results in such as x and y coordinates of the tile with power, size of the
blue curve). tile and power magnitude etc., a parameterized PINN is uti-
As for practical engineering problems, PINNs have been lized where these parameters together with the independent
used for solving 2D heat transfer on a chip, which is illus- variables of the PDE are taken as inputs to the network. The
trated in Fig. 14(c). Discrete and drastically distinct power trained model would then be used to make inferences on any
is applied on each tile of the chip and the final temperature points (unseen combinations of the parameters) within the
distribution on the chip needs to be obtained. As shown in input space. Temperature predictions by the parameterized
Fig. 14(c), PINNs not only display satisfactory prediction on PINN for different power source locations, tile sizes with
the overall temperature distribution, but they also demon- power, power magnitudes, are demonstrated in Fig. 14(e-1),
strate adequate accuracy for the temperature profile on the (e-2) and (e-3), respectively.
Journal of Heat Transfer. Received February 08, 2021;
Accepted manuscript posted March 17, 2021. doi:10.1115/1.4050542
Copyright (c)
(a)2021 by ASME gle geometry simulations. A forward solution of parame-
terized complex geometry with turbulent fluid flow between
thinly spaced fins without training data makes this problem
extremely challenging for the neural networks. Once the
training is complete, several geometry, material or physi-
cal parameter combinations can be evaluated using infer-
ence as a post-processing step, without solving the forward
problem again. Such throughput enables more efficient de-
sign space exploration tasks for complex systems in science
and engineering. The neural networks in this example are

Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/doi/10.1115/1.4050542/6665384/ht-21-1083.pdf by Massachusetts Inst Of Tech. user on 26 March 2021

trained with 10 variables (with 9 of them being geometry pa-

d
rameters) with min, max and median values in the range of
these parameters. The OpenFOAM ® and commercial solver

ite
(b) runs are on 12 CPU cores (DUAL 3.4 GHz Intel Xeon Gold
x6128), and the SimNet runs are on 8 V100 GPUs (DGX1).

ed
It can be seen that SimNet can solve the problem faster than
traditional solvers by several orders of magnitude.

py
6 Summary and Outlook
Employing neural networks in heat transfer applications

Co
could accelerate progress in heat transfer enhancement, ther-
mal design, and complex multi-phase systems. However,
the usual data-driven approach of neural networks is not ap-

ot
propriate due to the lack of big data and the various limita-
tions on measuring velocity and temperature fields that may
tN
Fig. 13: Inverse one-dimensional two-phase Stefan prob- change rapidly in space and time. It is possible to measure
lem Type II with unknown parameters: Convergence of temperature on part of a surface, in some cases, by employ-
the learned thermal diffusivity coefficients k1 and k2 using ing the temperature-sensitive paint method (TSP) [61], while
ip

physics-informed neural networks with (a) or without (b) for higher temperatures phosphors can be used. The infrared
the adaptive learning rate annealing put forth by Wang et. cameras can also be used for measurements on a patch of
r

al. [58]. (a) Inferred coefficients: k1 = 1.751, k2 = 1.117. a heated surface. Their resolutions varies from 320x320 to
sc

(b) Inferred coefficients: k1 = 2.003, k2 = 1.000. 1280x1024, they have good dynamic range at 14 bits, and un-
certainties are ±2◦C for T < 100◦C, and ±2% of the reading
for T > 100◦C. They can measure up to 600◦C, 2000◦C, and
nu

3000◦C depending of which of the detectors is used. Within

5.2 SimNet for heat sink design the flow, one can use the DPITV method [36], but further
Ma

To date, the published literature on PINNs has been enhancements are required to improve its accuracy. In these
able to demonstrate the forward solution of only simple methods, the response times are slow, on the order of mil-
problems when not using training data. The gradients, liseconds, so it will be necessary for critical dynamic applica-
singularities and discontinuities introduced by complex ge- tions to improve the response times down to 10s of microsec-
ed

ometries or complex physics make the forward solution of onds. The surface film TSP approach has a resolution that is
real world problems without training data extraordinarily equal to the size of the imaged pixel. For the bulk flow meth-
challenging. Nvidia’s SimNet ® (http://developer. ods, the resolution would be on the order of the mean particle
pt

nvidia.com) is a toolkit for researchers and engineers distance, which can be improved with standard interpolation
with dual goals of being an extensible research platform methods but also with neural networks and especially PINNs
ce

as well as a solver for real world and industrial problems. that can fill gaps and provide high accuracy [20].
Fig. 15 shows results of a conjugate heat transfer problem PINNs can play a catalytic role in bridging the gap be-
Ac

for the Nvidia’s DGX-A100 NVSwitch heat sink whose fin tween experimental and computational heat transfer. Us-
geometry are variable. In heat sink design, the objective is ing sparse measurements by employing the aforementioned
to minimize the peak temperature that can be reached at the multi-fidelity methods and encoding directly the conserva-
source chip while satisfying a maximum pressure drop con- tion laws into the neural network architecture, inference of
straint. This is necessary to meet the operating temperature the velocity and temperature fields as well as unknown ther-
requirements of the chip on which the heat sink is mounted mal boundary conditions or interfaces can be accurately ob-
for cooling. In this example, SimNet trains on a parametrized tained. PINNs offer a hybrid model, where we can use any
geometry with multiple design variables in a single train- data that is available at any locations and asynchronously
ing run thereby significantly accelerating design space ex- in time while enforcing the governing equations using au-
ploration. In contrast, traditional solvers are limited to sin- tomatic differentiation at random points in the space-time
Journal of Heat Transfer. Received February 08, 2021;
Accepted manuscript posted March 17, 2021. doi:10.1115/1.4050542
Copyright (c) 2021 by ASME

Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/doi/10.1115/1.4050542/6665384/ht-21-1083.pdf by Massachusetts Inst Of Tech. user on 26 March 2021

d
ite
ed
py
Co
ot
tN
Fig. 14: Solving the heat equation on electronic chips. (a) 2D transient heat transfer problem with Dirichlet boundary
ip

conditions; (b) 2D transient heat transfer problem with heat source at the center of the square. Comparisons are made on
the horizontal center line within the training time zone (interpolation) and beyond the training time zone (extrapolation). (c)
r
sc

2D heat transfer on a chip. Comparisons are performed on the red section lines passing through the hottest spot on chip.
(d) Thermal analysis for 2D and 3D chips using PINN-inspired Auto Encoder Image Gradient based approach [22]. (e)
Temperature prediction on a chip with a single tile as power source by a parameterized PINN, for different power source
nu

locations, tile sizes with power and power magnitudes. (Courtesy of the Central ML Ansys team.)
Ma

domains without the need for elaborate and expensive mesh discover the entire boundary condition while simultaneously
generation. Hence, PINNs are based on compact computer infer the velocity and temperature fields in the domain. The
codes of 100s of lines compared to 1000s lines of code for presented results indicate good performance of the method,
ed

traditional numerical solvers. e.g., for the case of constant temperature surface, using at
most six measurements can correctly infer the temperature
Moreover, PINNs can solve ill-posed inverse problems and Nusselt number profiles on the boundary with less than
pt

not possible with standard approaches, such as the om- 5% error. As the selection of the sensor location is very im-
nipresent problem of unknown thermal boundary conditions. portant for this inverse problem, we also proposed a method
ce

For decades, both experimental and computational research for active sensor placement. The residual of the heat transfer
in heat transfer relied on idealized thermal boundary condi- equation is considered as the criterion for selecting the sen-
Ac

tions of constant temperature or constant heat flux, neither of sor location. This metric is computed by the network, thus no
which is valid in practice. The pioneering experimental work prior knowledge is required. We assume that adding sensors
of Robert Moffat [62], see, e.g., the PhD thesis of [63], first at the location with highest value of residual can efficiently
addressed this issue by relaxing these assumptions, and we improve the performance of thermal boundary inference. Al-
believe that PINNs is an effective approach to finally resolve though the residual is related to the network structure and
this limitation. We demonstrated this possibility in the cur- training process, we demonstrate the correlation between the
rent paper by considering forced and mixed heat transfer with residual and the posterior error of Nusselt number. As our
unknown thermal boundary conditions, having only a small test results show, this strategy can adaptively select the loca-
number of temperature measurements on the heated cylinder tions to place sensors while iteratively increase the prediction
or in the wake. Upon training the neural network, we can
Journal of Heat Transfer. Received February 08, 2021;
Accepted manuscript posted March 17, 2021. doi:10.1115/1.4050542
Copyright (c) 2021 by ASME

Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/doi/10.1115/1.4050542/6665384/ht-21-1083.pdf by Massachusetts Inst Of Tech. user on 26 March 2021

d
ite
ed
py
Co
ot
tN
r ip
sc
nu

Fig. 15: Solution of conjugate heat transfer problem on a complex parameterized geometry of a heat sink for NVSwitch in
DGX-A100. (A) Geometry for a heat sink with fins and heat pipes to dissipate heat from GPU, (B) Pressure color coded
Ma

flow streamlines and result comparisons between SimNet and CFD commercial code for (C) U-velocity, (D) V-velocity and
(E) Pressure. (Courtesy of the Nvidia team).
ed

accuracy. Hence, PINNs can effectively guide the experi- surements as inputs to PINNs, more efficient methods in
mental work as well as bridging the gap between simulations quantifying uncertainties due to data as well as model uncer-
pt

and experiments as stated earlier by fusing any multi-fidelity tainties, tackling multi-dimensional multi-phase problems,
measurements directly with the governing equations, whicy and producing proper benchmarks and data sets that can be
ce

are encoded in the deep neural networks. used to further accelerate development of PINNs, especially
Here we presented the basic version of PINNs but there in the industrial setting for thermal design, similar to the two
applications we highlighted in the present paper.
Ac

have been many developments already, e.g., in employing

domain decomposition in CPINN [64] and XPINN [65] that
provide great flexibility with complex geometries but also in
assigning a different neural network on each subdomain as Acknowledgements
well as parallel execution to accelerate inference. Another This work was supported by PhILMs (DOE DE-
extension is the the use of variational formulation in hp- SC0019453), MURI/OSD (FA9550-20-1-0358), DOE grant
VPINN [66]. Moreover, to quantify uncertainties one can use DE-SC0019116, AFOSR grant FA9550-20-1-0060, and
spectral expansions as in [67], generative models [68, 69], DOE-ARPA grant 1256545. We would like to thank Prof.
or a Bayesian formulation as in [70]. Future work should Dana Dabiri of Washington University for providing the in-
address the possibility of multi-modality/multi-fidelity mea- formation on current measurement techniques. We would
Journal of Heat Transfer. Received February 08, 2021;
Accepted manuscript posted March 17, 2021. doi:10.1115/1.4050542
Copyrightalso
(c) like
2021toby ASME
thank the Nvidia team (Oliver Hennigh, Susheela adakis, G. E., 2019. “Deep learning of vortex-induced
Narasimhan, Mohammad Amin Nabian, Akshay Subrama- vibrations”. J. Fluid Mech., 861, pp. 119–137.
niam, Kaustubh Tangsali, Max Rietmann, Jose del Aguila [14] Mao, Z., Jagtap, A. D., and Karniadakis, G. E.,
Ferrandis, Wonmin Byeon, Zhiwei Fang, Sanjay Choudhry) 2020. “Physics-informed neural networks for high-
and Mr. Jay Pathak and Dr. Haiyang He of Ansys for pro- speed flows”. Comput. Method. Appl. M., 360,
viding their results for section 5 of the paper. p. 112789.
[15] Jin, X., Cai, S., Li, H., and Karniadakis, G. E., 2021.
“NSFnets (Navier-Stokes flow nets): Physics-informed
References neural networks for the incompressible Navier-Stokes
[1] Jambunathan, K., Hartle, S., Ashforth-Frost, S., and equations”. J. Comput. Phys., 426, p. 109951.

Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/doi/10.1115/1.4050542/6665384/ht-21-1083.pdf by Massachusetts Inst Of Tech. user on 26 March 2021

Fontama, V., 1996. “Evaluating convective heat trans- [16] Hennigh, O., Narasimhan, S., Nabian, M. A., Sub-

d
fer coefficients using neural networks”. Int. J. Heat ramaniam, A., Tangsali, K., Rietmann, M., Ferran-
Mass Transfer, 39(11), pp. 2329–2332. dis, J. d. A., Byeon, W., Fang, Z., and Choudhry, S.,

ite
[2] Liu, Y., Dinh, N., Sato, Y., and Niceno, B., 2018. 2020. “NVIDIA SimNetˆ{TM}: an AI-accelerated
“Data-driven modeling for boiling heat transfer: using multi-physics simulation framework”. arXiv preprint

ed
deep neural networks and high-fidelity simulation re- arXiv:2012.07938.
sults”. Appl. Therm. Eng., 144, pp. 305–320. [17] Cai, S., Wang, Z., Chryssostomidis, C., and Karni-
[3] Kwon, B., Ejaz, F., and Hwang, L. K., 2020. “Machine adakis, G. E., 2020. “Heat transfer prediction with

py
learning for heat transfer correlations”. Int. Commun. unknown thermal boundary conditions using physics-
Heat Mass Transfer, 116, p. 104694. informed neural networks”. In ASME 2020 Flu-
[4] Tamaddon-Jahromi, H. R., Chakshu, N. K., Sazonov, ids Engineering Division Summer Meeting collocated

Co
I., Evans, L. M., Thomas, H., and Nithiarasu, P., 2020. with the ASME 2020 Heat Transfer Summer Confer-
“Data-driven inverse modelling through neural network ence and the ASME 2020 18th International Confer-
(deep learning) and computational heat transfer”. Com- ence on Nanochannels, Microchannels, and Minichan-

ot
put. Method. Appl. M., 369, p. 113217. nels, American Society of Mechanical Engineers Digi-
[5] Fonda, E., Pandey, A., Schumacher, J., and Sreeni- tal Collection.
tN
vasan, K. R., 2019. “Deep learning in turbulent con- [18] Wang, T., Huang, Z., Sun, Z., and Xi, G., 2020. “Re-
vection networks”. Proc. Natl. Acad. Sci., 116(18), construction of natural convection within an enclosure
pp. 8667–8672. using deep neural network”. Int. J. Heat Mass Transfer,
ip

[6] Li, Y., Wang, H., and Deng, X., 2019. “Image-based 164, p. 120626.
reconstruction for a 3d-pfhs heat transfer problem by [19] Lucor, D., Agrawal, A., and Sergent, A., 2021.
r

reconnn”. Int. J. Heat Mass Transfer, 134, pp. 656– “Physics-aware deep neural networks for metamodel-
sc

667. ing of turbulent natural convection”.

[7] Edalatifar, M., Tavakoli, M. B., Ghalambaz, M., and [20] Cai, S., Wang, Z., Fuest, F., Jeon, Y. J., Gray, C.,
Setoudeh, F., 2020. “Using deep learning to learn and Karniadakis, G. E., 2021. “Flow over an espresso
nu

physics of conduction heat transfer”. J. Therm. Anal. cup: Inferring 3D velocity and pressure fields from
Calorim., pp. 1–18. tomographic background oriented schlieren videos via
Ma

[8] Kim, J., and Lee, C., 2020. “Prediction of turbulent physics-informed neural networks”. J. Fluid Mech.
heat transfer using convolutional neural networks”. J. [21] Wang, S., and Perdikaris, P., 2021. “Deep learning of
Fluid Mech., 882. free boundary and stefan problems”. J. Comput. Phys.,
[9] Ronneberger, O., Fischer, P., and Brox, T., 2015. “U- 428, p. 109914.
ed

net: Convolutional networks for biomedical image seg- [22] He, H., and Pathak, J., 2020. “An unsupervised learn-
mentation”. In International Conference on Medical ing approach to solving heat equations on chip based
image computing and computer-assisted intervention, on auto encoder and image gradient”. arXiv preprint
pt

Springer, pp. 234–241. arXiv:2007.09684.

[10] Smith, R., and Dutta, S., 2021. “Conjugate ther- [23] Raissi, M., Perdikaris, P., and Karniadakis, G. E., 2019.
ce

mal optimization with unsupervised machine learning”. “Physics-informed neural networks: A deep learning
JOURNAL OF HEAT TRANSFER-TRANSACTIONS framework for solving forward and inverse problems
Ac

OF THE ASME . involving nonlinear partial differential equations”. J.

[11] Beintema, G., Corbetta, A., Biferale, L., and Toschi, Comput. Phys., 378, pp. 686–707.
F., 2020. “Controlling rayleigh–bénard convection via [24] Psichogios, D. C., and Ungar, L. H., 1992. “A hy-
reinforcement learning”. J. Turbul., 21(9-10), pp. 585– brid neural network-first principles approach to process
605. modeling”. AIChE J., 38(10), pp. 1499–1511.
[12] Hachem, E., Ghraieb, H., Viquerat, J., Larcher, A., and [25] Lagaris, I. E., Likas, A., and Fotiadis, D. I., 1998. “Ar-
Meliga, P., 2020. Deep reinforcement learning for the tificial neural networks for solving ordinary and partial
control of conjugate heat transfer with application to differential equations”. IEEE T. Neural. Networ., 9(5),
workpiece cooling. pp. 987–1000.
[13] Raissi, M., Wang, Z., Triantafyllou, M. S., and Karni- [26] Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A.,
Journal of Heat Transfer. Received February 08, 2021;
Accepted manuscript posted March 17, 2021. doi:10.1115/1.4050542
Copyright (c) 2021
Dean,byJ.,ASME
Devin, M., Ghemawat, S., Irving, G., Isard, [43] Dupret, F., Nicodeme, P., Ryckmans, Y., Wouters, P.,
M., et al., 2016. “Tensorflow: A system for large-scale and Crochet, M., 1990. “Global modelling of heat
machine learning”. In 12th {USENIX} symposium on transfer in crystal growth furnaces”. Int. J. Heat Mass
operating systems design and implementation ({OSDI} Transfer, 33(9), pp. 1849–1871.
16), pp. 265–283. [44] Madejski, J., 1976. “Solidification of droplets on a cold
[27] Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, surface”. Int. J. Heat Mass Transfer, 19(9), pp. 1009–
E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., and 1013.
Lerer, A., 2017. “Automatic differentiation in pytorch”. [45] Sackinger, P., Brown, R., and Derby, J., 1989. “A finite
[28] Incropera, F. P., Lavine, A. S., Bergman, T. L., and De- element method for analysis of fluid flow, heat transfer
Witt, D. P., 2007. Fundamentals of Heat and Mass and free interfaces in czochralski crystal growth”. Int.

Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/doi/10.1115/1.4050542/6665384/ht-21-1083.pdf by Massachusetts Inst Of Tech. user on 26 March 2021

Transfer. Wiley. J. Numer. Methods Fluids, 9(4), pp. 453–492.

d
[29] Bejan, A., and Kraus, A. D., 2003. Heat Transfer [46] Cahn, J. W., 1960. “Theory of crystal growth and in-
Handbook. John Wiley & Sons. terface motion in crystalline materials”. Acta Metall.,

ite
[30] Wang, Z., Chalfant, G. E. K. J., Chryssostomidis, C., 8(8), pp. 554–562.
and Babaee, H., 2017. “High-fidelity modeling and op- [47] Tekriwal, P., Mazumder, J., et al., 1988. “Finite element

ed
timization of conjugate heat transfer in arrays of heated analysis of three-dimensional transient heat transfer in
cables”. In 2017 IEEE Electric Ship Technologies gma welding”. Weld. J., 67(7), pp. 150s–156s.
Symposium (ESTS), pp. 557–563. [48] David, S., Babu, S., and Vitek, J., 2003. “Welding:

py
[31] D’Elia, M., Perego, M., and Veneziani, A., 2012. “A Solidification and microstructure”. Jom, 55(6), pp. 14–
variational data assimilation procedure for the incom- 20.
pressible Navier-Stokes equations in hemodynamics”. [49] Friedman, A., and Hu, B., 1995. “Stefan problem with

Co
J. Sci. Comput., 52(2), pp. 340–359. kinetic condition arising in semiconductor processing”.
[32] Ozisik, M. N., 2018. Inverse heat transfer: fundamen- PITMAN RESEARCH NOTES IN MATHEMATICS SE-
tals and applications. Routledge. RIES, pp. 121–121.

ot
[33] Karniadakis, G. E., and Sherwin, S., 2005. Spectral/hp [50] Duvant, G., and Lions, J. L., 2012. Inequalities in
Element Methods for Computational Fluid Dynamics, mechanics and physics, Vol. 219. Springer Science &
tN
2nd edition. Oxford University Press, Oxford,UK. Business Media.
[34] Glorot, X., and Bengio, Y., 2010. “Understanding [51] Crowley, A., 1978. “Numerical solution of Stefan prob-
the difficulty of training deep feedforward neural net- lems”. Int. J. Heat Mass Transfer, 21(2), pp. 215–219.
ip

works”. In Proceedings of the thirteenth interna- [52] Voller, V., 1987. “An implicit enthalpy solution for
tional conference on artificial intelligence and statistics, phase change problems: with application to a binary al-
r

pp. 249–256. loy solidification”. Appl. Math. Model., 11(2), pp. 110–
sc

[35] Kingma, D. P., and Ba, J., 2014. “Adam: A 116.

method for stochastic optimization”. arXiv preprint [53] Date, A., 1992. “Novel strongly implicit enthalpy for-
arXiv:1412.6980. mulation for multidimensional Stefan problems”. Nu-
nu

[36] Dabiri, D., 2009. “Digital particle image thermom- mer. Heat. Tr. B-Fund., 21(2), pp. 231–251.
etry/velocimetry: a review”. Exp. Fluids, 46(2), [54] Fix, G. J., 1982. “Phase field methods for free boundary
Ma

pp. 191–241. problems”.

[37] Jagtap, A. D., Kawaguchi, K., and Em Karniadakis, [55] Mackenzie, J., and Robertson, M., 2002. “A moving
G., 2020. “Locally adaptive activation functions with mesh method for the solution of the one-dimensional
slope recovery for deep and physics-informed neural phase-field equations”. J. Comput. Phys., 181(2),
ed

networks”. Proc. R. Soc. London, Ser. A, 476(2239), pp. 526–544.

p. 20200334. [56] Chen, S., Merriman, B., Osher, S., and Smereka, P.,
[38] Stefan, J., 1891. “Über die theorie der eisbildung, ins- 1997. “A simple level set method for solving Stefan
pt

besondere über die eisbildung im polarmeere”. Ann. problems”. J. Comput. Phys., 135(1), pp. 8–29.
Phys. (Berlin, Ger.), 278(2), pp. 269–286. [57] Osher, S., Fedkiw, R., and Piechor, K., 2004. “Level set
ce

[39] Tarzia, D. A., et al., 2000. “A bibliography on moving- methods and dynamic implicit surfaces”. Appl. Mech.
free boundary problems for the heat-diffusion equa- Rev., 57(3), pp. B15–B15.
Ac

tion”. The Stefan and related problems, MAT-Serie A, [58] Wang, S., Teng, Y., and Perdikaris, P., 2020. “Un-
2. derstanding and mitigating gradient pathologies in
[40] Busse, F., 1978. “Non-linear properties of thermal con- physics-informed neural networks”. arXiv preprint
vection”. Rep. Prog. Phys., 41(12), p. 1929. arXiv:2001.04536.
[41] Tsai, M., and Kou, S., 1989. “Marangoni convection in [59] Wang, S., Yu, X., and Perdikaris, P., 2020. “When and
weld pools with a free surface”. Int. J. Numer. Methods why pinns fail to train: A neural tangent kernel per-
Fluids, 9(12), pp. 1503–1516. spective”. arXiv preprint arXiv:2007.14527.
[42] Friedman, A., 2000. “Free boundary problems in sci- [60] Garimella, S. V., Persoons, T., Weibel, J. A., and Gek-
ence and technology”. Notices of the AMS, 47(8), tin, V., 2017. “Electronics thermal management in
pp. 854–861. information and communications technologies: Chal-
Journal of Heat Transfer. Received February 08, 2021;
Accepted manuscript posted March 17, 2021. doi:10.1115/1.4050542
Copyright (c) 2021 byand
lenges ASME
future directions”. IEEE T. Comp. Pack.
Man., 7(8), pp. 1191–1205.
[61] Prasad, A., 2016. “A detailed uncertainty analysis of
heat transfer experiments using temperature sensitive
paint”. PhD thesis.
[62] Moffat, R. J., 1990. “Some experimental methods for
heat transfer studies”. Exp. Therm. Fluid Sci., 3(1),
pp. 14–32.
[63] Maciejewski, P., 1991. “Heat transfer with very high
free-stream turbulence”. PhD thesis.

Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/doi/10.1115/1.4050542/6665384/ht-21-1083.pdf by Massachusetts Inst Of Tech. user on 26 March 2021

[64] Jagtap, A. D., Kharazmi, E., and Karniadakis, G. E.,

d
2020. “Conservative physics-informed neural networks
on discrete domains for conservation laws: Appli-

ite
cations to forward and inverse problems”. Comput.
Method. Appl. M., 365, p. 113028.

ed
[65] Jagtap, A. D., and Karniadakis, G. E., 2020. “Extended
physics-informed neural networks (XPINNs): A gen-
eralized space-time domain decomposition based deep

py
learning framework for nonlinear partial differential
equations”. Commun. Comput. Phys., 28(5), pp. 2002–
2041.

Co
[66] Kharazmi, E., Zhang, Z., and Karniadakis, G. E., 2021.
“hp-VPINNs: Variational physics-informed neural net-
works with domain decomposition”. Comput. Method.

ot
Appl. M., 374, p. 113547.
[67] Zhang, D., Lu, L., Guo, L., and Karniadakis, G. E.,
tN
2019. “Quantifying total uncertainty in physics-
informed neural networks for solving forward and in-
verse stochastic problems”. J. Comput. Phys., 397,
ip

p. 108850.
[68] Yang, Y., and Perdikaris, P., 2019. “Adversarial un-
r

certainty quantification in physics-informed neural net-

sc

works”. J. Comput. Phys., 394, pp. 136–152.

[69] Yang, Y., and Perdikaris, P., 2018. “Physics-
informed deep generative models”. arXiv preprint
nu

arXiv:1812.03511.
[70] Yang, L., Meng, X., and Karniadakis, G. E., 2021. “B-
Ma

PINNs: Bayesian physics-informed neural networks

for forward and inverse PDE problems with noisy
data”. J. Comput. Phys., 425, p. 109913.
ed
pt
ce
Ac

View publication stats