Journal Pre-proofs

Probabilistic physics-guided machine learning for fatigue data analysis

Jie Chen, Yongming Liu

PII: S0957-4174(20)31011-3
DOI: https://doi.org/10.1016/j.eswa.2020.114316
Reference: ESWA 114316

To appear in: Expert Systems with Applications

Received Date: 24 June 2020

Revised Date: 29 October 2020
Accepted Date: 12 November 2020

Please cite this article as: Chen, J., Liu, Y., Probabilistic physics-guided machine learning for fatigue data
analysis, Expert Systems with Applications (2020), doi: https://doi.org/10.1016/j.eswa.2020.114316

This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover
page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version
will undergo additional copyediting, typesetting and review before it is published in its final form, but we are
providing this version to give early visibility of the article. Please note that, during the production process, errors
may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

© 2020 Published by Elsevier Ltd.

 Probabilistic physics-guided machine learning for fatigue data analysis

Jie Chena, Yongming Liua,*

E-mail: Jie Chen [email protected]; Yongming Liu [email protected]

a School for Engineering of Matter, Transport, and Energy, Arizona State University,
Tempe, AZ 85287, USA

*Corresponding author
Email: [email protected]
Tel.: +1-480-965-6883
 Abstract: A Probabilistic Physics-guided Neural Network (PPgNN) is proposed in this
paper for probabilistic fatigue S-N curve estimation. The proposed model overcomes the
limitations in existing parametric regression models and classical machine learning models for
fatigue data analysis. Compared with explicit regression-type models (such as power law
fitting), the PPgNN is flexible and does not impose restrictions on function types at different
stress levels, mean stresses, or other factors. One unique benefit is that the proposed method
includes the known physics/knowledge constraints in the machine learning model; the method
can produce both accurate and physically consistent results compared with the classical
machine learning model, such as neural network models. In addition, the PPgNN uses both
failure and runout data in the training process, which encodes the runout data using a new
proposed loss function, and is beneficial when compared with some existing models using only
numerical point value data. A mathematical formulation is derived to include different types of
physics constraints, which can deal with mean value, variance, and derivative/curvature
constraints. Several data sets from open literature for fatigue S-N curve testing are used for
model demonstration and model validation. Next, the proposed network architecture is
extended to include multi-factor (e.g., mean stress, corrosion, frequency effect, etc.) fatigue
data analysis. It is shown that the proposed PPgNN can serve as a flexible and robust model
for general fitting and uncertainty quantification of fatigue data. This paper provides a feasible
way to incorporate known physics/knowledge in neural network-based machine learning. This
is achieved by properly designing the network topology and constraining the neural network’s
biases and weights. The benefits for the proposed physics-guided learning for fatigue data
analysis are illustrated by comparing results from neural network models with and without
physics guidance. The neural network model, without physics guidance, produces results
contradictory to the common knowledge, such as a monotonic decrease of S-N curve slope and
a monotonic increase of fatigue life variance as the stress level decreases. This problem can be
avoided using the physics-guided learning model with encoded prior physics knowledge.

Keywords: probabilistic, physics-guided machine learning, neural network, uncertainty

quantification, S-N curve

1. Introduction
The relationship between probabilistic fatigue lives of materials and applied stresses or
strains is fundamental and critical for the safe-life and damage tolerance design processes
considering reliability (Chen, Diao, He, Pang, & Guan, 2018; Chen, Imanian, Wei, Iyyer, &
Liu, 2020; Chen, Liu, Zhang, & Liu, 2020; Chen & Liu, 2020; Kim, Song, & Park, 2009; Y.
Liu & Mahadevan, 2009; Pascual & Meeker, 1999). A fatigue life prediction under general
random loadings requires the analysis of the probabilistic fatigue damage accumulation at
various stress levels. Thus, a reliable fatigue data analysis and uncertainty quantification
method plays a significant role for both structural designs and analyses (X. Liu, Xuan, Si, &
Tu, 2008; Y. Liu, Liu, Stratman, & Mahadevan, 2008).
The current attempts to establish the fatigue life – stress relationship can be categorized
into explicit regression models and machine learning models. For explicit regression models,
2
a well-known example is the Basquin relation (Basquin, 1910), which states a linear
relationship between logarithm of the fatigue life (N) and logarithm of the stress level (S) i.e.,
log10N = β0 + β1log10S. Only two unknown parameters (slope β1 and intercept β0) need to be
estimated when using the Basquin relation. However, the Basquin relation is only capable of
describing the finite-life regime where a linear function is applicable. To characterize the effect
that the fatigue life tends to be infinity when the stress level is below the fatigue limit S0 (a low
stress level), a non-linear S-N equation is proposed by Stromeyer (Stromeyer & Dalby, 1914)
log10N = β0 + β1log10(S – S0) for S > S0. Both the Basquin and the Stromeyer equation are
deterministic models. To describe the fatigue life scatter dependency with applied stress levels,
some statistical models have been proposed, such as the random fatigue limit model (Pascual
& Meeker, 1999), the bilinear random fatigue limit model (D’Angelo & Nussbaumer, 2017),
and the 6-parameter random fatigue limit model (Leonetti, Maljaars, & Snijder, 2017). The
explicit regression models have the advantage that only a small number of parameters need to
be estimated. However, they impose restrictions for a certain type of functional relationship
between an applied loading and a fatigue life, which may or may not be true.
Due to the limitations of explicit regression models, machine learning models are used
with succuss in a variety of engineering fields, such as the solar radiation estimation (Karasu
& Altan, 2019; Karasu, Altan, Sarac, & Hacioglu, 2017), forecasting the future price of the
crude oil (Karasu, Altan, Bekiros, & Ahmad, 2020), and the cognitive decision-making (Altan
& Karasu, 2019). Among machine learning models, neural networks (NN) (Yaghobi, Rajabi
Mashhadi, & Ansari, 2011) are widely used to model the fatigue life with the stress level and
other influencing factors (e.g., mean stress, environmental factors) as inputs. A deterministic
relationship can be learned through the training process. The neural network is used for
predicting the fatigue life of steel in the corrosive environment in Ref. (Pleune & Chopra, 1996).
In their work, overtraining and bad extrapolation problems are observed and reported. The
combination of four neural networks is applied in estimating the finite-life fatigue strength and
the fatigue limit in Ref. (Artymiak, Bukowski, Feliks, Narberhaus, & Zenner, 1999) with inputs
including the stress level, notch factor, tensile strength, and yield strength. Another study
(Vassilopoulos, Georgopoulos, & Dionysopoulos, 2007) models the fatigue life of
multidirectional composite laminates using the neural network with one hidden layer and inputs
being the stress level, stress ratio, and material properties. In a different study (Figueira Pujol
& Andrade Pinto, 2011), the neural network model is fit to experimental data on the fatigue
life of steel under step-stress conditions. The constant life diagram for metallic materials is
generated using the neural network in (Barbosa, Correia, Júnior, & Jesus, 2020) for the high
cycle fatigue. In Ref. (Herzog, Marwala, & Heyns, 2009), a multi-layer perceptron neural
network is used for residual life prediction. The neural network models are flexible in modeling
the fatigue life - stress relation since no specific function types need to be assumed explicitly.
Those current attempts to model fatigue life using neural network fall in the deterministic
category. The probabilistic neural network was utilized in Ref. (Woźniak, et al., 2018) and
(Capizzi, Sciuto, Napoli, Połap, & Woźniak, 2020) in the medical field for classification. Good
correct classification rates can be achieved. It is worthwhile to explore the probabilistic neural
network for the regression analysis of fatigue data.
3
 The limitations of explicit regression models are simplified representations of the reality
due to incomplete knowledge, which introduce bias (Jia, et al., 2020). For the fatigue life
modeling, it is well known that factors except stress levels also have a dramatic influence on
the fatigue life, such as the surface roughness, temperature and corrosion (Guo, Ma, Wang,
Zhang, & E, 2020; Ma, Guo, Wang, & Zhang, 2020; Wu, Diao, Xu, Zhang, & Zhang, 2020;
Wu, Zhang, Diao, Zhang, & Xu, 2019). Without a thorough understanding of the mechanics of
the process, it is not an easy task to incorporate those influencing factors in explicit regression
models. Due to those limitations, machine learning models are considered as promising
alternatives to regression models. However, a direct application of black-box machine learning
models to the fatigue life modeling encounters the following issues: (i) Fatigue lives show a
significant scatter range even at well-controlled testing conditions. Most existing machine
learning models aim to find the mean response and are not capable of flexible variance
estimation at different fatigue loading conditions. (ii) The statistical relations between inputs
and outputs are solely learned from the data, which may violate some commonly known
physics laws or knowledge (Jia, et al., 2020); (iii) In addition, machine learning models may
overfit existing data and produce unreasonable results for scenarios outside the training data
range (e.g., pure extrapolation performance) (Pleune & Chopra, 1996).
To address the above-mentioned difficulties and deficiencies in existing two types of
models (explicit regression models and machine learning models), this paper proposes a novel
probabilistic physics-guided machine learning model for the fatigue data analysis. It aims to
impose known physics/knowledge constraints to the learning process of machine learning
models. The proposed model is called Probabilistic Physics-guided Neural Network (PPgNN)
hereafter. Correspond to the limitations existing in both types of models in the last paragraph,
the novelties of the proposed model are as follows: 1) Compared with explicit regression
models, influencing factors other than the stress level are easier to be incorporated in the model
by reconstructing the network architecture and the proposed model is not limited to a particular
regression function in handling fatigue data under different conditions. 2) Compared with
classic neural network models: (i) Fatigue life variances with respect to different stress levels
can be characterized. The confidence bounds can be obtained to show the increase of the
variance as the applied stress decreases. (ii) The trend of the fatigue curves is constrained
according to commonly known knowledge. By imposing physics-based constraints on
parameters and building a physics-based network architecture, the model is able to produce
both accurate and physically consistent results. (iii) The overfitting issue is avoided by
controlling the curvature of fitted curves from known patterns of fatigue S-N curves. Thus,
fatigue life predictions outside the training data range are more robust (e.g., better extrapolation
performance). (iv) What is more, both failure and runout data can be considered in the training
process. That is achieved by replacing the commonly used loss metric (e.g., mean square error)
by a custom loss function.
The paper is organized as follows. First, a brief introduction for neural networks is given
with basic terminologies and math formulations. Next, a probabilistic method for S-N curve
estimation using the neural network is proposed. The physics-guided machine learning method
is described including physics-based constraints on the neural network parameters and the
4
construction of the neural network architecture. Following that, extensive experiments are
conducted for model validations considering both single factor and multi-factor. Detailed
discussions are given for the effect of encoded physics-constraints on the results. The
necessities of the physics guidance in the neural network for the fatigue data analysis are
illustrated with examples where issues occur if the classic neural network is employed. Finally,
the conclusions and future work are drawn based on the current study.

2. Methodology

2.1 Brief Review of Neural networks

Neural networks, such as feedforward neural networks or multilayer perceptrons, are the
deep learning models. The goal of a neural network is to approximate some function f *. For
example, for regression, y = f * (x) maps an input x to a numerical y. A neural network defines
a mapping y = f (x; θ) and learns the value of the parameters θ that result in the best function
approximation. The network is called neural because it is loosely inspired by the neuroscience
(Goodfellow, Bengio, & Courville, 2016).
A typical single hidden layer neural network shown in Fig. 1 will be used as a
demonstration for the basic knowledge of neural networks and to give the preliminary before
the probability learning and imposing the physics guidance. This example consists of three
layers, each of which has a particular number of nodes (neurons). The first layer is called the
input layer and consists of a number of sensory neurons (neurons that make no processing, they
just sense incoming signals and pass them to the next layer). Note that this figure is for a two-
dimensional input x. The last layer (the one that produces final results of the network) is called
the output layer and consists of a number of computational neurons. There is one output in this
example. The layer between the input and output layer is called the hidden layer with a fixed
number of computational neurons (Vassilopoulos, et al., 2007). The layers are fully connected
in this example. Typically, the number of hidden units is somewhere in the range of 5 to 100,
with the number increasing as the number of inputs and the number of training cases increase.
Choices of the number of hidden layers are guided by the background knowledge and
experimentation (Hastie, Tibshirani, & Friedman, 2009). This neural network is called
feedforward because the information flows from the function being evaluated from x, through
the intermediate computations used to define f, and finally to the output y (Goodfellow, et al.,
2016).

5
 z 1( 2 )  a 1( 2 )

z 2( 2 )  a 2( 2 )
x
1

z 3( 2 )  a 3( 2 ) z 1( 3 )  a 1( 3 )
x
2

z 4( 2 )  a 4( 2 )

z 5( 2 )  a 5( 2 )

Layer 1 Layer 2 Layer 3

Input Layer Hidden Layer Output Layer

Fig. 1 An example of a single hidden layer neural network.

The mathematical expression for the neuron within each layer for Fig. 1 can be given in
the following equations (Xu & Yu, 2010).
pl 1
z k( l )  bk( l 1)   wkj( l 1) a (jl 1) l  2,..., L , (1)
j 1

 
ak(l )  g k(l ) zk(l ) k  1, 2,..., pl . (2)

where l from 2 to L is the index of the layer with l = 1 being the input layer and l = L being the
output layer. pl is the number of neurons at the lth layer. For the input layer,

ak(1)  xk k  1,..., p1 . (3)

gk(l )   is the activation function associated with the kth neuron at the lth layer. The activation

function of a neuron defines the output of that neuron given the inputs from the previous layer.
The intercepts and the coefficients of Eq. (1) are called biases and weights, respectively. The
biases and weights are obtained by training the network given the data. The loss function is
minimized during training. It is a quantity that represents a measure of success for the task
(Ketkar, 2017). The stochastic gradient descent method is employed for solving the
optimization problem. Stochastic gradient descent computes the gradient using subsets of the
data called minibatches. The weights and biases are updated minibatch by minibatch in one
cycle until the entire data set are gone through, and then followed by another cycle. Each cycle
is called an epoch.
Fig. 1 shows a standard neural network architecture, which is not necessarily the most
suitable one for a specific problem. The term neural network architecture refers to the
arrangement of neurons into layers and the connection patterns between layers, activation
functions, and learning methods. The neural network model and the architecture of a neural
6
network determine how a network transforms its inputs into outputs (Kalogirou, 2013). To
extend the standard neural network for the fatigue data analysis, an appropriate architecture
named Probabilistic Physics-guided Neural Network (PPgNN) is proposed and illustrated in
the next sections. Two major characteristics of the proposed architectures for fatigue life data
modeling are as follows. 1). The proposed architecture is designed to account for not only the
change of the mean of fatigue life with respect to the influence factors (e.g., the stress or strain)
but also the change of the scatter at different stress levels. This is achieved by assigning two
neurons at the output layer rather than one. They are the mean and standard deviation of fatigue
lives, respectively. Also, a custom loss function is proposed to consider both failure data and
censored data (runouts). 2). The proposed neural network produces physically consistent results
to account for the life-stress curvature and nonconstant variability. The neural network is
trained through a constrained optimization process. The constraints are imposed to guide the
neural network to follow existing physical knowledge during training. The above two main
characteristics show the probabilistic and physic-guidance features of the proposed PPgNN,
respectively. The proposed methodology is described in detail in the following two sub sections.
Firstly, the fatigue life data are modeled as the function of a single factor, i.e., stress or strain.
Next, the proposed PPgNN is extended to incorporate other fatigue life-influencing factors,
e.g., mean stress.

2.2 Probabilistic modeling of fatigue data in neural networks

The architecture of the probabilistic neural network is shown in Fig. 2. Three layers are
included in this network: the input layer, one hidden layer, and the output layer. The input layer
consists of stress or strain level (S), logarithmic of fatigue life (log (N)), and an index to indicate
if a data is a failure or runout. There are 5 neurons in the hidden layer. The number of neurons
in the hidden layer can be adjusted and is not necessarily to be 5 as shown in Fig. 2. We choose
this number to be 5 because of the consideration that fatigue S-N curve is not expected to have
complicated curve shapes and only one factor stress (strain) is considered. Thus, the minimum
number is chosen according to the recommendation (5 – 100 neurons per hidden layer (Hastie,
et al., 2009)). The probabilistic aspect of the neural network is reflected in the output layer.
Unlike the regular neural network that only learns the mean from the collected distributed data,
there are two output neurons in the output layer of the proposed network, both the mean and
standard deviation.

Activation function: linear

S
Activation function: tanh
μ
log (N) Activation function: elu + 1
σ
Activation function: none
Index

Fig. 2 Architecture of probabilistic neural network.

7
 Different colors of neurons in Fig. 2 indicate specific activation functions. For the input
layer, no activation functions are used by the definition of the neural network. The hyperbolic
tangent activation function (named tanh) is used for the neurons at the hidden layer. The
expression of tanh activation function is

e z  e z
g  z  . (4)
e z  e z
The tanh activation function is commonly used for the hidden layer neurons and gives good
results in this work. The mean (μ) in the output layer has the linear (i.e. identity) activation
function which is usually used for outputs in the regression problem,
g z  z . (5)

Due to the fact that the standard deviation has a non-negative value, the exponential linear unit
(elu) activation function is selected and modified by adding 1 as following
 z  1 z  0
g  z   z . (6)
e z  0
This is named elu+1 in this paper and has the property that it can transform an arbitrary number
to a non-negative value. Other activation functions that have this property also works, for
example, the absolute value function (Zychlinski, 2018). Elu+1 seems to work well in this work
as the validation later in this paper shows very good results.
In the neural network architecture in Fig. 2, the inputs are S, log (N) and Index. Only the S
input is passed to the next layers. The inputs log (N) and Index are not connected with the next
layer, and the data for these two inputs are used in the custom loss function for measuring the
performance of the model. The loss function is customized to be able to estimate both the mean
and standard deviation using all data including failures and runouts. The loss function is defined
as the negative logarithm of the likelihood (He, Chen, & Guan, 2020). In other words, the
neural network is trained to maximize the likelihood. The new loss function in this work is
proposed as
n
 
L    i log  f  log( N ) |  ,     1   i  log 1  F  log( N ) |  ,    , (7)
i 1

where δi is the index of a failure or runout, and

1 for failure
i   .
0 for runout
f (log (N) | μ, σ) and F (log (N) | μ, σ) are the probability density function and cumulative
distribution function respectively with the location parameter μ and scale parameter σ. The
normal distribution or smallest extreme value distribution can be adopted. The neural network
is used for regression in this work. The probabilistic characteristic is reflected in the
construction of the loss function. The negative log likelihood of the data is used for the loss
function. That is, the likelihood is maximized after training the neural network. A form of
8
distribution is presumed (normal distribution in this work) for the loss function. A normal
distribution contains two unknown parameters: the mean μ, and standard deviation σ. These
two parameters are the outputs. The normal distribution adopted is a reasonable distribution for
the log fatigue life. However, the distribution form can be changed according to applications
in different types of data. For example, if investigated data follow three-parameter Weibull
distribution, the number of outputs will be 3: shape, scale, and location parameter.
The architecture shown in Fig. 2 with the custom loss function is feasible to obtain the
mean curve and confidence bounds of the fatigue life data versus the stress (strain). The curves
are obtained with the maximum likelihood. However, the best estimated curves that are learned
solely from the data may present physically inconsistent curvature within the range of observed
data and will perform very badly for extrapolation. To overcome these drawbacks, physics
knowledge is imposed during the training process to guide the neural network to produce
physically reasonable results. In the case of the fatigue data analysis, the desired features of
S-N curves are expected to show the increase scatter with the decrease of stress level and an
infinity or specified large number of cycles at low stress levels. The proposed techniques for
imposing the physics guidance are described in the following subsection.

2.3 Physics-guided machine learning

There are two main considerations in modeling the fatigue life with the applied stress or
strain. First, the standard deviation of the fatigue life, in general, increases as the stress (strain)
decreases. Second, the curvature of the fatigue curve decreases as the stress decreases and
shows an asymptotic behavior near the fatigue limit or very long life if the material does not
have an apparent fatigue limit (Pascual & Meeker, 1999). These two aspects are the physics
knowledge for guiding the machine learning process to obtain the results with those desired
characteristics.
The physics knowledge is incorporated to the neural network by means of imposing
appropriate constraints on weights, biases or both. The derivations of different types of
constraints are as follows. According to the neural network architecture in Fig. 2 and
mathematical expressions in Eqs. (1) – (6), with the input stress (strain) level S, the kth neurons
in the hidden layer (the second layer) have the form of

zk(2)  bk(1)  wk(1)1 S  (8)

 
ak(2)  g k(2) zk(2) (9)

with g k(2)  z  being the tanh activation function shown in Eq. (4). The outputs μ and σ in the

output layer have the forms of

z1(3)  b1(2)   w1(2) (2)
j aj  (10)
j

 
  g1(3) z1(3) (11)

9
with g1(3)  z  being the linear activation function shown in Eq. (5), and

z2(3)  b2(2)   w2(2)j a (2)

j  (12)
j

  g 2(3) z2(3)  (13)

with g 2(3)  z  being the elu+1 activation function shown in Eq. (6).

The derivations of different types of constraints are as follows. From Eq. (8) – (13), it can
be seen that the outputs μ and σ are the functions of the input S. The first and second derivatives
of μ with respect to S are
 
d   dz1(3) da j dz j   
(2) (2)
d 4
 (3)   (2) (2)   1   w1(2) w(1) 
dS dz1  j da j dz j dS 
 
j 2 j1
 j z( 2)
e j e j
 z( 2) 
 , (14)
4
  w(1) (2)
j1 w1 j
 
2
z (2)
 z( 2)
j e j e j
and

 
2
d   4  dz (2)
  w(1) (2) j
j1 w1 j  2 
 
2
dS  e z j  e  z j  dS
(2) (2)
j
  , (15)

    w(1)  (2)
2 
8 e
z (j 2 )
e
 z (j 2 )

j1  w1 j
e 
3
z (j 2 )  z (j 2 )
j e

respectively. The first and second derivatives of σ with respect to S are

  
  4  (3)
1   w2 j
(2) (1)
w  z2  0
 
(2) 2
j1
  j z (2)

e j e j
 z 
d  dz2(3) da j dz j   
(2) (2)
d 
 (3)   (2) (2) 
dS dz2  j da j dz j dS    
 z2  4  (3)

( 3)
(2) (1)
 e  w w j1  z2  0
 
2j 2
 
 j z (j 2 )
e e
 z (j 2 )  , (16)
  
 1
 4 w j1 w2 j z ( 2 )
(1) (2)
z2(3)  0
 
(2) 2
z
 j e j e j


e z2  4 w(1) w(2)
( 3) 1
z2(3)  0
 
j1 2 j
 z ( 2 )
z ( 2 ) 2

j e j e j

10
 and

  
  1  dz (2)
 4 w(1) (2)
j1 2 j 
w 
j
z2(3)  0
 
(2) 2
 j  e z j  e  z j
(2)
 dS
  

d 2   z2( 3) dz2(3) da j dz j 
(2) (2)
1
   e    4 w(1) (2)
j1 w2 j 
 
2 (2) (2) (2) 2
dS   da dz dS 
 z (2)
 z
j j j j e e
j j



  
  1  dz (2)
 j1 2 j  z( 2)  z( 2) 2  dS z2(3)  0
z2( 3) (1) (2) j
 e 4 w w


j
 
 e j  e j 
 

 2 e 
z (j 2 )
 e
 z (j 2 )

 8  w j1  w2 j z ( 2 )
 (1)  (2) z2(3)  0
 
(2) 3
z
 j e j e j

 2
  
 ( 3)  1 
 e z2   4 w(1) w(2)
2

 
j1 2 j
  j z (j 2 )
e e
 z (j 2 ) 
   , (17)

 
 zj(2)
 z (j 2 )
 2 e  e
e z2  8  w(1)  (2)
( 3)
 j1  w2 j z2(3)  0
 
(2) 3
 z (2)
z

j e j e j

respectively.
For the first desired characteristic that the standard deviation of the fatigue life increases
as the stress (strain) decreases, it requires that
d
0. (18)
dS
From Eq. (16), the first derivative of σ is the summation of several terms. In this work, each
term is constrained to be non-positive to make sure the sum to be non-positive. This is a
somewhat stronger constraint than the original requirement, but it is easier to implement and
ensure the final results are consistent with the known physics. Thus,
w(1) (2)
j1 w2 j  0 . (19)

In other words,
w(1) (2)
j1  0, w2 j  0 (20)

or

11
 w(1) (2)
j1  0, w2 j  0 . (21)

For the second desired characteristic that the curvature of the fatigue curve decreases as
the stress (strain) decreases, it can be satisfied if

d 2
0. (22)
dS 2
Similarly to the constraints for the standard deviation, each term of Eq. (15) is restricted to be
non-negative, which gives the following constraints
w1(2) (2)
j  0, z j  0 (23)

or
w1(2) (2)
j  0, z j  0 . (24)

(2)
To have non-negative or non-positive values for z j , from Eq. (8), since the stress (strain) is

always positive, if
b(1) (1)
j  0, w j1  0 , (25)

then

z (2)
j 0 (26)

can be guaranteed. And similarly, if

b(1) (1)
j  0, w j1  0 , (27)

then

z (2)
j  0. (28)

Thus, for the curvature to be physically consistent, either set of the following constraints needs
to be satisfied
b(1) (1) (2)
j  0, w j1  0, w1 j  0 (29)

or
b(1) (1) (2)
j  0, w j1  0, w1 j  0 . (30)

From Eq. (15) and (17), if the second derivative of μ is constrained to be non-negative, the
second derivative of σ will also have the non-negative value. Experimental data for fatigue
curves does not show the evidence for the increase rate of the standard deviation with the
decrease of the stress (second derivative of σ). To avoid the correlation between the second
derivatives of μ and σ, the architecture is redesigned to separate the neurons that are connected
to μ and σ in the hidden layer. The updated architecture is shown in Fig. 3. The connection lines

12
are with different colors to show different types of constraints corresponding to constraints (20)
and (29) (or constraints (21) and (30)).

Constraint: weight and bias

Constraint: weight

S μ
Activation function: linear

log (N) Activation function: tanh

Activation function: elu + 1

Index
Activation function: none
σ

Fig. 3 Architecture of probabilistic neural network with weight and bias constraints.
The proposed neural network with the weight or bias constraints can produce the results
which satisfy that the curvature decreases as the stress decreases. Typical S-N curve shows an
asymptotic behavior near the fatigue limit. This means that the second derivative of μ tends to
be zero as the stress (strain) decreases approaching fatigue limit. However, due to the over
restrictive constraints for constraint (22) (i.e., each term is constrained to be non-negative to
guarantee the sum to be non-negative), the minimum value of the second derivative of μ is a
positive value. To relax the curvature, the architecture is modified by adding another layer with
a relaxed μr and connecting μ and σ to μr. The μr and σ are the final outputs. The modified
architecture is shown in Fig. 4 and is used as the proposed architecture of Probabilistic Physics-
guided Neural Network for single factor (S). The weights of the third layer are constrained to
be non-negative in order not to change the constraints at previous layers. The second derivative
of μr is the sum of the second derivatives of μ and σ. Since the second derivative of σ is
unconstrained, it can have either a positive or negative value, which provides the relaxation on
the second derivative of μr.

13
 Constraint: weight and bias

Constraint: weight
S μ
Activation function: linear
log (N) Activation function: tanh
μr

Activation function: elu + 1

Index
σ Activation function: none

Fig. 4 Architecture of probabilistic neural network with stress (strain) input.

The architecture shown in Fig. 4 is for single factor, i.e., stress (or strain) vs. life. An
advantage of proposed physics-guided machine learning model compared with classical
regression type model is the flexibility and scalability to consider the influence of other factors.
Other inputs can be added to the network according to the known physical knowledge. An
example is to consider the mean stress (Sm) effect. For a given fatigue load range, a tensile
mean stress has a detrimental effect on the fatigue strength, whereas, in general, a
compressive mean stress has a beneficial effect (Kumbhar & Tayade, 2014). The mean fatigue
life increases with the reduction of mean stresses. The influence of mean stress on the scatter
is unclear. Thus, the first derivative of μ with respect to the mean stress is non-positive, which
gives constraints analogous to constraint (20) and (21). Since no physics knowledge is given
to restrict the mean stress – variance relationship, no constraint is added, and the relationship
is obtained solely from the data training. The architecture to consider the mean stress is given
in Fig. 5.
In the next section (Section 3 Experimental validation), various data sets are used to
validate the proposed PPgNN architecture in Fig. 4 for the single factor experiment and Fig. 5
for the multi-factor experiment. Following the experimental validation, in Section 4
Discussions, the necessities for the physics guidance is described, and the comparison between
results from the neural network without and with physics guidance is discussed.

14
 Constraint: weight and bias
S

Constraint: weight
log (N)
μ
Constraint: none
Index

μr Activation function: linear

Activation function: tanh

σ Activation function: elu + 1

Activation function: none

Sm

Fig. 5 Architecture of probabilistic neural network with multiple inputs.

3. Experimental validation
In this section, extensive evaluations of the proposed PPgNN are conducted using various
fatigue data sets from open literature for both single factor and multi-factor cases.
The learning algorithm’s parameters are kept same for all the following experiments with
different data sets and are shown in Table 1.
Table 1 Learning algorithm’s parameters.
Epochs 500 for single factor experiment; 1000 for multi-factor experiment
Minibatch size 1
Learning rate 0.001
15
 Six different fatigue data sets are collected, which covers a wide range of metallic and
composite materials. Table 2 is a summary of the collected data.
Table 2 Summary of the data set used for validation.
Material Reference Sample size No. of Runouts
(Pascual &
Meeker,
1999;
Carbon eight-harness-satin/epoxy laminate Shimokawa 125 10
&
Hamaguchi,
1987)
(Pascual &
Meeker,
Nickel-base superalloy 246 4
1999; Shen,
1994)
(Shen,
Annealed aluminum 200 NA
1994)
(Shen,
Steel 75 10
1994)
(Shen,
Al 2024-T4 252 NA
1994)
(Zhao &
Al 7075-T651 131 8
Jiang, 2008)

3.1 Single factor PPgNN validation

For single factor experiment, all probabilistic S-N curves using the Probabilistic Physics-
guided Neural Network (PPgNN) are shown in Fig. 6 for various materials. All left column
figures are for the mean and 95% confidence bounds plot with experimental data. The solid
lines and dash lines represent the mean S-N curves and 95% confidence intervals (CI),
respectively. x-axis is the fatigue life and y-axis is the stress/strain. “Runout” data are
highlighted in the figures. All right column figures are training convergence plots using the
proposed loss function. x-axis is the Epoch number and y-axis is the loss value. Very good
convergence behavior is observed using the proposed loss function.
By using the PPgNN model, the desired results (i.e., not violating known physics about
the fatigue performance of a material) can be achieved. With the physics constraints in the
neural network, the obtained P-S-N curves show consistent physics results. First, the standard
deviation of fatigue life increases as the stress (strain) decreases. Second, the curvature of the
fatigue curve decreases as the stress decreases and shows an asymptotic behavior near the
fatigue limit or very long life if the material does not have an apparent fatigue limit. PPgNN
model is shown to be a model suitable for the fatigue life data fitting for a variety of different
materials. Also, the PPgNN is not limited to a particular stress (strain) region. For example, the
nickel-base superalloy data (Fig. 6 (b)) was used for P-S-N curve fitting in Ref. (Pascual &
Meeker, 1999) using the random fatigue limit model which belongs to the explicit regression
model category. This model did not fit data well in the high-strain region (strain above 0.007)

16
(Pascual & Meeker, 1999). A possible reason is that this type of model has a fixed function
form which may not be appropriate for all strain regions. This issue does not exit using the
proposed PPgNN model due to the flexibility of the neural network. Thus, both low-strain and
high-strain data can be used together for the P-S-N curve fitting. In addition, both deterministic
and probabilistic fatigue properties can be obtained simultaneously using the same framework.
Not only the mean S-N curves but also confidence bounds can be estimated from the data. It
can be seen that almost all the data points are located within the 95% CI.

(a) Carbon eight-harness-satin/epoxy laminate.

(b) Nickel-base superalloy.

17
 (c) Annealed aluminum.

(d) Steel.

(e) Al 2024-T4.
Fig. 6 Validation of PPgNN using single factor experimental data for various materials.
18
3.2 Multi-factor PPgNN validation
One advantage of the Probabilistic Physics-guided Neural Network (PPgNN) compared
with explicit regression models is that it is easier to incorporate other influencing factors other
than stress (strain). This experiment is to show the effectiveness of the PPgNN shown in Fig.
5 with multi-factor inputs.
The data set used contains strain level, mean stress, and fatigue life (Zhao & Jiang, 2008).
The material is 7075-T651 aluminum alloy. The fatigue testing is conducted under uniaxial
compression-compression, tension-compression, and tension-tension fatigue loading
conditions. The mean stress can have negative value, zero, or positive value. A total of 131
data are used for fitting among which 8 data are right-censored (runouts).
The architecture proposed in Fig. 5 is used for the curve fitting. The results are shown in
Fig. 7. Fig. 7 (a), (b) and (c) are the probabilistic S-N curves with 95% confidence intervals for
Sm = -250 MPa, 0 MPa and 150 MPa, respectively. The testing data corresponding to each
mean stress case are also plotted. It can be seen that for each of the three mean stress cases,
almost all data points are located within the 95% confidence interval. Again, the results show
that the standard deviation of fatigue life increases and the curvature of the fatigue curve
decreases as the strain decreases. Fig. 7 (d) is the model training history, the loss function value
vs. the epoch.
Ref. (Pascual, 2003) extended the random fatigue limit model (Pascual & Meeker, 1999)
to be able to consider influencing factors other than stress (strain) level, i.e., multi-factor fatigue
data analysis. This is a type of explicit regression models, and the function form needs to be
known prior to data fitting. Thus, to incorporate multiple factors in the model, data need to be
pre-analyzed to find a reasonable functional form. However, using the PPgNN model, those
pre-analyses are not mandatory. The neural network in the PPgNN model learns the function
from the data, and constraints in PPgNN ensures that the results are not overfitted with the
guidance of physics knowledge.

(a) (b)

19
 (c) (d)
Fig. 7 Validation of PPgNN using multi-factor experimental data with mean stress effect.

4. Discussions
The PPgNN architecture is validated using fatigue testing data for single and multi-factor
in Section 3. In his section, we discuss the impacts/benefits for the physics guidance in the
neural network for fatigue data analysis by comparing results from the neural network in Fig.
2 (without physics guidance) and Fig. 4 and Fig. 5 (with physics guidance). Also, the
effectiveness of the relaxation on the mean prediction by adding another output layer is
discussed and the results from the neural network architectures in Fig. 3 (without relaxation on
mean) and Fig. 4 (with relaxation on mean) are compared.

4.1 Impact of the physics guidance in PPgNN

The laminate panel data set is used again for the illustration of the necessities for imposing
physics knowledge in the neural network. The neural network architecture in Fig. 2 produces
results that are learned solely from the training data. Without any physics guidance, the results
are shown in Fig. 8 (a). The improvements of the proposed PPgNN compared with the neural
network without any physics guidance are reflected in the curvature and extrapolation
performance below the minimum testing stress. The slope dμ / dS in Fig. 8 (a) does not change
in a monotonic manner. Below some stress level (about 260 MPa), dμ / dS increases as the
stress decreases. This is contradictory to the common knowledge as the slope will increase as
the stress decreases. If there is a fatigue limit, the S-N curve will asymptotically approach
infinity for low stress values. The reason for obtaining the physics inconsistent results is due
to the overfitting. As can be seen in Fig. 6 (a), fatigue life data are distributed at five stress
levels. The overfitted mean S-N curve can pass through the mean of collected data at each of
the five stress levels. Due to the data censoring, the mean fatigue life at the low stress levels
calculated from the collected failure data cannot reflect the true distribution mean. Thus, by
tracking the sample mean, the overfitted S-N curve cannot have the desired monotonic slope

20
decrease as stress level decreases. This issue is solved by imposing this physics knowledge
while training the neural network, which is done by solving the optimization problem with bias
and weight constraints. The improved results obtained from PPgNN are shown in Fig. 8(b). It
can be seen that dμ / dS decreases monotonically as the stress decreases, and the S-N curves
tends to be flat when approaching the fatigue limit.

(a) (b)
Fig. 8 Comparison of slope behavior of (a) classical NN without physics guidance and (b)
proposed PPgNN with physics guidance.
We also refit the multi-factor experiment using Al 7075-T651 data and the neural network
with no physics knowledge. The neural network architecture without physics guidance is
similar to that in Fig. 2. The input layer contains S: strain, mean stress, log (N) and Index. One
hidden layer with 20 neurons which is the same number as that in the architecture in Fig. 5.
The output layer has μ and σ. The results are shown in Fig. 9 (a) for the variance variation. It
is seen that the standard deviation for each of the three mean stress cases does not decrease
monotonically as the strain level increases. This is also due to the overfitting. Without physics
knowledge guidance (i.e., bias and weight constraints), the neural network is learned solely
from the data. Since the sample size is not big enough, the overfitting issue occurs. The
prediction with physics guidance is shown in Fig. 9 (b) and it is clearly seen that the variance
shows a monotonic behavior. It is also interesting to see that the uncertainty in this material is
independent of mean stress with the proposed method.

21
 (a) (b)
Fig. 9 Comparison of variance variation of (a) classical NN without physics guidance and (b)
proposed PPgNN with physical guidance.

4.2 Impact of the relaxation layer in PPgNN

The nickel-base superalloy data set is used to illustrate the effectiveness of the relaxation
on the mean of fatigue life by adding another output layer with the neuron μr and connecting
μr with μ and σ at the previous layer. The results using the neural network architecture in Fig.
3 without relaxation on μ are plotted in Fig. 10 (a). Compared with the results in Fig. 10 (b)
using the architecture with relaxation on μ, an obvious difference is reflected in the low strain
region. In this region, the fatigue life data tend to asymptotically approach the fatigue limit and
the curvature should be close to zero. For Fig. 10 (a), the mean S-N curve can hardly capture
this trend of the data. This is due to the over restrictive constraints for constraint (22). With the
over-restricted constraints, the second derivative of μ with respect to the strain cannot achieve
the value of zero, which leads to that the first derivative of μ with respect to the strain cannot
achieve an adequate small negative value. By adding another output layer with the neuron μr
and connecting μr with μ and σ at the previous layer, the imposed over-restricted constraints
can be relaxed and the fitted S-N curve has more flexibility to follow the trend of the data at
the low strain level. The values of dμ / dS are shown in Fig. 10 (c) and (d) for the two approaches.
It is clearly seen that the proposed PPgNN with relaxation can successfully predict zero
curvature at a very low strain region.

22
 (a) (b)

(c) (d)
Fig. 10 Comparison prediction results of (a) strain-life prediction without relaxation in NN,
(b) strain-life prediction with relaxation in NN, (c) dμ / dS prediction without relaxation in
NN, and (d) dμ / dS prediction with relaxation in NN.

4.3 Remarks
The work in this paper is under the general topic “Physics-guided data science”. Data
science models, although successful in several domains, have difficulties in many scientific
and engineering problems involving complex physical phenomena, especially when the
available data are limited due to high cost for generating such results. One example is the high
cycle fatigue failure of materials and structures in this study, where the time and expense
associated with experimental data are prohibitive for a pure data-based learning. On the other
hand, pure physics-based models may suffer from a simplified representation or partial
understanding of physical phenomena. The physics-guided data science aims to leverage the
scientific knowledge for improving the effectiveness of data science models in enabling
scientific discoveries and engineering applications (Karpatne, et al., 2017). Within this topic,
23
we focus on developing the physics-guided neural network model for fatigue data analysis,
specifically, probabilistic stress-life curve estimations. We attempt to develop this model
because limitations exist in both physics-based models and neural network models for fatigue
data analysis. That is, a physics-based model has difficulty in handling multi-factor inputs, and
a neural network model overfits the data which leads to the physics violation. Thus, a physics-
guided machine learning model is a natural solution.
A difficulty in achieving physics-guided machine learning is how to incorporate physics
knowledge into machine learning models. This paper proposes to solve this problem by using
scientific knowledge as constraints in optimization processes and by designing appropriate
neural network architectures. There are other ways of achieving the combination between
physics knowledge and machine learning models, such as using physically consistent solutions
as initial points in iterative learning algorithms (e.g., gradient descent methods) (Yao, Gao, &
Liu, 2020), restricting the space of probabilistic models with the help of theory-guided priors
(Yu, Yao, & Liu, 2019), encoding scientific knowledge as regularization terms in the objective
function (Yu, Yao, & Liu, 2020), and refining outputs of machine learning models to be
consistent with physics knowledge (Karpatne, et al., 2017).
The physics-guided machine learning is proposed and applied for fatigue data analysis in
this work. It should be noted that physics-guided machine learning models are also applicable
to a wide variety of scientific and engineering problems where a direct application of machine
learning models fails if the scientific knowledge is ignored and when the available data is
limited. For example, in personalized medicine, physics-guided machine learning can be used
to classify patients into specific treatment regimens. While this is typically done by genome
profiling alone, greater classification power can be achieved if the training data are augmented
with simulations based on biological or physical principles compared with models solely built
on observed data (Alber, et al., 2019). Another example is to build predictive models that use
multi-spectral data from satellite images as input features to classify pixels of the image as
water or land. The pure machine learning models suffer from the poor quality of labeled data,
noise, and missing values in remote sensing signals, etc. The quality of classification maps can
be improved by bringing the domain knowledge that water bodies have a concave elevation
structure (Karpatne, et al., 2017). In aeronautical engineering, the simulation costs can be
expensive or even prohibitive. The physics-based machine learning methods outperform
classical numerical methods in terms of computational efficiency and data-driven methods in
terms of both training cost and prediction accuracy (Yu, et al., 2019).

5. Conclusions
The Probabilistic Physics-guided Neural Network (PPgNN) proposed in this paper
provides a flexible and robust tool for probabilistic fatigue S-N estimation. Compared with an
explicit regression model, the PPgNN is not restricted to a single function format when fitting
with multiple factors and at different stress/strain regimes. It also avoids the non-physical
predictions occurring for the machine learning model and produces both accurate and
physically consistent results.
Several major conclusions can be obtained from the current study using investigated data:
24
 - Both the mean and variance can be accurately predicted using the proposed PPgNN
architecture with a wide range of fatigue data from both metallic and composite
materials. Thus, it appears that the proposed framework is a universal approach not
limited to a single material system.
- The proposed new loss function, which integrates both failure and runout data,
performs well and shows good convergence behavior for all investigated materials.
- It is shown that the embedded physics guidance is important for fatigue data analysis,
especially for extrapolation behavior. The proposed study offers a unique and flexible
way to combine data analytics and prior physics knowledge for fatigue data analysis.
- It appears that the relaxation in NN is necessary to reduce the enforced constraints for
some materials; it is shown that the relaxation is important for fitting near the fatigue
limit region.
- The proposed PPgNN is scalable and can easily extend to include other factors
affecting fatigue performance, such as mean stress, frequency, corrosion, etc., by
adding additional nodes and layers.
It should be noted that the proposed framework is not limited to the fatigue data analysis
and it can be applied to other survival data analysis by adjusting the network parameter
constraints and the network architecture according to the physics knowledge. For example,
creep-life data or stress-corrosion cracking (SCC) life data. Further validations are needed for
the above life prediction applications. The following extensions deserve to be explored for
future work. 1). The constraints according to the physics requirements are over-restrictive for
the implementation convenience of coding. This paper solves this issue by adding a relaxation
layer. It would be useful to explore the use of other types of constraints in constructing the NN.
2). The current investigation focuses on the application for a single hierarchy, i.e., no multiscale
data. It will be interesting to explore the possibility to extend the framework to deal with the
multi-fidelity and multiscale datasets with an imbalanced sample size. This will, at least to the
authors’ opinion, offer more powerful analysis tools for integrated computational material
engineering under uncertainties.
The novelty of the proposed PPgNN model is that the physics knowledge in the fatigue
field can guide the training process of the neural network. This paper proposes a method for
transforming the physics knowledge to constraints on biases and weights of the neural network.
The accomplishment of this transformation is specific to the neural network model. Future
work can be imposing the physics knowledge onto other types of machine learning model, such
as the support vector machine and k-nearest neighbors. Then, the performance of different
types of physics guided machine learning models can be compared.

Acknowledgements
The research is partially supported by fund from NSF (award No: 1536994). The support
is greatly appreciated.

25
References
Alber, M., Buganza Tepole, A., Cannon, W. R., De, S., Dura-Bernal, S., Garikipati, K., Karniadakis,
G., Lytton, W. W., Perdikaris, P., Petzold, L., & Kuhl, E. (2019). Integrating machine learning
and multiscale modeling—perspectives, challenges, and opportunities in the biological,
biomedical, and behavioral sciences. npj Digital Medicine, 2, 115.
Altan, A., & Karasu, S. (2019). The effect of kernel values in support vector machine to forecasting
performance of financial time series. The Journal of Cognitive Systems, 4, 17-21.
Artymiak, P., Bukowski, L., Feliks, J., Narberhaus, S., & Zenner, H. (1999). Determination of S–N
curves with the application of artificial neural networks. Fatigue & Fracture of Engineering
Materials & Structures, 22, 723-728.
Barbosa, J. F., Correia, J. A. F. O., Júnior, R. C. S. F., & Jesus, A. M. P. D. (2020). Fatigue life prediction
of metallic materials considering mean stress effects by means of an artificial neural network.
International Journal of Fatigue, 135, 105527.
Basquin, O. (1910). The exponential law of endurance tests. Proc Am Soc Test Mater, 10, 625-630.
Capizzi, G., Sciuto, G. L., Napoli, C., Połap, D., & Woźniak, M. (2020). Small Lung Nodules Detection
Based on Fuzzy-Logic and Probabilistic Neural Network With Bioinspired Reinforcement
Learning. IEEE Transactions on Fuzzy Systems, 28, 1178-1189.
Chen, J., Diao, B., He, J., Pang, S., & Guan, X. (2018). Equivalent surface defect model for fatigue life
prediction of steel reinforcing bars with pitting corrosion. International Journal of Fatigue, 110,
153-161.
Chen, J., Imanian, A., Wei, H., Iyyer, N., & Liu, Y. (2020). Piecewise stochastic rainflow counting for
probabilistic linear and nonlinear damage accumulation considering loading and material
uncertainties. International Journal of Fatigue, 140, 105842.
Chen, J., Liu, S., Zhang, W., & Liu, Y. (2020). Uncertainty quantification of fatigue S-N curves with
sparse data using hierarchical Bayesian data augmentation. International Journal of Fatigue,
134, 105511.
Chen, J., & Liu, Y. (2020). Uncertainty quantification of fatigue properties with sparse data using
hierarchical Bayesian model. In AIAA Scitech 2020 Forum: American Institute of Aeronautics
and Astronautics.
D’Angelo, L., & Nussbaumer, A. (2017). Estimation of fatigue S-N curves of welded joints using
advanced probabilistic approach. International Journal of Fatigue, 97, 98-113.
Figueira Pujol, J. C., & Andrade Pinto, J. M. (2011). A neural network approach to fatigue life
prediction. International Journal of Fatigue, 33, 313-322.
Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning: MIT press.
Guo, Z., Ma, Y., Wang, L., Zhang, J., & E, H. (2020). Corrosion fatigue crack propagation mechanism
of high strength steel bar in various environments. Journal of Materials in Civil Engineering,
32, 04020115.
Hastie, T., Tibshirani, R., & Friedman, J. (2009). The elements of statistical learning: data mining,
inference, and prediction: Springer Science & Business Media.
He, J., Chen, J., & Guan, X. (2020). Lifetime distribution selection for complete and censored multi-
level testing data and its influence on probability of failure estimates. Structural and
Multidisciplinary Optimization, 1-17.

26
Herzog, M. A., Marwala, T., & Heyns, P. S. (2009). Machine and component residual life estimation
through the application of neural networks. Reliability Engineering & System Safety, 94, 479-
489.
Jia, X., Willard, J., Karpatne, A., Read, J. S., Zwart, J. A., Steinbach, M., & Kumar, V. (2020). Physics-
Guided Machine Learning for Scientific Discovery: An Application in Simulating Lake
Temperature Profiles. arXiv preprint arXiv:2001.11086.
Kalogirou, S. A. (2013). Solar energy engineering: processes and systems: Academic Press.
Karasu, S., & Altan, A. (2019). Recognition Model for Solar Radiation Time Series based on Random
Forest with Feature Selection Approach. In 2019 11th International Conference on Electrical
and Electronics Engineering (ELECO) (pp. 8-11).
Karasu, S., Altan, A., Bekiros, S., & Ahmad, W. (2020). A new forecasting model with wrapper-based
feature selection approach using multi-objective optimization technique for chaotic crude oil
time series. Energy, 212, 118750.
Karasu, S., Altan, A., Sarac, Z., & Hacioglu, R. (2017). Prediction of solar radiation based on machine
learning methods. The Journal of Cognitive Systems, 2, 16-20.
Karpatne, A., Atluri, G., Faghmous, J. H., Steinbach, M., Banerjee, A., Ganguly, A., Shekhar, S.,
Samatova, N., & Kumar, V. (2017). Theory-guided data science: A new paradigm for scientific
discovery from data. IEEE Transactions on knowledge and data engineering, 29, 2318-2331.
Ketkar, N. (2017). Deep Learning with Python (Vol. 1): Springer.
Kim, Y.-H., Song, J.-H., & Park, J.-H. (2009). An expert system for fatigue life prediction under
variable loading. Expert Systems with Applications, 36, 4996-5008.
Kumbhar, S., & Tayade, R. (2014). A case study on effect of mean stress on fatigue life. International
Journal of Engineering Development and Research, 2, 304-308.
Leonetti, D., Maljaars, J., & Snijder, H. H. (2017). Fitting fatigue test data with a novel S-N curve using
frequentist and Bayesian inference. International Journal of Fatigue, 105, 128-143.
Liu, X., Xuan, F.-Z., Si, J., & Tu, S.-T. (2008). Expert system for remnant life prediction of defected
components under fatigue and creep–fatigue loadings. Expert Systems with Applications, 34,
222-230.
Liu, Y., Liu, L., Stratman, B., & Mahadevan, S. (2008). Multiaxial fatigue reliability analysis of railroad
wheels. Reliability Engineering & System Safety, 93, 456-467.
Liu, Y., & Mahadevan, S. (2009). Efficient Methods for Time-Dependent Fatigue Reliability Analysis.
AIAA Journal, 47, 494-504.
Ma, Y., Guo, Z., Wang, L., & Zhang, J. (2020). Probabilistic life prediction for reinforced concrete
structures subjected to seasonal corrosion-fatigue damage. Journal of Structural Engineering,
146, 04020117.
Pascual, F. (2003). The random fatigue-limit model in multi-factor experiments. Journal of Statistical
Computation and Simulation, 73, 733-752.
Pascual, F., & Meeker, W. (1999). Estimating Fatigue Curves with the Random Fatigue-Limit Model.
Technometrics, 41, 277-290.
Pleune, T. T., & Chopra, O. K. (1996). Artificial neural networks and the effects of loading conditions
on fatigue life of carbon and low-alloy steels. In: Oak Ridge Inst. for Science and Education,
TN (United States).

27
Shen, C.-L. (1994). The statistical analysis of fatigue data. In P. H. Wirsching (Ed.): ProQuest
Dissertations Publishing.
Shimokawa, T., & Hamaguchi, Y. (1987). Statistical evaluation of fatigue life and fatigue strength in
circular- hole notched specimens of a carbon eight-harness-satin/epoxy laminate. Statistical
research on fatigue, 159-176.
Stromeyer, C. E., & Dalby, W. E. (1914). The determination of fatigue limits under alternating stress
conditions. Proceedings of the Royal Society of London. Series A, Containing Papers of a
Mathematical and Physical Character, 90, 411-425.
Vassilopoulos, A. P., Georgopoulos, E. F., & Dionysopoulos, V. (2007). Artificial neural networks in
spectrum fatigue life prediction of composite materials. International Journal of Fatigue, 29,
20-29.
Woźniak, M., Połap, D., Capizzi, G., Sciuto, G. L., Kośmider, L., & Frankiewicz, K. (2018). Small lung
nodules detection based on local variance analysis and probabilistic neural network. Computer
Methods and Programs in Biomedicine, 161, 173-180.
Wu, J., Diao, B., Xu, J., Zhang, R., & Zhang, W. (2020). Effects of the reinforcement ratio and chloride
corrosion on the fatigue behavior of RC beams. International Journal of Fatigue, 131, 105299.
Wu, J., Zhang, R., Diao, B., Zhang, W., & Xu, J. (2019). Effects of pre-fatigue damage on high-cycle
fatigue behavior and chloride permeability of RC beams. International Journal of Fatigue, 122,
9-18.
Xu, H., & Yu, B. (2010). Automatic thesaurus construction for spam filtering using revised back
propagation neural network. Expert Systems with Applications, 37, 18-23.
Yaghobi, H., Rajabi Mashhadi, H., & Ansari, K. (2011). Artificial neural network approach for locating
internal faults in salient-pole synchronous generator. Expert Systems with Applications, 38,
13328-13341.
Yao, H., Gao, Y., & Liu, Y. (2020). FEA-Net: A physics-guided data-driven model for efficient
mechanical response prediction. Computer Methods in Applied Mechanics and Engineering,
363, 112892.
Yu, Y., Yao, H., & Liu, Y. (2019). Aircraft dynamics simulation using a novel physics-based learning
method. Aerospace Science and Technology, 87, 254-264.
Yu, Y., Yao, H., & Liu, Y. (2020). Structural dynamics simulation using a novel physics-guided
machine learning method. Engineering Applications of Artificial Intelligence, 96, 103947.
Zhao, T., & Jiang, Y. (2008). Fatigue of 7075-T651 aluminum alloy. International Journal of Fatigue,
30, 834-849.
Zychlinski, S. (2018). Predicting probability distributions using neural networks. In: Taboola
Engineering.

28
 Jie Chen: Conceptualization, Methodology, Software, Validation, Formal analysis,
Investigation, Writing - Original Draft, Writing - Review & Editing
Yongming Liu: Conceptualization, Validation, Writing - Review & Editing, Supervision

29
Highlights

 Probabilistic Physics-guided Neural Network (PPgNN) is proposed for fatigue data.

 PPgNN is flexible and does not impose restrictions on function types.

 PPgNN includes known physics/knowledge constraints in the machine learning model.

 PPgNN produces both accurate and physically consistent results.

 Extensive experiments for fatigue P-S-N curves are conducted for model validation.

30