Bayesian fatigue damage and reliability analysis using Laplace approximation and

inverse reliability method

Xuefei Guan 1 , Jingjing He 2 , Ratneshwar Jha 1 , Yongming Liu 2
1
Department of Mechanical & Aeronautical Engineering, Clarkson University, Potsdam, NY, 13699, USA
[email protected]
[email protected]
2
Department of Civil & Environmental Engineering, Clarkson University, Potsdam, NY, 13699, USA
[email protected]
[email protected]

ABSTRACT The simulation-based method includes direct Monte Carlo

This paper presents an efficient analytical Bayesian method (MC) (Kalos & Whitlock, 2008), Importance Sampling
for reliability and system response estimate and update. The (IS)(Gelman & Meng, 1998; Liu, 1996), and other MC sim-
method includes additional data such as measurements to re- ulations with different sampling techniques. Analytical ap-
duce estimation uncertainties. Laplace approximation is pro- proximation methods, such as first- and second- order relia-
posed to evaluate Bayesian posterior distributions analyti- bility methods (FORM/SORM) have been developed to esti-
cally. An efficient algorithm based on inverse first-order re- mate the reliability without large numbers of MC simulations.
liability method is developed to evaluate system responses FORM and SORM computations are based on linear (first-
given a reliability level. Since the proposed method in- order) and quadratic (second-order) approximations of the
volves no simulations such as Monte Carlo or Markov chain limit-state surface at the most probable point (MPP)(Madsen
Monte Carlo simulations, the overall computational efficiency et al., 1986; Ditlevsen & Madsen, 1996). Under the condition
improves significantly, particularly for problems with com- that the limit-state surface at the MPP is close to its linear
plicated performance functions. A numerical example and or quadratic approximation and that no multiple MPPs ex-
a practical fatigue crack propagation problem with experi- ist in the limit-state surface, FORM/SORM are sufficiently
mental data are presented for methodology demonstration. accurate for engineering purposes(Bucher et al., 1990; Cai
The accuracy and computational efficiency of the proposed & Elishakoff, 1994; Zhang & Mahadevan, 2001; Zhao &
method is compared with simulation-based methods. Ono, 1999). If the final objective is to calculate the sys-
tem response given a reliability index, the inverse reliabil-
ity method can be used. The most well-known approach is
1. I NTRODUCTION inverse FORM method proposed in (Der Kiureghian, Yan,
Efficient inference on reliability and responses of engineer- & Chun-Ching, 1994; Der Kiureghian & Dakessian, 1998;
ing systems has drawn attention to the prognostics and health Li & Foschi, 1998). Several studies for static failure us-
management society due to the increasing complexity of those ing the inverse FORM method have been reported in the
systems (Melchers, 1999; Brauer & Brauer, 2009). For high literature. (Du, Sudjianto, & Chen, 2004) proposed an in-
reliability demanding systems such as aircraft and nuclear verse reliability strategy and applied it to the integrated robust
facilities, time-dependent reliability degradation and perfor- and reliability design of a vehicle combustion engine piston.
mance prognostics must be quantified to prevent potential (Saranyasoontorn & Manuel, 2004) developed an inverse re-
system failures. Reliable predictions of system reliability and liability procedure for wind turbine components. (Lee, Choi,
system responses are usually required for decision-making in Du, & Gorsich, 2008) used the inverse reliability analysis
a time and computational resource constrained situation. The for reliability-based design optimization of nonlinear multi-
basic idea of time-independent component reliability analysis dimensional systems. (Cheng, Zhang, Cai, & Xiao, 2007)
involves computation of a multi-dimensional integral over the presented an artificial neural network based inverse FORM
failure domain of the performance function (Madsen, Krenk, method for solving problems with complex and implicit per-
& Lind, 1986; Ditlevsen & Madsen, 1996; Rackwitz, 2001). formance functions. (Xiang & Liu, 2011) applied the inverse
For many practical problems with high-dimensional parame- FORM method to time-dependent fatigue life predictions.
ters, the exact evaluation of this integral is either analytically Conventional forward and inverse reliability analysis is
intractable or computationally infeasible with a given time based on the existing knowledge about the system (e.g., un-
constraint. Analytical approximations and numerical simula- derlying physics, distributions of input variables). Time-
tions are two major computational methods to solve this prob- dependent reliability degradation and system response chang-
lem (Rebba & Mahadevan, 2008). ing are not reflected. For many practical engineering prob-
lems, usage monitoring or inspection data are usually avail-
Xuefei Guan et.al. This is an open-access article distributed under the terms
of the Creative Commons Attribution 3.0 United States License, which per-
able at a regular time interval either via structural health mon-
mits unrestricted use, distribution, and reproduction in any medium, provided itoring system or non-destructive inspections. The new in-
the original author and source are credited. formation can be used to update the initial estimate of sys-

1
 Annual Conference of the Prognostics and Health Management Society, 2011

tem reliability and responses. The critical issue is how to in- 2. P ROBABILISTIC MODELING AND L APLACE
corporate the existing knowledge and new information into APPROXIMATION
the estimation. Many methodologies have been proposed to In this section, a generic posterior model for uncertain param-
handle reliability updating problems. Bayesian updating is eters is formulated using Bayes’ theorem to incorporate addi-
the most common approach to incorporate these additional tional data such as measurements. Uncertainties from model
data. By continuous Bayesian updating, all the variables parameters, measurement, and model independent variables
of interest are updated and the inference uncertainty can be are systematically included. To avoid MCMC simulations as
significantly reduced, provided the additional data are rele- in classical Bayesian applications, Laplace approximation is
vant to the problem and they are informative. (Hong, 1997) derived to obtain an analytical representation of the posterior
presented the idea of reliability updating using inspection distribution. The updated reliability and system responses can
data. (Papadimitriou, Beck, & Katafygiotis, 2001) reported readily be evaluated using this posterior approximation.
a reliability updating procedure using structural testing data.
(Graves, Hamada, Klamann, Koehler, & Martz, 2008) applied 2.1 Bayesian modeling for uncertain parameters
the Bayesian network for reliability updating. (Wang, Rabiei, Consider a generic parameterized model M(y; x) describing
Hurtado, Modarres, & Hoffman, 2009) used Bayesian reli- an observable event d, where x is an uncertain model param-
ability updating for aging airframe. A similar updating ap- eter vector and y is model independent variable. If the model
proach using Maximum relative Entropy principles has also is perfect, one obtains M(y; x) = d. In reality, such a perfect
been proposed in (Guan, Jha, & Liu, 2009). In those studies, model is rarely available due to uncertainties such as the sim-
MCMC simulations have been extensively used to draw sam- plification of the actual complex physical mechanisms, statis-
ples from posterior distributions. The Convergence Theorem tical error in obtaining the parameter x, and the measurement
ensures the resulting Markov chain converges to the target error in d. Using probability distributions to describe those
distribution (Gilks, Richardson, & Spiegelhalter, 1996) and uncertainties is a common practice.
it becomes almost a standard approach for Bayesian analysis Given the prior probability distribution of x, p(x|M), and
with complex models. For practical problems with compli- the known relationship (conditional probability distribution or
cated performance functions, simulations are time-consuming likelihood function) between d and x, p(d|x, M), the pos-
and efficient computations are critical for time constrained re- terior probability distribution p(x|d, M) is expressed using
liability evaluation and system response prognostics. Some Bayes’ theorem as
of the existing analytical methods includes variational meth-
ods (Ghahramani & Beal, 2000) and expectation maximiza- p(x|d, M) = p(x|M)p(d|x, M) Z1 ∝ p(x|M)p(d|x, M), (1)
tion methods (Moon, 1996). Those methods usually focus on R
the approximation of distributions and does not provide a sys- where Z = X p(x|M)p(d|x, M)dx is the normalizing con-
tematical procedure for inverse reliability problems. In struc- stant.
tural health management settings, simulation-based method The model M is assumed to be the only feasible model
may be infeasible because updating is frequently performed and M is omitted hereafter for simplicity. Let m be the model
upon the arrival of sensor data. All these application require prediction and e the error term (for example, the measurement
efficient and accurate computations. However, very few stud- error of d). The variable d reads
ies are available on the investigation of complete analytical d = m + e. (2)
updating and estimation procedure without using simulations.
The objective of the proposed study is to develop an effi- The probability distribution for m is represented by the func-
cient analytical method for system reliability and response up- tion p(m|x) = fM (m) and the probability distribution for e
dating without using simulations. Three computational com- is by the function p(e|x) = fE (e). The conditional proba-
ponents evolved in this approach are Bayesian updating, relia- bility distribution of p(d|x) can be obtained by marginalizing
bility estimation, and system response estimation given a reli- the joint probability distribution of p(d, m, e|x) as follows:
Z Z
ability or a confidence level. For Bayesian updating, Laplace
method is proposed to obtain an analytical representation of p(d|x) = p(m|x)p(e|x)p(d, m, e|x)dedm. (3)
the Bayesian posterior distribution and avoid MCMC simula- M E
tions. Once the analytical posterior distribution is obtained, Because d = m + e,
FORM method can be applied to update system reliability or
probability of failure. In addition, predictions of system re- p(d, z, e|x) = δ(d − m − e). (4)
sponse associated with a reliability or a confidence level can Substitute Eq. (4) into Eq. (3) to obtain
also be updated using inverse FORM method to avoid MC Z
simulations. p(d|x) = fM (m)fE (d − m)dm. (5)
The paper is organized as follows. First, a general Bayesian M
posterior model for uncertain variables is formulated. Rele-
vant information such as response measures and usage mon- Next, terms fM (m) and fE (e) need to be determined. Con-
itoring data are used for updating. Then an analytical ap- sider a general case where the model prediction m has a sta-
proximation to the posterior distribution is derived based on tistical noise component ǫ ∈ E with a distribution function
Laplace method. Next, FORM method is introduced to es- p(ǫ|x) = fE (ǫ) due to the modeling error m = M(y; x) + ǫ.
timate system reliability levels and a simplified algorithm Equation (2) is revised as
based on inverse FORM method is formulated to calculate d = M(y; x) + ǫ + e. (6)
system response given a reliability level or a confidence level.
Following that, numerical and application examples are pre- Marginalizing p(m|ǫ, θ) = δ(m − M(y; x) − ǫ) over ǫ to
sented to demonstrate the proposed method. The efficiency obtain
Z
and accuracy of the proposed method are compared with sim-
ulation results. fM (m) = p(ǫ|x)p(m|x, θ)dǫ = fE (m − M(y; x)). (7)
E

2
 Annual Conference of the Prognostics and Health Management Society, 2011

For the purpose of illustration, ǫ and e are assumed to be two The last term of Eq. (12) resembles remarkably a multivari-
independent Gaussian variables with standard deviations of ate Gaussian distribution
 with a mean vector of x0 and a co-
σǫ and σe , respectively. This assumption is usually made 2 −1
variance matrix Σ = −∇ lnp(x0 |d) . The normalizing
when no other information about the uncertain variables is constant is
available (Gregory, 2005). Equation (5) is the convolution of
two Gaussians and it can be further reduced to another Gaus- Z p
sian distribution as Z= elnp(x|d) dx ≈ p(x0 |d) (2π)n |Σ|, (13)
  X
1 −(d − M(y; x))2
p(d|x) = p exp . (8)  −1
2π(σǫ2 + σe2 ) 2(σǫ2 + σe2 ) where Σ = −∇2 lnp(x0 |d) , n is the dimension of the
variable x, and |Σ| is the determinant of Σ.
Substituting Eq. (3) into Eq. (1) yields the posterior prob-
ability distribution of the uncertain parameter x incorporat- The non-normalized Bayesian posterior distribution p(x|d)
ing the observable event d. The reliability or system state is now approximated as
variables can readily be updated with Eq. (1). For problems  1
1
with high dimensional parameters, the evaluation of Eq. (1) p(x|d) ≈ √ exp − 2 (x − x0 )T [Σ−1 ](x − x0 ) , (14)
(2π)n |Σ|
is rather difficult because the exact normalizing constant Z,
which is a multi-dimensional integral, is either analytically in-
tractable or computationally expensive. Instead of evaluating which is a multivariate Gaussian distribution with a mean vec-
this equation directly, the most common approach is to draw tor of x0 and a covariance matrix Σ. To compute x0 and Σ,
samples from it using MCMC simulations. For applications the first step is to find the local maxima of lnp(x|d) and eval-
where performance functions are computationally expensive uate the Hessian of lnp(x|d) at the local maxima. Numeri-
to evaluate, this approach is time-consuming and hence not cal root-finding algorithms can be used to find local maxima,
suitable for online updating and prognostics. To improve the such as Gauss-Newton algorithm (Dennis Jr, Gay, & Walsh,
overall computational efficiency, Laplace method is proposed 1981), Levenberg-Marquardt algorithm (More, 1978), trust-
to approximate the non-normalized Bayesian posterior distri- region dogleg algorithm (Powell, 1970), and so on. Laplace
bution of p(x|d). The derivation of Laplace approximation is method can yield accurate results given the target distribution
presented below. is approximately Gaussian distributed, which is quite com-
mon for practical problems (Gregory, 2005).
2.2 Laplace approximation for Bayesian posterior With the analytical representation of the posterior distribu-
distributions tion p(x|d), the updated reliability index can be calculated us-
ing the FORM method. In addition, updated system response
Consider the above non-normalized multivariate distribution predictions associated with a reliability index or a confidence
p(x|d) in Eq. (1) and its natural logarithm lnp(x|d). Expand- level can also be calculated using inverse FORM method. For
ing lnp(x|d) using Tylor series around an arbitrary point x∗ the sake of completeness, the basic concept of the FORM and
yields inverse FORM methods are introduced briefly.
lnp(x|d) =lnp(x∗ |d) + (x − x∗ )T ∇lnp(x∗ |d)+
1   3. FORM AND INVERSE FORM METHODS
(x − x∗ )T ∇2 lnp(x∗ |d) (x − x∗ )+ (9)
2!
O((x − x∗ )3 ), The time-invariant reliability analysis entails computation of
a multi-dimensional integral over the failure domain of the
where ∇lnp(x∗ |d) is the gradient of lnp(x|d) evaluated at x∗ , performance function.
∇2 lnp(x∗ |d) is the Hessian matrix evaluated at x∗ , and O(·)
are higher-order terms. Assume that the higher-order terms Z
are negligible in computation with respect to the other terms. PF ≡ P [g(x) < 0] = fX (x)dx, (15)
We obtain g(x)<0

lnp(x|d) ≈ lnp(x∗ |d) + (x − x∗ )T ∇lnp(x∗ |d) + where x ∈ Rn is a real-valued n-dimensional uncertain vari-
| {z }
(∗) able, g(x) is the performance function, such that g(x) < 0
(10) represents the failure domain, PF is the probability of fail-
1   ure, and fX (x) is the joint probability distribution of x.
(x − x∗ )T ∇2 lnp(x∗ |d) (x − x∗ ).
2! The surface g(x) = 0 is usually called limit-state surface.
In FORM/SORM methods, the uncertain variable is usually
The term (∗) is zero at local maxima (denoted as x0 ) of the transformed from the standard probability space to the stan-
distribution since ∇lnp(x0 |d) = 0. Therefore, if we choose dard Gaussian space, also referred to as reduced variable
to expand lnp(x|d) around x0 , we can eliminate term (∗) in space. Denote the transformed performance function as g(z),
Eq. (10) to obtain where z ∈ Rn is an n-dimensional standard Gaussian vari-
  able, also called reduced variable. The distance between the
lnp(x|d) ≈ lnp(x0 |d) + 12 (x − x0 )T ∇2 lnp(x0 |d) (x − x0 ). (11) closest point (most probable point (MPP), labeled as MPP in
Figure 1) on the limit-state surface g(z) = 0 to the origin in
Exponentiating lnp(x|d) of Eq. (11) yields the reduced variable space is the Hasofer-Lind reliability in-

1
 dex (Madsen et al., 1986), denoted as βHL in Figure 1. MPP
elnp(x|d) ≈ p(x0 |d)exp − (x − x0 )T [−∇2 lnp(x0 |d)](x − x0 ) . (12) is also known as the design point.
2

3
 Annual Conference of the Prognostics and Health Management Society, 2011

limit-state surface to linear and quadratic terms. For example, using FORM
g(z)=0 method yields the probability of failure as
PFFORM ≅ Φ(−βHL ), (18)
FORM where Φ is the standard Gaussian CDF. The precision of this
failure domain
approximation depends on the non-linearity of the limit-state
g(z)<0 surface. Experience shows that FORM method yields accu-
rate results for general engineering purposes (Cheng et al.,
2007). FORM is a widely used computational model in reli-
MPP ability index approach (RIA) for reliability-based design op-
timization (RBDO) since it finds the reliability index βHL .
βHL SORM The advantage of RIA is that the probability of failure is for-
wardly calculated for a given design. However, inverse reli-
ability analysis in performance measure approach (PMA) is
known to be more robust and informative than the reliabil-
ity analysis in RIA (Tu, Choi, & Park, 1999; Youn, Choi, &
Figure 1: Linear (FORM) and quadratic (SORM) approxima- Du, 2005). The idea of inverse reliability analysis in PMA is
tions of the performance function at MPP on the limit-state to investigate whether a given design satisfies the probabilis-
surface. tic constraint with a target reliability index βt . The inverse
reliability analysis can also be expressed as an optimization
Reliability analysis entails the computation of βHL and the problem such that
design point, which is a standard constrained optimization
problem defined as minimize: g(z) subject to ||z|| = βt . (19)
In inverse reliability analysis, among the different values of
minimize: ||z|| subject to g(z) = 0, (16) performance function g(z) taking on z that pass through the
where ||z|| denotes the distance between the point z and the βt curve in the reduced variable space, the one z ∗ that min-
origin in the reduced variable space. imizes the performance function is sought. Figure 2 illus-
The design point is generally not known a priori, hence an trates the inverse reliability analysis. The point z ∗ is also
iterative process is required to find the design point z ∗ in the called MPP and the corresponding minimal value of g(z ∗ )
reduced variable space such that βHL ≡ ||z ∗ || corresponds to is called probabilistic performance measure (PPM). Both re-
the shortest distance between z ∗ and the origin of the reduced liability analysis and inverse reliability analysis search for
variable space. Because reduced variables are based on the MPPs. The difference is that the former search for the MPP
mean and standard deviation of a normal distribution, the non- on the limit-state surface g(z) = 0 while the latter search for
normal variables must be transferred to its equivalent normal MPP on the βt curve. Based on the idea of inverse FORM
distribution. Rackwitz-Fiessler (Madsen, 1977) procedure is procedure proposed in (Der Kiureghian et al., 1994), an effi-
usually adopted for this purpose. The idea requires the cu- cient and simplified iterative formula in the reduced variable
mulative density function (CDF) and the probability density space is formulated as:
function (PDF) of the target distribution be equal to a nor- 
∇g(zk )

mal CDF and PDF at the value of variable x on the limit- zk+1 = zk + λ −βt − zk , (20)
state surface. This procedure finds the mean µeq and standard |∇g(zk )|
deviation σeq of the equivalent normal distribution and thus where ∇ is the gradient vector with respect to z and λ is the
the variable x can be reduced to a standard Gaussian variable step size at the kth iteration (a small constant is used in this
z = (x − µeq )/σeq . Several algorithms are available to lo- formula instead of an adaptive value). The initial value z0 is
cates the design point z ∗ , for example the Hasofer & Lind usually assigned to the distribution mean value. The iterative
- Rackwitz & Fiessler (HL-RF) algorithm (Hasofer & Lind, procedure proceeds until a convergence is achieved, i.e., when
1974; Rackwitz & Flessler, 1978). With an initial guess of z0
on the limit-state surface, the basic procedure computes the |zk+1 − zk |
≤ ε, (21)
new location for z ∗ iteratively according to |zk+1 |
1 where ε is a small quantity assigned by the user. For practical
zk+1 = [∇g(zk )zk − zk ] ∇g(zk )T . (17)
|∇g(zk )|
2 problems, ǫt = 10−4 to 10−3 usually yields satisfactory es-
timates (Cheng et al., 2007). Based on the iterative formula,
A reasonable guess can be fixing the first n−1 components of an algorithm locating MPP in inverse reliability problems is
z0 to its distribution means and solving for the last component given as Algorithm 1.
on the limit-state surface. The iterative procedure terminates
based on some criteria such as |βk+1 − βk | < ǫβ , where ǫβ is
a small control parameter assigned by users. Usually a value Algorithm 1 Inverse FORM algorithm solving MPP given a
target reliability index βt
of ǫβ = 10−4 to 10−3 yields accurate results for βHL and the
design point(Cheng et al., 2007). 1: Initiate z0 and λ, set k = 0
After finding the design point and βHL by solving Eq. (16) 2: repeat
using the iterative formula of Eq. (17), FORM or SORM 3: calculate zk+1 according to Eq. (20)
can approximate the probability of failure using a linear or 4: calculate d = |zk+1 −zk |
|zk+1 |
quadratic approximation of the performance function, respec- 5: k ←k+1
tively. Both of them are based on Taylor series expansion of 6: until d ≤ ǫt
the performance function around the design point truncated

4
 Annual Conference of the Prognostics and Health Management Society, 2011

For a given confidence level, the reliability indexes asso- 4.1 A numerical example with two uncertain variables
ciated with that level should be first calculated using inverse Consider a performance function f (x, y) = x + y describ-
Gaussian CDF then the MPPs associated with these indexes ing an observable event z = f (x, y) + ǫ, where x and y are
can be calculated using Algorithm 1. System responses are two uncertain variables and ǫ is an Gaussian error term with
readily evaluated with these MPPs. zero mean and a standard deviation of σǫ = 0.5. Variable x
is normally distributed with a mean of µx = 2 and a stan-
dard deviation of σx = 0.5 and variable y is also normally
distributed with a mean µy = 5 and a standard deviation
g(z)=4.0 σy = 1.5. Variables x and y are correlated with a corre-
g(z)=2.5
g(z)=-1.0 lation coefficient
h of ρxy =i −0.5. The covariance matrix is
2
σx σx σy ρxy
Σxy = σ σ ρ σ2
. f (x, y) > 9 is defined as fail-
x y xy y
ure event and the limit-state surface is f (x, y) − 9 = 0. The
MPP likelihood function can be expressed according to Eq. (8) as
(  2 )
1 1 z − f (x, y)
p(z|x) = √ exp − . (22)
β=1 FORM
2πσǫ 2 σǫ
β=2
Assume that the evidence of z = 8 is observed. The poste-
βt rior distribution that encodes this information is formulated
according to Eq. (1) as,
Figure 2: Inverse reliability analysis and MPP for target prob- 1
(
1

x − µx
T 
x − µx
)
ability of failure of βt and the linear approximation of the per- p(x|z) = p
2π |Σxy |
exp −
2 y − µy [Σxy ]−1 y − µy ×
formance function at MPP labeled as FORM. Values of g(z) (  2 ) (23)
are for illustration purposes only. 1 1 z − f (x)
√ exp − .
2πσǫ 2 σǫ

Both iterative formulae in Eq. (17) and Eq. (20) implicitly Based on the information given above, the prior estimate
assumes that the components of z are uncorrelated. For cor- of the probability of failure for event f (x, y) > 9 and the
related component variables in z, the correlated components prediction of system response z associated with a given relia-
need to be transformed into uncorrelated components via the bility or confidence level can be calculated using FORM and
orthogonal transformation of z ′ = L−1 (z T ), where L is the inverse FORM methods. After obtaining the additional data
lower triangular matrix obtained by Cholesky factorization of z = 8, those estimates can be updated using the proposed an-
the correlation matrix R such that LL′ = R, where L′ is the alytical procedure. The updating process firstly involves the
transpose of L. Laplace approximation for the posterior of Eq. (23). Then the
iterative formula of Eq. (17) is employed to find the design
The overall computational procedure according to the pro- point (x∗ , y ∗ ) and β, and the probability of failure PF can be
posed method is summarized as follows: estimated using FORM according to Φ(−β).
1. Formulate Bayesian posterior distributions according to To calculate the confidence bound (e.g., [lo, up] =
Eq. (8). [0.025, 0.975] bound) of z, reliability indexes associated with
the upper and lower limits are first calculated according to
2. Compute the posterior approximation according to Eq. βlo = Φ−1 (lo) and βup = Φ−1 (up). The iterative inverse
(14). FORM formula of Eq. (20) solves the required design point
3. Reliability or probability of failure estimation is calcu- for βlo and βup . Finally the confidence bound of z can be
lated using iterative formula of Eq. (17) and Eq. (18). computed using these two design points. To compare the effi-
ciency and the accuracy, MC and MCMC simulations serve
4. To estimate system responses associated with a reliabil- as benchmark solutions to this example. Table 1 presents
ity level or confidence level, calculate MPPs using Al- results for this example. The prior estimates for probability
gorithm 1 and then calculate system responses with the of failure (PoF) and interval prediction are calculated using
obtained MPPs. FORM and inverse FORM methods, respectively. For this
simple example, just a few function evaluations ensure ob-
Prior estimations are evaluated according to Steps 3 and 4 taining converged results. A crude Monte Carlo simulation
using prior distributions. To illustrate the proposed method, with 106 samples yields very close results. For the posterior
several examples are presented in the next section. estimate with Bayesian updating, the proposed analytical so-
lutions using Laplace, FORM, and inverse FORM (results are
4. E XAMPLES labeled as iFORM in all the tables hereafter) methods are very
close to the solution obtained using MCMC simulation with
A numerical example is given first to illustrate the overall a chain length of 106 . By comparing the number of function
procedure, and a practical fatigue crack propagation prob- evaluations between the analytical and MC or MCMC solu-
lem with experimental data and a beam example with finite tions, it is observed that the proposed analytical method can
element analysis data are demonstrated. Comparisons with reduce the computational cost by several orders of magnitude.
traditional simulation-based methods are made to investigate It would be significantly advantageous to use the proposed
the accuracy and computational efficiency of the proposed analytical procedure for time constrained or online prognosis
method. systems.

5
 Annual Conference of the Prognostics and Health Management Society, 2011

Table 1: Probability of Failure (PoF), confidence interval (CI) estimates, and the number of function evaluations (NFE) for
f (x, y) = x + y. Both x and y are normally distributed with means of 2 and 5 and standard deviations of 0.5 and 1.5,
respectively. The correlation coefficient between x and y is −0.5. The failure is defined as f (x, y) > 9.
Method PoF 95 CI NFE
prior FORM,iFORM 0.030181 4.2570∼9.7430 21
MC 0.030417 4.2493∼9.7455 106
Laplace,FORM,iFORM 0.0045455 6.6536∼8.9852 47
posterior
MCMC 0.0046910 6.6506∼8.9898 106

4.2 Reliability updating and response prognostics of a For this particular crack and geometry configuration, ∆K =
p
fatigue crack damaged system with experimental πa[sec(πa/w)]∆σ. Terms c and m are uncertainty model
data parameters that are usually obtained via statistical regression
In this section, a practical fatigue crack damage problem is analysis of experimental testing data. For convenience, lnc is
presented with experimental data. As a typical damage mode usually used instead of c. Given a specific number of loading
in many structures, the reliability of a system with possible cycles, solving the ordinary differential equation in Eq. (24)
fatigue cracks must be accurately quantified in order to avoid gives the corresponding crack size.
severe failure events. Because fatigue crack propagation is The first fifteen crack growth trajectories from Virkler’s
a time-dependent process, crack growth prognosis provides dataset identifies these two parameters using Maximum Like-
valuable information for system maintenance or unit replace- lihood Estimation as a joint Gaussian distribution of (lnc, m)
ment. Due to the stochastic nature of fatigue crack propaga- with a mean vector of µ0 = [−26.7084,  2.9687] and a co-
tion, fatigue crack growth is not a smooth and stable process. 0.5435 −0.0903
variance matrix of Σ0 = −0.0903 0.0150 .
Therefore additional information such as usage information
from health monitoring systems and crack size measures from 1
p0 (lnc, m) = p ×
inspections can be used to update various quantities of inter- 2π |Σ0 |
est. By performing continuous updating, uncertainties asso-   (25)
1 T
ciated with system reliability and crack size prognosis can be exp − [(lnc, m) − µ0 ] Σ−1
0 [(lnc, m) − µ 0 ]
2
reduced for decision-making. Because crack growth equa-
tions are usually in the forms of differential equations or fi- As we mentioned earlier in this section, another specimen
nite element models, simulation-based methods are relatively from the rest of the dataset is arbitrarily chosen to represent
more expensive in terms of computational cost. To demon- the target system. The reliability and crack growth progno-
strate the updating procedure with the proposed method and sis of this target system are of interest. The prior estimate
validate its effectiveness and efficiency, experimental data are of reliability and fatigue crack growth prognosis of the tar-
incorporated in this example. A portion of the experimental get system can then be estimated using this joint distribution
data is used to obtain the parameter distributions of the crack and the model in Eq. (24). Let M(N ; lnc, m) denotes the
growth equation and one from the rest of the dataset is arbi- model output (crack size) given a number of loading cycles
trarily chosen to represent the ”actual” target system. First N and parameters lnc and m. Three crack size measures
we estimate PoF and crack growth prognosis with the prior ai with corresponding numbers of loading cycles Ni at the
parameter distributions. Then we choose a few points from early stage of the target system are chosen to represent the
the ”acutal” target system to represent measurements from actual inspection data. They are (a1 , N1 ) = (10, 33062),
crack size inspections. These measures are used to perform (a2 , N2 ) = (11, 55101), and (a3 , N3 ) = (12, 75569). The
Bayesian updating with the analytical methods proposed in standard deviation of Gaussian likelihood is also estimated
previous sections. Both system reliability and crack growth as σa = 0.304mm. The failure event is defined as the crack
prognosis are updated. Results are compared with simulation- size exceeding 40.0mm given the number of loading cycles as
based methods in terms of accuracy and efficiency. 220,000. With these additional measurement data, the poste-
(Virkler, Hillberry, & Goel, 1979) reported a large set of rior distribution of (lnc, m) (with r response measures) reads
fatigue crack propagation data on aluminum alloy 2024-T3.
The dataset consists of fatigue crack propagation trajecto- pn (lnc, m) ∝ p0 (lnc, m)×
( 2 )
ries recorded from 68 center-through crack specimens, each r 
of which has the same geometry, loading, and material con- 1 X ai − M(Ni ; lnc, m (26)
exp −
figurations. Each specimen has a width of w = 154.2mm 2 i=1 σa
and a thickness of d = 2.54mm. The initial crack size is
a0 = 9.0mm. A constant cyclic loading with a stress range Following the proposed analytical procedure, we obtain
of ∆σ = 48.28MPa was applied. Without loss of generality, updated results of reliability and crack size prognosis. Ta-
the classical Paris’ equation (Paris & Erdogan, 1963) is cho- ble 2 shows the prior and posterior (updated) results of PoF
sen as the crack growth rate governing equation. Other crack and 95% interval predictions of crack size at 220,000 load-
growth equations can also be applied with the same proce- ing cycles. We can observe from this table that the simulation
dure. Paris’ equation describes the crack growth rate per one method requires 200,000 function evaluations while the ana-
constant cyclic load as lytical method requires less than 200 function evaluations to
produce similar results.
da
= c(∆K)m , (24) Figure 3 presents crack growth prognosis results obtained
dN by the proposed analytical method. MCMC simulation re-
where ∆K is the stress intensity range in one loading cycle. sults are displayed in the same figure for comparison. Several

6
 Annual Conference of the Prognostics and Health Management Society, 2011

Table 2: Prior and updated estimates of Probability of Failure (PoF), confidence interval (CI) for crack size, and the number
of function evaluations (NFE) for fatigue crack problem. The failure is defined as the crack size exceeds ac = 40mm at the
number of loading cycles Nc = 220, 000. 95% CI predictions are calculated at the number of loading cycles equal to Nc .
Measures Method PoF 95% CI NFE
0(prior) FORM,iFORM 0.0467 28.8290∼41.3095 60
MCMC 0.0498 28.9417∼41.4685 2 × 105
Laplace,FORM,iFORM 0.0225 28.3694∼39.8084 96
1
MCMC 0.0186 28.3563∼39.5466 2 × 105
2 Laplace,FORM,iFORM 0.0042 27.7926∼37.4207 105
MCMC 0.0039 27.6989∼37.3537 2 × 105
Laplace,FORM,iFORM 0.0002 27.2484∼34.9112 111
3
MCMC 0.0001 27.0817∼34.6913 2 × 105

50 50
95% CI:proposed 95% CI:proposed
median:proposed median:proposed
40 40
95% CI:MCMC 95% CI:MCMC
crack size (mm)

crack size (mm)

median:MCMC median:MCMC
30 actual 30 actual
measures
20 20

10 10

0 0
0 50000 100000 150000 200000 250000 0 50000 100000 150000 200000 250000
number of cycles number of cycles
(a) (b)

50 50
95% CI:proposed 95% CI:proposed
median:proposed median:proposed
40 40
95% CI:MCMC 95% CI:MCMC
crack size (mm)

crack size (mm)

median:MCMC median:MCMC
30 actual 30 actual
measures measures
20 20

10 10

0 0
0 50000 100000 150000 200000 250000 0 50000 100000 150000 200000 250000
number of cycles number of cycles
(c) (d)

Figure 3: Prior and posterior prognostics of fatigue crack growth trajectory using the proposed method (Laplace,iFORM) and
the traditional simulation-based methods (MCMC). Median and 95% interval predictions are presented: (a) prior estimation;
(b) updated with 1 measure; (c) updated with 2 measures; (d) updated with 3 measures.

aspects can be observed and interpreted: 1) The proposed an- These uncertainties cause the prior estimation deviates from
alytical Laplace and (inverse) FORM method yields almost the actual target system. 3) Inspection data, or crack size mea-
identical prognostic results to those obtained using traditional surement in this example, is critical to improve the accuracy
MCMC simulations, which can be confirmed by observing for time-dependent nonlinear system prognostics. With in-
Figure 3(a-d). 2) In Figure 3(a), the prior median and interval spection data, uncertainties can be greatly reduced. As shown
prediction of the crack growth is far from the actual target sys- in Figure 3(b-d), both the median and interval predictions for
tem because of various uncertainty associated with the crack crack growth trajectories become closer to the actual trajec-
propagation process such as the material uncertainty, model- tories as more measurements are integrated into the Bayesian
ing uncertainty, as well as measurement uncertainty. These updating process.
uncertainties are finally encoded into the model parameter
(lnc, m) in form of distributions through statistical regression.

7
 Annual Conference of the Prognostics and Health Management Society, 2011

4.3 A cantilever beam example Using the proposed method we obtain results shown in Ta-
A beam example is used to examine the proposed method ble (3). Simulation-based results are also listed in this table
through finite element analysis (FEA). Data from FEA pro- for comparison.
vide representative sensor output. By analyzing the sensor
output data, frequency information of the beam is extracted to
update the finite element model and also the reliability level. Table 3: Prior and posterior estimates of probability of
For the sake of illustration and simplification, we use a simple failure (PoF) in the beam example. Frequency data (first four
cantilever beam. More complex full-scale structural finite ele- natural frequency extracted from synthesized noisy data via
ment model analysis follows the same procedure as presented FFT) are used to perform Bayesian updating. Statistics of
here. α (mean,standard deviation(SD)) and computational cost in
A cantilever aluminum beam is divided into ten elements term of number of function evaluations (NFE) are shown.
using finite element modeling, as shown in Figure (4). The
beam is 1m long, 0.1m wide and 0.01m thick. The design Method PoF NFE α(mean,SD)
FORM 0.004435 11 1.0, 0.02
cross section area is A = 0.001m2 . Assume the cross section prior:
MC 0.004428 106 1.0, 0.02
area of the first segment of the beam (attached to the wall) Laplace,FORM 0.0182 57 0.9757, 0.0134
is modeled by A1 = αA due to manufacturing uncertainty, posterior:
MCMC 0.0164 106 0.9760, 0.0135
where term α is a Gaussian variable with a mean of 1 and
a standard deviation of 0.5. Because of usage (aging) and
material degradation, α may vary along time. Other segments Results of the proposed method are similar to those ob-
have deterministic cross section dimensions that are equal to tained using traditional simulation-based methods. However,
the design value of A. The material has a Young’s modulus the computational cost is much smaller. Finite element mod-
of E = 6.96 × 1010 Pa and a density of 2.73 × 103 kg/m3 . els in practical problems are usually more sophisticated than
this beam example, and simulation-based methods are not
feasible for such computationally extensive problems. The
proposed method provides an alternative to solving such prob-
lems and it yields accurate results under the condition that
0.01m
uncertain variables are approximately Gaussian-like.
fixed
end 1 2 3 4 5 6 7 8 9 10
In this section, three examples are presented to demonstrate
1m
0.1m and validate the proposed analytical method. Some important
the cross section area of fixed-end element (#1) is uncertain aspects of the proposed method are closely revealed, includ-
ing the computational benefits in terms of efficiency and accu-
Figure 4: The cantilever beam finite element model. The racy. Appropriate conditions to assure these benefits are also
cross section area of the first element (attached to the wall) analyzed.
is uncertain due to manufacture and usage and is modeled by
A1 = αA, where A = 0.001m2 is design cross section area 5. C ONCLUSIONS
and α ∼ Norm(1, 0.022 ). In this paper, an efficient analytical Bayesian method for re-
liability and system response updating is developed. The
The failure event is defined as the first natural frequency method is capable of incorporating additional information
is less than 8Hz due to the degradation of the stiffness of the such as inspection data to reduce uncertainties and im-
beam. The sensor data are synthesized by setting α = 0.95 prove the estimation accuracy. One major difference be-
and solving the dynamical equation of the beam under a free tween the proposed work and the traditional approach is that
vibration. After adding 5 percent of Gaussian white noise, the proposed method performs all the calculations includ-
the first four mode frequency data are extracted from the sen- ing Bayesian updating without using MC or MCMC simu-
sor data using Fast Fourier Transformation (FFT). They are lations. A numerical example, a practical fatigue crack prop-
(f1 , f2 , f3 , f4 ) = (8.03, 50.5, 142, 280)Hz. agation problem with experimental data, and a finite element
Based on the above information, the Bayesian posterior beam problem with FEA data are presented to demonstrate
for uncertain variable α given the frequency information ex- the proposed method. Comparisons are made with traditional
tracted from the sensor data is simulation-based methods to investigate the accuracy and ef-
  ficiency. Based on the current study, several conclusions are
1 (α − 1)2
p(α) ∝ exp − 2
drawn.
2 0.5
 
(27) 1. The proposed method provides an efficient analytical
F X N 
 X
× exp −
 2

 2 
{ω}i −(2πfi ) M(α) + K(α) {φi } j ,
computational procedure for computing and updating system

j=1 i=1
 reliability responses. No MC or MCMC simulation is re-
quired therefore it provides an feasible and practical solu-
where N is the number of measured mode and F is the num- tion to time constrained or online prognostics. The method
ber of measured mode shape coordinates. Term {ω}i is the is also beneficial for structural health monitoring problems
ith weighting factor for ith frequency component in the like- where Bayesian updating and system response predictions are
lihood function. For the purpose of illustration, {ω} is con- frequently performed upon the arrival of sensor data.
figured such that each frequency component has a coefficient 2. The proposed method is capable of incorporating addi-
of variation of 0.1. Terms M(α) and K(α) are the mass and tional information such as the inspection data and usage data
stiffness matrices, respectively. Because α is a variable, ac- from health monitoring system by way of Bayesian updat-
tual values for M(α) and K(α) depends on each realization ing. This property is beneficial for highly stochastic time-
of α. Term {φ}i is the ith mode shape. For the current data, dependent nonlinear system where prior estimates for relia-
N = 4 and F = 20. bility and system response may become unreliable along with

8
 Annual Conference of the Prognostics and Health Management Society, 2011

system developing. By continuous Bayesian updating, esti- lower-level data in reliability assessments. Reliability
mation uncertainties can be reduced. Engineering & System Safety, 93(8), 1273-1279.
3. The proposed method yields almost identical results Gregory, P. (2005). Bayesian logical data analysis for the
to those produced by traditional simulation-based methods physical sciences: a comparative approach with Math-
given that uncertain variables are approximately Gaussian ematica support. Cambridge Univ Pr.
distributed. This is true for most of the engineering problems Guan, X., Jha, R., & Liu, Y. (2009). Probabilistic fa-
where the uncertain parameters are normal or log-normal tigue damage prognosis using maximum entropy ap-
variables (which can be transformed and truncated into nor- proach. Journal of Intelligent Manufacturing, 1-9.
mal variables). When these conditions are not assured, the re- (10.1007/s10845-009-0341-3)
sults need careful interpretations. The efficiency and accuracy Hasofer, A., & Lind, N. (1974). Exact and invariant second-
of the proposed method is demonstrated and verified using moment code format. Journal of the Engineering Me-
three examples. The proposed method provides an alternative chanics Division, 100(1), 111–121.
for time-constrained prognostics problems. If the problem Hong, H. (1997). Reliability analysis with nondestructive
involves too many random variables, traditional simulation- inspection. Structural Safety, 19(4), 383–395.
based method may be more appropriate. Systematical com- Kalos, M., & Whitlock, P. (2008). Monte carlo methods.
parisons of the method with other approaches such as varia- Wiley-VCH.
tional method will be conducted in the future. Lee, I., Choi, K., Du, L., & Gorsich, D. (2008). Inverse anal-
ysis method using MPP-based dimension reduction for
ACKNOWLEDGMENTS reliability-based design optimization of nonlinear and
multi-dimensional systems. Computer Methods in Ap-
The research reported in this paper was supported by plied Mechanics and Engineering, 198(1), 14–27.
the NASA ARMD/AvSP IVHM project under NRA Li, H., & Foschi, R. (1998). An inverse reliability method
NNX09AY54A. The support is gratefully acknowledged. and its application. Structural Safety, 20(3), 257–270.
Liu, J. (1996). Metropolized independent sampling with com-
R EFERENCES parisons to rejection sampling and importance sam-
pling. Statistics and Computing, 6(2), 113–119.
Brauer, D., & Brauer, G. (2009). Reliability-centered main- Madsen, H. (1977). Some experience with the Rackwitz-
tenance. Reliability, IEEE Transactions on, 36(1), 17– Fiessler algorithm for the calculation of structural reli-
24. ability under combined loading. DIALOG-77, Danish
Bucher, U., et al. (1990). A fast and efficient response surface Engineering Academy, Lyngby, Denmark, 73–98.
approach for structural reliability problems. Structural Madsen, H., Krenk, S., & Lind, N. (1986). Methods of struc-
Safety, 7(1), 57–66. tural safety. Prentice-Hall, Inc., Englewood Cliffs, NJ.
Cai, G., & Elishakoff, I. (1994). Refined second-order relia- Melchers, R. (1999). Structural reliability analysis and pre-
bility analysis. Structural Safety, 14(4), 267–276. diction. John Wiley & Son Ltd.
Cheng, J., Zhang, J., Cai, C., & Xiao, R. (2007). A new Moon, T. (1996). The expectation-maximization algorithm.
approach for solving inverse reliability problems with Signal Processing Magazine, IEEE, 13(6), 47–60.
implicit response functions. Engineering structures, More, J. (1978). The Levenberg-Marquardt algorithm: imple-
29(1), 71–79. mentation and theory. Numerical analysis, 105–116.
Dennis Jr, J., Gay, D., & Walsh, R. (1981). An adaptive Papadimitriou, C., Beck, J., & Katafygiotis, L. (2001). Up-
nonlinear least-squares algorithm. ACM Transactions dating robust reliability using structural test data. Prob-
on Mathematical Software (TOMS), 7(3), 348–368. abilistic Engineering Mechanics, 16(2), 103–113.
Der Kiureghian, A., & Dakessian, T. (1998). Multiple design Paris, P., & Erdogan, F. (1963). A critical analysis of crack
points in first and second-order reliability. Structural propagation laws. Journal of Basic Engineering, 85(4),
Safety, 20(1), 37–49. 528–534.
Der Kiureghian, A., Yan, Z., & Chun-Ching, L. (1994). In- Powell, M. (1970). A FORTRAN subroutine for solving sys-
verse reliability problem. Journal of engineering me- tems of nonlinear algebraic equations. Numerical meth-
chanics, 120(5), 1154–1159. ods for nonlinear algebraic equations, 115.
Ditlevsen, O., & Madsen, H. (1996). Structural reliability Rackwitz, R. (2001). Reliability analysis–a review and some
methods (Vol. 315). Citeseer. perspectives. Structural Safety, 23(4), 365–395.
Du, X., Sudjianto, A., & Chen, W. (2004). An integrated Rackwitz, R., & Flessler, B. (1978). Structural reliability
framework for optimization under uncertainty using in- under combined random load sequences. Computers &
verse reliability strategy. Journal of Mechanical De- Structures, 9(5), 489–494.
sign, 126, 562. Rebba, R., & Mahadevan, S. (2008). Computational methods
Gelman, A., & Meng, X. (1998). Simulating normalizing for model reliability assessment. Reliability Engineer-
constants: From importance sampling to bridge sam- ing & System Safety, 93(8), 1197–1207.
pling to path sampling. Statistical Science, 13(2), 163– Saranyasoontorn, K., & Manuel, L. (2004). Efficient models
185. for wind turbine extreme loads using inverse reliabil-
Ghahramani, Z., & Beal, M. (2000). Variational inference ity. Journal of Wind Engineering and Industrial Aero-
for Bayesian mixtures of factor analysers. Advances in dynamics, 92(10), 789–804.
neural information processing systems, 12, 449–455. Tu, J., Choi, K., & Park, Y. (1999). A new study on reliability-
Gilks, W., Richardson, S., & Spiegelhalter, D. (1996). based design optimization. Journal of Mechanical De-
Markov chain Monte Carlo in practice. Chapman & sign, 121, 557.
Hall/CRC. Virkler, D., Hillberry, B., & Goel, P. (1979). The Statisti-
Graves, T., Hamada, M., Klamann, R., Koehler, A., & Martz, cal Nature of Fatigue Crack Propagation. Journal of
H. (2008). Using simultaneous higher-level and partial Engineering Materials and Technology, 101, 148.

9
 Annual Conference of the Prognostics and Health Management Society, 2011

Wang, X., Rabiei, M., Hurtado, J., Modarres, M., & Hoff-
man, P. (2009). A probabilistic-based airframe integrity
management model. Reliability Engineering & System
Safety, 94(5), 932–941.
Xiang, Y., & Liu, Y. (2011). Application of inverse first-order
reliability method for probabilistic fatigue life predic-
tion. Probabilistic Engineering Mechanics, 26(2), 148-
156.
Youn, B., Choi, K., & Du, L. (2005). Enriched perfor-
mance measure approach for reliability-based design
optimization. AIAA journal, 43(4), 874–884.
Zhang, R., & Mahadevan, S. (2001). Integration of compu-
tation and testing for reliability estimation. Reliability
Engineering & System Safety, 74(1), 13–21.
Zhao, Y., & Ono, T. (1999). A general procedure
for first/second-order reliability method (form/sorm).
Structural Safety, 21(2), 95–112.
Xuefei Guan is a graduate research assistant in department of
mechanical engineering at Clarkson University. He received
his B.S. degree in Reliability Engineering and M.S. degree in
Aeronautical Engineering from Beihang University in China
in 2005 and 2008, respectively. His research interests are
probabilistic analysis, Bayesian and entropy-based methods
and applications, Markov Chain Monte Carlo simulation, and
reliability analysis.

Jingjing He is a graduate research assistant in department

of civil engineering at Clarkson University. Her research
interests are fatigue analysis, structural dynamics, diagnosis
and prognosis. She received her B.S. degreee in Reliability
Engineering and M.S. degree in Aerospace System Engi-
neering from Beihang University in China in 2005 and 2008,
respectively.

Ratneshwar Jha is an Associate Professor in the Department

of Mechanical and Aeronautical Engineering at Clarkson
University. His research interests include structural health
monitoring, modeling of composite and smart structures,
adaptive control of structural vibrations, intelligent flight
controls, and multidisciplinary design optimization. Dr. Jha
is an Associate Fellow of AIAA and a member of ASME
and ASEE. Dr. Jha earned PhD in Mechanical Engineering
from Arizona State University in 1999, MS in Aerospace
Engineering from Georgia Institute of Technology in 1983,
and B. Tech in Aeronautical Engineering from Indian Insti-
tute of Technology in 1981. Dr. Jha worked in the aerospace
industry from 1983 to 1995 where he led a team of engineers
working on conceptual and preliminary designs of combat
aircraft.
Yongming Liu is an assistant Professor in the department of
civil and environmental engineering. His research interests
include fatigue and fracture analysis of metals and composite
materials, probabilistic methods, computational mechanics,
and risk management. He completed his PhD at Vanderbilt
University, and obtained his Bachelors’ and Masters’ degrees
from Tongji University in China. Dr. Liu is a member of
ASCE and AIAA and serves on several technical committees
on probabilistic methods and advanced materials.

10