Performance of reliability

methods in geomechanical
applications

Pablo A. Vásconez
Master of Science Thesis

Geo-Engineering Section
 mscconfidential

Performance of reliability methods in

geomechanical applications

Master of Science Thesis

For the degree of Master of Science in Civil Engineering, Track in

Geo-Engineering at Delft University of Technology

Pablo A. Vásconez

August 23, 2013

Faculty of Civil Engineering and Geosiences (CiTG) · Delft University of Technology

This document is classified as Restricted. Access is allowed to Shell personnel, designated Associate
Companies and Contractors working on Shell projects who have signed a confidentiality agreement
with a Shell Group Company. ’Shell Personnel’ includes all staff with a personal contract with a Shell
Group Company. Issuance of this document is restricted to staff employed by a Shell Group Company.
Neither the whole nor any part of this document may be disclosed to Non-Shell Personnel without the
prior written consent of the copyright owners.

Copyright c Delft University of Technology and Shell Global Solutions International, B.V. 2013.
All rights reserved.

Geo-Engineering Section Shell Global Solutions International B.V.

Rijswijk, The Netherlands
 Delft University of Technology
Geo-Engineering Section

The undersigned hereby certify that they have read and recommend to the Faculty of
Civil Engineering and Geosiences (CiTG) for acceptance a thesis entitled
Performance of reliability methods in geomechanical applications
by
Pablo A. Vásconez
in partial fulfillment of the requirements for the degree of
Master of Science Civil Engineering, Track in Geo-Engineering

Dated: August 23, 2013

Supervisor(s):
Prof.dr. Michael A. Hicks

Dr.ir. Pieter van Gelder

Reader(s):
Dr.ir. Peter A. J. van den Bogert

Dr. Phillip J. Vardon

Dr. Jonathan D. Nuttall

 Abstract

Reliability-based design is an approach currently gaining more popularity for geotechnical engineer-
ing. Neverthless, its implementation poses several challenges, one being the additional time (usually
significant) in order to perform it. An important portion of this additional time comes from the need
to perform many scenarios to compute the probability of failure: a Monte Carlo analysis (the most
widespread procedure) can easily take more than tens of thousands of realisations to converge. More-
over, there is a tendency to use complex models in practice (e.g. finite element analysis), producing
more often prohibitive computation time.
This thesis attempts to find alternative reliability methods to Monte Carlo capable of reducing the
computation time for geomechanical problems in the oil and gas industry. The methodology is to eval-
uate several different methods (named workflows) for specific examples. The terms of the evaluations
are in accuracy (defined as the closeness to the exact result obtained by a Monte Carlo analysis) and
efficiency (measured in the number of geomechanical model realisations).
Among the proposed workflows (i.e. methods), there is one which is capable of computing accurately
and efficiently the probability of failure for all the analysis made, and others which have a potential to
do so, if some improvements are performed. The worklow currently capable is First-Order Reliability
Method (FORM), requiring around 30 to 100 realisations for cases with 6 and 11 random variables,
respectively. The workflows with potential to be accurate and efficient are FORM and Monte Carlo
applied on a response surface that updates (instead of directly using the actual complex model),
which required between 10 to 30 realisations. Both represent a strong reduction versus a Monte Carlo
analysis, which at took approximately 20,000 to 60,000 realisations in this study.
However, for the latter workflows the accuracy on the probability of failure depends on the degree
of non-linearity of the response and the target probability of failure. When the non-linearity is high
and/or the target low, the updates of the response surface stops before it can accurately represent
the actual response in the most probable failure region, subsequently obtaining low accuracies on the
probability of failure. A connection between the updates and the prediction of the actual response
surface is thought to be able to improve this workflows in order to achieve good accuracies for all the
examples analysed in the study.
Therefore, it is feasible with the methods proposed to compute accurately and efficiently the reliability
in geotechnical engineering, but this conclusion is restricted to for the specific examples evaluated in
this study. If a new type of problem is desired to be addressed, it is recommended to validate them
first (by comparing their results to Monte Carlo analysis), before using them confidently in practice.

Master of Science Thesis RESTRICTED Pablo A. Vásconez

 Table of Contents

Acknowledgements x

1 Introduction 1
1-1 Motivation of the Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1-2 Objectives of the Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1-3 Methodology of the Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2 A review on the reliability methods 6

2-1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2-2 Definition of Reliability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2-3 Numerical Integration to determine reliability . . . . . . . . . . . . . . . . . . . . . . . . 9
2-4 Simulation Methods for determining the reliability . . . . . . . . . . . . . . . . . . . . . . 11
2-4-1 Crude Monte Carlo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2-4-2 Importance sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2-4-3 Directional sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2-5 Methods approximating the performance function . . . . . . . . . . . . . . . . . . . . . . 20
2-5-1 First-Order Reliability Method (FORM) . . . . . . . . . . . . . . . . . . . . . . . 21
2-6 Summary of the chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

3 Response Surfaces on the Reliability Computation 30

3-1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3-2 Definition of Response Surface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3-3 Computation of a Response Surface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3-3-1 Selection of the center of sampling and sampling technique . . . . . . . . . . . . . 33
3-3-2 Computation of the coefficients . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3-4 Response surfaces in the computation of reliability . . . . . . . . . . . . . . . . . . . . . . 38
3-5 Advantages and Limitations of the Response Surface . . . . . . . . . . . . . . . . . . . . 42
3-6 Summary of the chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

Pablo A. Vásconez RESTRICTED Master of Science Thesis

Table of Contents iii

4 Workflows under Evaluation 44

4-1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4-2 Definition of the specific performance functions for this study . . . . . . . . . . . . . . . . 44
4-3 General description of the workflows under evaluation . . . . . . . . . . . . . . . . . . . . 49
4-4 Graphical description of workflows using a Response Surface with updates Rit . . . . . . . 52
4-5 Response Surfaces used in the workflows . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
4-6 Reliability Methods used in the workflows . . . . . . . . . . . . . . . . . . . . . . . . . . 56
4-7 Summary of the chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

5 Evaluation of Workflows for

Quasi-Linear Responses 62
5-1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
5-2 Definition of efficient workflows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
5-3 Methodology for the evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
5-4 Evaluation for Subsidence with Homogeneous Subsurface . . . . . . . . . . . . . . . . . . 65
5-4-1 Description of the model EWG and limit state values . . . . . . . . . . . . . . . . 65
5-4-2 Sensitivities with the Tornado Plot . . . . . . . . . . . . . . . . . . . . . . . . . . 66
5-4-3 Results of the evaluation for Maximum Subsidence . . . . . . . . . . . . . . . . . 67
5-4-4 Results of the selected workflows for Average Subsidence . . . . . . . . . . . . . . 83
5-4-5 Results of selected workflows for Failure Area Subsidence . . . . . . . . . . . . . . 85
5-5 Evaluation for Shear with Homogeneous Subsurface . . . . . . . . . . . . . . . . . . . . . 87
5-5-1 Description of the model EWG02 . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
5-5-2 Sensitivities with the Tornado Plot . . . . . . . . . . . . . . . . . . . . . . . . . . 87
5-5-3 Results and Discussion for the Selected Workflows . . . . . . . . . . . . . . . . . 88
5-6 Summary of evaluations for quasi-linear examples . . . . . . . . . . . . . . . . . . . . . . 91
5-7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

6 Evaluation of Workflows for Non-Linear Responses 95

6-1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
6-2 Evaluation for Subsidence with Homogeneous Subsurface . . . . . . . . . . . . . . . . . . 96
6-2-1 Description of the example EWQ01 . . . . . . . . . . . . . . . . . . . . . . . . . 96
6-2-2 Sensitivities with Tornado Plot . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
6-2-3 Results and Discussion of selected workflows . . . . . . . . . . . . . . . . . . . . . 98
6-3 Evaluation for Subsidence with Layered Subsurface . . . . . . . . . . . . . . . . . . . . . 101
6-3-1 Description of the model EWQ02 . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
6-3-2 Sensitivities with Tornado Plot . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
6-3-3 Results and Discussion for selected workflows . . . . . . . . . . . . . . . . . . . . 102
6-4 Summary of evaluations for examples with non-linear response . . . . . . . . . . . . . . . 106
6-5 Influence of the target probability of failure on the accuracy . . . . . . . . . . . . . . . . . 109
6-6 Limitation of the workflows Rit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
6-7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

7 Conclusions and Recommendations 119

7-1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
7-2 Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
7-3 Future research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

Master of Science Thesis RESTRICTED Pablo A. Vásconez

iv Table of Contents

A Geomechanical Models 126

B Additional Results 131

B-1 Example: Average SCU EWG02 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
B-2 Example: Average Subsidence EWQ01 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
B-3 Example: Average Subsidence EWQ02 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136

References 138

Glossary 141
List of Acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

Pablo A. Vásconez RESTRICTED Master of Science Thesis

 List of Figures

1-1 Probabilities of failure occurred in practice classified by type of structure (Baecher, 1982) . 3

2-1 Safe and Failure Regions (Du, 2005) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2-2 Grid in the x-space for Numerical Integration method (∆X1 is the distance between the
vertical lines and ∆X2 , between the horizontal ones . . . . . . . . . . . . . . . . . . . . . 10
2-3 Crude Monte Carlo simulation in the x-space . . . . . . . . . . . . . . . . . . . . . . . . . 12
2-4 Two variations of importance sampling: (a) Increase of variance of the density function, (b)
Shifting towards failure domain (Waarts, 2000) . . . . . . . . . . . . . . . . . . . . . . . 16
2-5 Schematic representation of the Directional Sampling. Dashed lines are the random direc-
tions α . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2-6 Schematic representation of the Directional Sampling. Dashed line is a random direction
α(i) and the numbers represent realisations for the linear search method in chronological
order . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2-7 FORM and SORM approximation of the performance function g(X) . . . . . . . . . . . 21
2-8 Most probable failure point u∗ and reliability index β. Colours are PDF contours. . . . . . 23
∗
2-9 Vectors u and α lie in the same line with opposite signs . . . . . . . . . . . . . . . . . . 24
2-10 Step 1 of the search of the most probable point for two random variables. Colours are real
contours of the performance function. Star represents the mean-values point. . . . . . . . 25
2-11 Step 4 of the search of the most probable point for two random variables. Colours are linear
contours of the performance function considered by FORM. Star represents the mean-values
point. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2-12 All iterations for the search of the most probable point for two random variables, starting
in a different point than the mean-values point. Star represents the mean-values point. . . 27
∗
2-13 Comparison between β and the minimum distance min|x | from the mean-values point to
the limit state function. Colours are PDF contours. . . . . . . . . . . . . . . . . . . . . . 28

3-1 First-order response surface on function peaks from MATLAB . . . . . . . . . . . . . . . 32

3-2 Second-order response surface on function peaks from MATLAB . . . . . . . . . . . . . . 32
3-3 Common choices for center of sampling (marked with a cross) . . . . . . . . . . . . . . . 34
3-4 Different experimental designs (or sampling techniques). Adapted from (Montgomery, 2008) 36

Master of Science Thesis RESTRICTED Pablo A. Vásconez

vi List of Figures

3-5 Schematic representation of the iteration to center the response surface near the design
point (Bucher and Bourgund, 1990) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3-6 Comparison between the experimental design and the vector projected sampling (Kim and
Na, 1997) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
∗
3-7 Derivation by of the empirical distribution for U using the jackknife technique (Gayton
et al., 2003) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3-8 Resampling area derived considering U ∗ as a Student type variable with N − 1 degrees of
freedom (Gayton et al., 2003) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

4-1 Failure Indicators for Subsidence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

4-2 Failure Indicators for Formation Shear Capacity Utilization (SCU) . . . . . . . . . . . . . 48
4-3 Diagrams representing the proposed workflows in the study: Res, Rit and Mod. Colors
indicate inputs (yellow), process (red) and outputs (green). GM = geomechanical model . 51
4-4 Presentation of the threshold range in a fictional case . . . . . . . . . . . . . . . . . . . . 52
4-5 Sampling falling outside the threshold range. Only the response surface is used. . . . . . . 53
4-6 Updating procedure of response surface and threshold when a samples falls inside the thresh-
old range. Legend from Figure 4-4 is applicable . . . . . . . . . . . . . . . . . . . . . . . 54
4-7 Next update of the response surface and threshold . . . . . . . . . . . . . . . . . . . . . 54
4-8 Sampling distance is d = 1.28σi , represented by selection of percentiles P10 and P90 and
low and high values of the random variables . . . . . . . . . . . . . . . . . . . . . . . . . 55
4-9 Sampling for the different response surfaces under evaluation, for a case with 2 random
variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
(i)
4-10 Discretisation of the direction α in 20 intervals if failure is not found during the linear
search method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

5-1 Sketch of the model EWG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

5-2 Tornado plot for maximum subsidence using the response surface FOD . . . . . . . . . . . 66
5-3 Performance of workflow Rit{Monte Carlo} for Maximum Subsidence of example EWG. (a)
Probability of Failure pf (b) Accuracy of the probability of failure Apˆf and (c) Number
of geomechanical model realizations Nmod for different initial threshold values th;ini and
response surfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
5-4 Resulting sampling and prediction of the response surface for different initial thresholds . . 71
5-5 Maximum subsidence for a specific direction . . . . . . . . . . . . . . . . . . . . . . . . . 73
5-6 Performance of workflow Rit{Importance Sampling} for Maximum Subsidence of example
EWG. (a) Probability of Failure pf (b) Accuracy of the probability of failure Apˆf and (c)
Number of geomechanical model realizations Nmod for different initial threshold values th;ini
and response surfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
5-7 Comparison between the actual response and response surface, including the importance
sampling cumulative density . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
5-8 Performance of workflow Rit{Directional Sampling} for Maximum Subsidence of example
EWG. (a) Probability of Failure pf (b) Accuracy of the probability of failure Apˆf and (c)
Number of geomechanical model realizations Nmod for different initial threshold values th;ini
and response surfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
(i)
5-9 Comparison between the actual response and response surface at the direction α =
(D, R, h, ∆p, ν, cm ) = (0.3178, 0.3445, 0.4846, 0.6702, 0.3103, 0.0059), for a simulation with
Directional Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
5-10 Performance of workflow Rit{FORM} for Maximum Subsidence of example EWG. (a) Prob-
ability of Failure pf (b) Accuracy of the probability of failure Apˆf and (c) Number of ge-
omechanical model realizations Nmod for different initial threshold values th;ini and response
surfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

Pablo A. Vásconez RESTRICTED Master of Science Thesis

List of Figures vii

5-11 Probability of failure as a function of the number of samples for a Monte Carlo (MC) and
Importance Sampling (IS) simulation. The variance factor used for Importance sampling is
1.5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
5-12 Accuracy of the probability of failure Apˆf for different initial threshold values th;ini , response
surfaces and group of workflows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
5-13 Typical shape of the response for Failure Area . . . . . . . . . . . . . . . . . . . . . . . . 86
5-14 Sketch of the model EWG02 with properties of the base case . . . . . . . . . . . . . . . . 87
5-15 Tornado plot for Maximum SCU using the sampling for response surface FOD . . . . . . . 88
5-16 Performance of workflow Rit{Monte Carlo} for Maximum SCU of example EWG02. (a)
Probability of Failure pf (b) Accuracy of the probability of failure Apˆf . . . . . . . . . . 89
5-17 (a) Probability of Failure pf (b) Accuracy of the probability of failure Apˆf . Workflow:
Rit{FORM} . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
5-18 Peak and residual accuracies on the probability of failure obtained in the evaluation of
quasi-linear examples (EWG and EWG02). Failure indicators of maximum (Max), average
(Ave) and Failure Area (Fai) are presented. Workflows Rit for different response surfaces
are represented by bars and workflow Mod{FORM} by dashed lines. . . . . . . . . . . . . 92
5-19 Number of geomechanical model realisations Nmod for the peak and residual accuracies
on the probability of failure obtained in the evaluation of quasi-linear examples (EWG and
EWG02). Failure indicators of maximum (Max) and average (Ave) are presented. Workflows
Rit for different response surfaces are represented by bars and workflow Mod{FORM} by
dashed lines. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

6-1 Sketch of the example EWQ01 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

6-2 Tornado plot for EWQ01 using sampling from response surface FOD . . . . . . . . . . . . 98
6-3 (a) Probability of Failure pf (b) Accuracy of the probability of failure Apˆf . Workflow:
Rit{Monte Carlo} . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
6-4 (a) Probability of Failure pf (b) Accuracy of the probability of failure Apˆf . Workflow:
Rit{FORM} . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
6-5 Sketch of the example EWQ02 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
6-6 Tornado plot for EWQ02 using sampling from response surface FOD . . . . . . . . . . . . 103
6-7 (a) Probability of Failure pf (b) Accuracy of the probability of failure Apˆf . Workflow:
Rit{Monte Carlo} . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
6-8 (a) Probability of Failure pf (b) Accuracy of the probability of failure Apˆf . Workflow:
Rit{FORM} . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
6-9 Peak and residual accuracies on the probability of failure obtained in the evaluation of
examples (EWQ01 and EWQ02) and for failure indicators of maximum (Max.Sub) and
average subsidence (Ave.Sub). Workflows Rit for different response surfaces are represented
by bars and workflow Mod{FORM} by dashed lines. . . . . . . . . . . . . . . . . . . . . . 107
6-10 Number of geomechanical model realisations Nmod for the peak and residual accuracies on
the probability of failure for the evaluation of examples with non-linear responses (EWQ01
and EWQ02). Workflows Rit for different response surfaces are represented by bars and
workflow Mod{FORM} by dashed lines. . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
6-11 Peak and residual accuracies on the probability of failure obtained in the evaluation of exam-
ples EWQ for two different targets of probability of failure pf = 0.002 and 0.12. Workflows
Rit for different response surfaces are represented by bars and workflow Mod{FORM} by
dashed lines. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
6-12 Quasi-linear vs Non-linear for Rit{Monte Carlo} peakAcc . . . . . . . . . . . . . . . . . . 114
6-13 Quasi-linear vs Non-linear for Rit{Monte Carlo} peakNmod . . . . . . . . . . . . . . . . . 114

Master of Science Thesis RESTRICTED Pablo A. Vásconez

viii List of Figures

6-14 Quasi-linear vs Non-linear for Rit{Monte Carlo} resAcc . . . . . . . . . . . . . . . . . . . 115

6-15 Quasi-linear vs Non-linear for Rit{FORM} peakAcc . . . . . . . . . . . . . . . . . . . . . 115
6-16 Quasi-linear vs Non-linear for Rit{FORM} peakNmod . . . . . . . . . . . . . . . . . . . . 116
6-17 Quasi-linear vs Non-linear for Rit{FORM} resAcc . . . . . . . . . . . . . . . . . . . . . . 116
6-18 Quasi-linear vs Non-linear for Mod{FORM} . . . . . . . . . . . . . . . . . . . . . . . . . 117
6-19 Quasi-linear vs Non-linear for Mod{FORM} . . . . . . . . . . . . . . . . . . . . . . . . . 117

A-1 Sketch of the Geomechanical model Geertsma . . . . . . . . . . . . . . . . . . . . . . . . 127

A-2 Subsidence profile with distance, starting from the center of the reservoir . . . . . . . . . 128
A-3 Sketch of the Geomechanical Model Quickblocks . . . . . . . . . . . . . . . . . . . . . . 130
A-4 Subsidence profile with distance from the center of the reservoir, using the same data but
different geomechanical models: Geertsma and Quickblocks . . . . . . . . . . . . . . . . . 130

B-1 Performance of workflow Rit{Monte Carlo} for Average SCU of example EWG02. (a)
Probability of Failure pf (b) Accuracy of the probability of failure Apˆf . . . . . . . . . . 132
B-2 Performance of workflow Rit{FORM} for Average SCU of example EWG02. (a) Probability
of Failure pf (b) Accuracy of the probability of failure Apˆf . . . . . . . . . . . . . . . . . 133
B-3 Performance of workflow Rit{Monte Carlo} for Average Subsidence of example EWQ01. (a)
Probability of Failure pf (b) Accuracy of the probability of failure Apˆf . . . . . . . . . . 134
B-4 Performance of workflow Rit{FORM} for Average Subsidence of example EWQ01. (a)
Probability of Failure pf (b) Accuracy of the probability of failure Apˆf . . . . . . . . . . 135
B-5 Performance of workflow Rit{Monte Carlo} for Average Subsidence of example EWQ02. (a)
Probability of Failure pf (b) Accuracy of the probability of failure Apˆf . . . . . . . . . . 136
B-6 Performance of workflow Rit{FORM} for Average Subsidence of example EWQ02. (a)
Probability of Failure pf (b) Accuracy of the probability of failure Apˆf . . . . . . . . . . 137

Pablo A. Vásconez RESTRICTED Master of Science Thesis

 List of Tables

2-1 Values of target probability of failure according to Eurocode 7 (REF) . . . . . . . . . . . . 14

4-1 Failure mechanisms for each failure type . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

4-2 Performance functions for Subsidence and Formation Shear Failure . . . . . . . . . . . . . 49
4-3 Parameters of each reliability method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
4-4 Default Values for the parameters of the reliability methods used in this study . . . . . . . 60

5-1 Distribution of the random variables for model EWG . . . . . . . . . . . . . . . . . . . . 66

5-2 Results of workflows Resfor model EWG . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
5-3 Results of the evaluation of workflows Mod for all the proposed reliability methods . . . . 81
5-4 Evaluation efficient workflows in Failure Area Subsidence . . . . . . . . . . . . . . . . . . 85
5-5 Efficient workflows for Failure Area Subsidence . . . . . . . . . . . . . . . . . . . . . . . 86
5-6 Distribution of the random variables for model EWG02 . . . . . . . . . . . . . . . . . . . 88

6-1 Distributions of the Random Variables for example EWQ01 . . . . . . . . . . . . . . . . . 97

6-2 Distributions of the Random Variables for example EWQ02 . . . . . . . . . . . . . . . . . 102
6-3 Relative Error and New Threshold versus update number for selected cases with high accuracies112
6-4 Relative Error and New Threshold versus update number for selected cases with low accuracies113

B-1 Results of the evaluation of workflows Mod for Average SCU EWG02 . . . . . . . . . . . 131
B-2 Results of the evaluation of workflows Mod for Average Subsience EWQ01 . . . . . . . . 134
B-3 Results of the evaluation of workflows Mod for Average Subsience EWQ02 . . . . . . . . 136

Master of Science Thesis RESTRICTED Pablo A. Vásconez

 Acknowledgements

I would like to thank the staff from Civil Engineering of Delft University of Technology, specially to
those who supported the development of this thesis as committee members. To Prof. dr. Michael A.
Hicks, for helping the initiation of the project with Shell Global Solutions and for his relevant feedback
about the final report. To Dr. Jon Nutall, for the direction that the research should take and the
discussions about the presentation and explanation of the results, which contributed significantly to
the conclusions. To Dr. Phil Vardon, for this vital support in the organisation of the structure of
the report and corrections regarding grammar errors. Special thanks to Dr.ir. Pieter van Gelder, for
participating as the external reader, but more importantly, for his interest and feedback about the
project.
I am grateful to Shell Global Solutions for giving me the opportunity to participate in the project
named Containment Reliability Assessment CORA, the development and deployment of software to
perform reliability analysis in a problem which gathers knowledge from different fields of science and
engineering, including geomechanics. Thanks to Peter Fokker, the head of the team in charge of this
project, for the trust deposited in me. To Dr.ir. Peter van den Bogert for his feedback about the
structure and content of the report. To Dr.ir. Famke Kraaijeveld and Dr.ir. Marcela Cid, for their
daily supervision in the office at Rijswijk.
Special thanks deserved also Dr.ir. Rob Brinkman and Dr.ir. Edwin Spee from Deltares, in order to
understand the details of the reliability methods that they have implemented in the CORA.
Finally, but equally in importance, I would like to thank to my family and friends, which always cheered
for me in order to accomplish my Master Degree in Geo-Engineering in general and this project in
particular.

August 23, 2013

Pablo A. Vásconez RESTRICTED Master of Science Thesis

“As far as the laws of mathematics refer to reality, they are not certain; and as far as they
are certain, they do not refer to reality.”
— Albert Einstein
 Chapter 1

Introduction

1-1 Motivation of the Thesis

Today, common geotechnical engineering practices consider the uncertainties in the design by the
use of total, partial or load and resistance factors of safety. Their use is based mainly on a significant
amount of valuable experience. Nevertheless, they have the disadvantage of potentially leading to over-
conservative designs because they were derived for calculations considering problems with particular
geometries, soil variability and by the use of specific models for the behaviour of the geotechnical
structure.
In view of the current knowledge about dealing with complex real problems and the current capacity
to perform a high amount of computations in a small time frame, there is a tendency for researchers
and some engineers to use the so-called reliability-based design (RBD) to consider the uncertainties.
The advantages of such an approach, listed below, makes this approach very promising in producing
design with greater quality.

Advantages of Reliability-based Design (RBD)

• Helps the decision of the engineers and managers of projects, by presenting the uncertainties in
terms of likelihood.
• Reduces the amount of empiricism used in the design, by truly separating the uncertainties in
loads, strength and deformability parameters and models.
• Allows the inclusion of spatial variability and its effect with respect to the geometry of the
structures.
• Eliminate the inconsistency of applying a factor of safety on more realistic geomechanical models,
which were actually recommended for simple geomechanical models.
• Leads to more economical or more safe designs, depending on the specific objectives of the
projects.

However, it is important to recognise that is has to face several challenges before it can be successfully
inserted into daily practice. These are:

Master of Science Thesis RESTRICTED Pablo A. Vásconez

2 Introduction

Challenges of Reliability-based Design (RBD)

• Insufficient amount of data to build probabilistic distributions

• Insufficient knowledge of the engineers about reliability methods and probability theory in gen-
eral.
• Discussion about how small should be the probability of failure in order to consider a design as
satisfactory
• Additional time is required in order to perform reliability analysis

Research has been done to suggest procedures to overcome these challenges. For instance, the insuffi-
cient amount of data in projects, can be overcome with the use of Bayesian statistics as explained in
Christian (2004). The insufficient knowledge by the engineers on reliability methods can be reduced
by including in graduate education more emphasis in this field and by the publication of simple-to-
implement approaches to encourage engineers to have a better feeling of their use and the analysis of
their outputs.
An important challenge is how to define a particular probability of failure as satisfactory for a certain
project. For instance, satisfactory probability of failure can be obtained from the historical performance
of previous constructions. An example of the result of such a procedure is presented in Figure 1-1,
proposed by Baecher (1982), in which the vertical axis represents the number of failure events occurring
in a year versus the economical or life losses in the horizontal axis. Note that they are divided by types
of structures. Taken the case of fixed drill rigs, the probability of failure that is actually occurring is
of 0.001. If a new fixed drill rigs is designed, that level of probability of failure should be targeted,
unless more people or more loss in costs than usual are engaged by that failure.
The last of the listed challenges can be overcome by evaluating whether it is feasible to reduce the
time that a reliability analysis takes. The biggest amount of additional time comes from the many
scenarios that have to be evaluated in order to compute the probability of failure. In Monte Carlo, a
very common method to compute probability of failure, the amount of realisations required can easily
surpass tens of thousands. If an analytical model is used to represent the behaviour of the soil, then
the computation time is usually very small and the reliability analysis is feasible. Nevertheless, usually
more complicated models are required, resulting in significant computation times. One example could
be the use of finite element analysis.
In academia, Monte Carlo is usually performed even when a finite element model is used. In industry,
however, the constrains of tight deadlines and lower computational resources available for the design,
make impractical the use of Monte Carlo nowadays.
This thesis focuses on finding reliability methods and procedures capable of reducing significantly the
computation time in the computation of the reliability for geomechanical problems in the oil and gas
industry. It is of interest not to create new methods, but to use existing ones in order to evaluate if,
by accepting a loss in accuracy, they can actually compute the probability of failure more efficiently
for some specific examples than a Monte Carlo analysis.

Pablo A. Vásconez RESTRICTED Master of Science Thesis

ertainties tend Two mechanisms for presenting the results of probabilistic
analyses in a form that can be grasped intuitively and used in
misleading. decision-making are the f – N and F – N diagrams, the latter being
hould be; that the cumulative form of the latter. Fig. 18 is a typical example of
the f – N diagram (Baecher 1982). The plot has on the horizontal
1-1 Motivation of the Thesis 3
sed on evalu- axis either cost in dollars or lives lost.8 The vertical axis is the
reasoning; it observed annual frequency of the losses for various activities.
peculation or Both axes are logarithmic. The results plot along a broad swath
running from the upper left (small costs and high frequency of
failure) to lower right (high costs and low frequency of failure).
This is an experimental result; it reveals the rates of failure and
costs that society—or at least some operating part of society—has

ke any analy-
ean? How are
ut the impor-
ies?

means, espe-
7). developed
ced by other
e asked ques-
imates of the
res these esti-
ual frequency
than 0.2, the
tes. However,
bjective rates
e figure, when
bjects thought
were overcon-
n, people are
rcial airliners
ely publicized Fig.1-1:
Figure 18.Probabilities
One version of foccurred
of failure – N plotin annual risk cost
practice classified by or
typenumber of (Baecher,
of structure lives.
1982)
In this plot both cost and lives are shown; it is customary to use one
an driving on
or the other rather than both on same plot (Baecher 1982).

RONMENTAL ENGINEERING © ASCE / OCTOBER 2004

otech. Geoenviron. Eng. 2004.130:985-1003.

Master of Science Thesis RESTRICTED Pablo A. Vásconez

4 Introduction

1-2 Objectives of the Thesis

The aim of this thesis is to investigate methods or procedures that can reduce the additional time
that a probabilistic analysis takes. Several methods have been proposed in literature, although mainly
applied to structural engineering. This thesis will investigate the application of the reliability methods
in geotechnical problems. The following objectives are set up:

General Objective

The general objective is to investigate the performance of methods and/or procedures to compute the
reliability geotechnical problems.

Specific Objectives

1. Test the accuracy and efficiency of several reliability methods for specific geotechnical examples.
2. Identify methods that have and do not have potential to compute the reliability.
3. Explain the reasons behind the methods that makes them to have or not to have potential to
compute the reliability
4. Recommend improvements to the methods that can either increase their efficiency or accuracy

1-3 Methodology of the Thesis

The methodology starts by collecting the most widespread reliability methods proposed in literature.
This is done in Chapters 2 and 3. Chapter 2 presents formal reliability methods, whereas Chapter 3
introduces the concept of response surface and how can it help the computation of the reliability. The
knowledge gained until that point, helps the reader to understand the basis of the procedures used as
support to the analysis, discussion and conclusions for the thesis.
In Chapter 4, the methods or procedures proposed (named workflows) are formulated based on the
literature from Chapter 2 and Chapter 3. One of the workflows contain an additional feature not
published in literature: the use of a threshold. The workflows used are classified in three types:
Res, which uses a response surface and a reliability method; Rit, which uses a response surface that
updates during the execution of reliability method according to the threshold, and Mod, which perform
a reliability method directly on the geomechanical model.
The evaluations of the workflows are performed in Chapters 5 and 6. It was convenient to separate
them in quasi-linear and non-linear responses to have a deeper understanding of the situations that
can be faced in practice. Both types of examples come from the oil depletions of a reservoir, being both
non-linear but differing in the degree of non-linearity. The examples identified as non-linear present
observable non-linearities already close to the mean values of the random variables. All the examples
(with the exception of the last case presented) have a reference probability of failure close to 0.002.
Chapter 5 presents the results for examples with quasi-linear responses. First, the terms of the
evaluation are stated. Second, a first example, the simplest between of the evaluated is introduces and
evaluated for all the workflows proposed. Third, the methods that have good potential to compute the
reliability are evaluated further for more complex examples. A detail explanation for the performance
is provided.
Chapter 6 presents the results for examples with non-linear responses, i.e. examples with a higher
degree of non-linearity. The reduction in performance for some methods, led to the re-evaluation of

Pablo A. Vásconez RESTRICTED Master of Science Thesis

1-3 Methodology of the Thesis 5

an example for a different target of probability of failure. Considering all the results, the limitation of
the applicability of workflows Rit is identified and further investigated.
Finally, Chapter 7 presents an overview of all the evaluations performed in the study, providing the
conclusions based in the examples performed, but also in view of future, more complicated applications.
The recommendations are formulated considering the use of the proposed workflows and also some
minor changes that could be attempted. Further research topics are also proposed regarding more
fundamental changes, targeting unresolved issues and suggesting more fundamental changes in the
procedures of the workflows.

Master of Science Thesis RESTRICTED Pablo A. Vásconez

 Chapter 2

A review on the reliability methods

2-1 Introduction
This chapter gives an introduction to the topic of reliability computation. It starts by defining what
the probability of failure mathematically is. From its definition, it follows that often only numerical
solutions are possible. Many approaches have been developed in literature, with the widely used
methods introduced here. The fundamentals of each method are explained, with reference to the
definition of the reliability. Finally, their advantages and limitations are presented.
The objectives of the chapter are therefore to:

• Introduce the mathematical definition of the reliability

• Describe the theory behind the widely used methods for the computation of the reliability
• Explain briefly the procedure of their implementation with emphasis on the methods used in
this study
• Highlight the advantages and limitations of the aforementioned methods

2-2 Definition of Reliability

The reliability of an engineering system, structure or element is the probability that it meets one or
several specified demands under specified environmental conditions (Ayyub and McCuen, 2003). A
typical example of an engineering structure in geotechnics is a shallow foundation. One of the demands
to be satisfied is the stability of the foundation element towards vertical loading. This is fullfilled when
the bearing capacity of the soil (here denoted by R) exceeds the total vertical load (here denoted by
L). Hence, the demand for this example is expressed mathematically by the notation R > L.
Let Z to denote the performance function of the shallow foundation (or the system or structure under
consideration) and is defined as:

Z =R−L (2-1)

Pablo A. Vásconez RESTRICTED Master of Science Thesis

2-2 Definition of Reliability 7

The notations R and L are given by the traditional structural mechanics definition of the performance
function which compares the resistance and the load. In a more general manner, R must be understood
as a limit value and L as the actual value, each of them selected or computed according to the specified
demands and conditions. In this sense, R could also express supply, capacity, maximum settlement,
etc. and L could also express actual settlement, demand (water supply systems), etc.
Furthermore, the values of R and L depend upon certain variables such as the soil properties and
parameters, and also Probabilistic
on the acting loads.Engineering
These constitute the n basic variables x1 , x2 , . . . , xn of the
Design
performance function and are represented by x. Hence, Eq. (2-1) could be rewritten as a function of
the basic variables:

ChapterZ Seven
= g(x) (2-2)

First Order and Second Reliability Methods

The performance function owes its name to the fact that its output allows to know how the system or
structure performed. Three outcomes of the performance are possible (Figure 2-1):
Xiaoping Du
• g(x) > 0: The shallow foundation is safe (safe region in the x-space)
University of Missouri – Rolla
• g(x) = 0: The shallow foundation is on the limit state
• g(x) < 0: The shallow foundation fails (failure region in the x-space)

Figure 2-1: Safe and Failure Regions (Du, 2005)

Two regions are identified in the x-space: September the safe2005 region, in which the demands are satisfied, and
the failure region, where the demands are not satisfied. The limit between these two regions are
combinations
Copyright in the
Agreement: x-space
All the materialswhich
are for are onofverge
the use of private
individual failure, i.e.only.a They
study limitmaystate. For this
not otherwise reason,
be copied or in this
document, the function that describes these combinations,
distributed in any way in whole or in part without the permission of the author. g(x) = 0, is called the limit state function

Master of Science Thesis RESTRICTED Pablo A. Vásconez

8 A review on the reliability methods

LSF. Note that the x-space is a set which contains all the possible combinations of the basic variables
in their original units.
To quantify the probabilities, it is necessary to treat the basic variables x as basic random variables
X. This means to define a set of possible different vectors X = (X1 , X2 , . . . , Xn ), each with an
associated probability. For the real-valued variables addressed in engineering, the function that allows
this treatment is the joint probability density function fX (x) (vertical axis of Figure 2-1). The integral
of fX (x) over a region of the x-space provides the probability of ocurrence for a basic random vector X
within the limits of the region. The lower and upper limits are xl and xu , respectively. The previous
concept is represented by Eq. (2-3). An important characteristic of fX (x) is that its integral of the
whole x-space is equal to 1 (xl → −∞ and xu → +∞).

Z xu
l u
p(x ≤ X ≤ x ) = fX (x) dx (2-3)
xl

The reliability Re can be now defined more specifically as the probability of that the performance
function is on the safe region. Mathemathically this is expressed by

Re = p(R > L) = p[g(X) > 0] (2-4)

By introducing Eq. (2-3) in (2-4), the equation to compute the reliability is

Z
Re = fX (x) dx (2-5)
g(X)>0

which means that the reliability is the integral of the joint probability density function (or joint PDF)
over the safe region. Note that the limit state function is a boundary of the integration.
If the performance function (and hence the limit state function) is linear or quadratic, then an exact
solution of Eq. (2-5) is possible. However, in practice this seldom or never occurs. Approximate
methods are required. The most obvious is to compute the integral by numerical methods. For the
usual high values of reliability to target in practice, it would be more efficient to compute the integral
of fX (x) over the failure region, i.e. the probability of failure. It is defined as

pf = p(R < L) = p[g(X) < 0] (2-6)

and computed by the equation

Z
pf = fX (x) dx (2-7)
g(X)<0

Once the pf is defined, the reliability can be computed by

Re = 1 − p f (2-8)

The methods proposed to compute the integral of Eq. (2-7) are:

• Numerical integration.

Pablo A. Vásconez RESTRICTED Master of Science Thesis

2-3 Numerical Integration to determine reliability 9

• Simluation methods (Monte Carlo, Importance and Directional Sampling), in which many dif-
ferent sets of values for the variables are sampled from their the probability density functions
to evaluate the performance function. The great advantage of these types of methods is their
robustness and the accuracy obtained. However, the amount of evaluations required makes them
expensive in time and/or computational resources, limiting their applicability in practice. Even
though, they are important for validation of less expensive methods.

• Methods approximating the failure surface g(x) = 0 (FORM and SORM), which first find the
most likely failure point (i.e., the design point) and then compute the reliability assuming the
failure surface either as a linear or quadratic. The advantage of these methods is their efficiency,
because less number of evaluations are required compared to the sampling methods, for the same
accuracy level. Nevertheless, in the presence of noise, high nonlinearities and multiple design
points their accuracy and even their robustness are in question.

A couple of remarks about the nomenclatures defined in this section:

• Several studies (Grooteman, 2011; Nie and Ellingwood, 2000; Harbitz, 1986) regard g(x) as the
limit state function. The nomenclature proposed here, as used in (Melchers, 1990), is considered
more appropriate because (i) g(x) 6= 0 does not represent a limit state and (ii) g(x) has one
more dimension than g(x) = 0. The name performance function, as used in (Phoon, 2008; Du,
2005), is appropriate for g(x).

• Note that in Figure 2-1, the x-space comprises two variables x1 and x2 only. Thus, the limit
state function is line (1D function) and the performance function is a surface (2D function, not
plotted in Figure 2-1). There is no restriction, however, to the number of basic variables that
the problem couuld have, i.e. n could be more than 2. It is preferable, to consider the limit
state and performance functions in the more general case as hypersurfaces (which are difficult
to imagine and impossible to plot them in a single graph).

2-3 Numerical Integration to determine reliability

In this method, Eq. (2-7) is solved by discretizing the random variables X. Notice that the limits
of the integral are defined over the failure region in terms of g(X). The correspondent limits in the
x-space, are obtained by introducing the indicator variable I(x), defined as:


1 if g(X) <= 0
I(X) = (2-9)
0 if g(X) > 0

It determines if failure has occured or not for a given set of values of the random variables. The
probability of failure is now defined by the equation:

Z ∞
pf = I(x)fX (x) dx (2-10)
−∞

This new equation of the probability of failure is the desired form: the integration limits are defined
over the x-space and only the failures will add up to the integral.
The aforementioned discretization is performed on Eq. (2-10), giving the Numerical Integration esti-
mator of the probability of failure p̂f,N I

Master of Science Thesis RESTRICTED Pablo A. Vásconez

10 A review on the reliability methods

m1 X
X m2 mn
X
p̂f,N I = ··· I(x0,1 + (j1 − 0.5)∆x1 , x0,2 + (j2 − 0.5)∆x2 , . . . , x0,n + (jn − 0.5)∆xn )
j1 =1 j2 =1 jn =1

fX (x0,1 + (j1 − 0.5)∆x1 , x0,2 + (j2 − 0.5)∆x2 , . . . , x0,n + (jn − 0.5)∆xn )

∆x1 ∆x2 , . . . , ∆xn
(2-11)

where x0,i is the lower range limit for the ith variable; ∆x0,i , the interval width of the ith
variable and mi is the upper bound of i such that x0,i + mi ∆xi is the upper bound of the ith
variable. This equation allows for a equidistant grid, but it could be modified to allow any
convenient discretization.
A graphical representation of the method is sketched in Figure 2-2, for 2 random variables X1
and X2 . The red grid points represent failure (g(X ≤ 0)) and the star grid points indicate
no failure (g(X > 0)). The joint PDF value of each red point (failure domain) is multiplied
by its area of influence (∆X1 × ∆X2 ). The sum of all these values is the approximation of
the probability of failure.
3

1
X2

−1

−2

−3
−3 −2 −1 0 1 2 3
X1

Figure 2-2: Grid in the x-space for Numerical Integration method (∆X1 is the distance between
the vertical lines and ∆X2 , between the horizontal ones

Advantages and Limitations

The numerical integration method has the advantage of being robust, i.e. an approximate
result of the probability of failure would be obtained. The reason lies in the formal mathe-
matical procedure behind the method. The required accuracy must be ensured by appropriate
sizing and refinement of the grid to overcome the following assumptions (Brinkman, 2012):

Pablo A. Vásconez RESTRICTED Master of Science Thesis

2-4 Simulation Methods for determining the reliability 11

• Each grid cell is assumed to be either completely in the failure domain or completely
outside the failure domain. In practice, there are cells partially in the failure and outside
the failure domain, changing the weights for the probability (Figure 2-2).

• The joint PDF is assumed to be a constant for the entire cell, which seldom is the case.

• A closed region of the x-space is covered, disregarding some possible outcomes of the
random variables.

The biggest disadvantage of the method is the large computation time. As discussed by
Caflisch (1998), the simplest hypercubic grid in n dimensions, requires at least 2n points. For
n = 20, more than 1 million points are needed to be computed. Furthermore, at least one or
two refinements of the grid would be necessary to check the convergence of the result. At the
end, several million calculations could be needed, restricting the applicability of the method
for practical problems.

2-4 Simulation Methods for determining the reliability

2-4-1 Crude Monte Carlo

The first publication of the Monte Carlo method traces back to (Metropolis and Ulam, 1949).
It is method that provides a numerical solution for a problem (complicated using analytical
means) by generating random numbers and observing the fraction of numbers that obey an
specified demand. The type of mathematics behind it is experimental, thus the results are
inferred from the observed data, rather than deducted (Hammersley, 1960). In the same way
that in a physical experiment, the results are subject to experimental error, therefore they
can be regarded only as approximate.
As seen in the previous section, the computation of the probability of failure using Eq. (2-10)
is complicated in practice. It is a suitable candidate for the use of Monte Carlo. But first, it
is necessary to analyze Eq. (2-10) in more detail.
According to probability theory, Eq. (2-10) is the mean or expectation value of the indicator
variable Ef [I(X)], with respect to the distribution fX (x). The Strong Law of Large Numbers
(Feller, 1968) supports that the sum of simulated values I(X1 ), I(X2 ), . . . , generated by a
sequence of independent values X1 , X2 , . . . (in accordance with fX (x)), converges to Ef [I(x)]
as the number of samples N tends to infinity. This is expressed by:

N
1 X
Ef [I(x)] = lim I(X) (2-12)
N →∞ N
i=1

In the Monte Carlo method, Eq. (2-12) is computed for a finite number of random samples
N . Considering the equality between pf and Ef [I(x)], the Monte Carlo estimator for the
probability of failure p̂f,MC is:

N
1 X Nf
p̂f,MC = I(X) = (2-13)
N i=1 N

Master of Science Thesis RESTRICTED Pablo A. Vásconez

12 A review on the reliability methods

where Nf denotes the number of simulations that fail. The generation of the random samples
is performed according to the joint PDF of the random variables. In this way, more vectors
X will be sampled from the region with higher values of the probability density, e.g. the peak
of the surface in Figure 2-1.
A representation of the Monte Carlo method in the x-space is shown in Figure 2-3 for 2
random variables X1 and X2 . The red points indicate failure (g(X) ≤ 0), whereas the black
star markers determine no failure (g(X) = 0). Notice that more points are sampled around
the mean value of (0,0) than far from it. This reflects the joint PDF used for this example.
In comparison with Figure 2-2, a bigger region of the x-space is covered in a more efficient
way: usually less number of points are necessary to converge to the result.

1
X2

−1

−2

−3
−3 −2 −1 0 1 2 3
X1

Figure 2-3: Crude Monte Carlo simulation in the x-space

The probability of failure is simply computed by getting the number of red circular points,
i.e. Nf , and the divide it by the number of samples taken, i.e. N .

Advantages and Limitations

The advantages of the Monte Carlo method are the following:

• It is a robust method, which means that it always converges to a numerical result with
the increase of number of samples.

• As it will be explained in the following paragraphs, the error does not depend on the
number of dimensions n, representing an improvement in efficiency compared to the
numerical integration method. The reason is that all the random variables are varied
at the same time they are sampled, covering more efficiently the x-space.

Pablo A. Vásconez RESTRICTED Master of Science Thesis

2-4 Simulation Methods for determining the reliability 13

The relative error of the probability of failure  is defined by the equation:

|p̂f,MC − pf |
= (2-14)
pf

It is observed that this error will also have a probabilistic distribution. A confidence interval
for the estimator of probability of failure could be defined as follows:

pf (1 − ) < p̂f,MC < pf (1 + ) (2-15)

The confidence level used in practice is usually 95%, meaning that there is a 95% of probability
that the estimator is in the range defined by Eq. (2-15) or, in order words, that the relative
error is smaller than . Assuming a normal distribution for p̂f,MC with mean pf and standard
deviation σp̂f,MC , the confidence interval could be expressed by

pf − kσp̂f,MC < p̂f,MC < pf + kσp̂f,MC (2-16)

where k is the inverse of the standard normal distribution, which has a direct relation with
the confidence level: a value k = 1.96 corresponds to a confidence level of 95%. Equating
Eq. (2-15) and Eq. (2-16), gives the following relationship between the parameters of the
distribution of the probability of failure, the relative error and the confidence level:

 σp̂
= f,MC
k pf (2-17)
 = kCOVp̂f,MC

which basically means that the  is proportional to the Coefficient of Variation (COV) of the
probability of failure. The latter is actually equal to the COV of Nf , since the distribution
of the estimator only depends on Nf . Furthermore, Nf follows a binomial distribution with
parameters N and pf (i.e. Nf ∼Bin(N, pf )). Therefore:

s
1 − pf
COVp̂f,MC = (2-18)
N pf

The number of samples N to take could be expressed as a function of the estimation of the
probability of failure pf and the accuracy level defined either by the relative error  or the
coefficient of variation COVp̂f,MC by the equations:

1 − pf
N=
pf (COVp̂f,MC )2
! (2-19)
k2 1 − pf
N= 2
 pf

Master of Science Thesis RESTRICTED Pablo A. Vásconez

14 A review on the reliability methods

Moreover, if a convention is adopted for the accuracy level, the efficiency would only depend
on the target probability of failure pf . A value of COVp̂f,MC = 0.3 is considered sufficiently
small by (Phoon, 2008), which implies  ≈ 0.6). By adopting this convention, the required
number of samples would be:

1 − pf 10 10
N= 2
≈ − 10 ≈ (2-20)
pf (0.3) pf pf

Therefore, the efficiency of this method does not depend on the number of n. For practical
purposes, this means that no matter how many random variables a problem has, the Monte
Carlo method requires the same amount of samples, which is a good characteristic compared
with numerical integration if several random variables n are considered.
However, in practice, the targets for probability of failure are very small. The standard for
geotechnical engineering, EN 1997 - Eurocode 7, suggests pf = 2 × 10−3 for Serviciability
Limit State (SLS) consideration (Table 2-1). An example of this is the subsidence produced
by the depletion of a reservoir during oil extraction. Applying Eq. (2-20), the problem will
require N = 5, 000 samples. Each realization represents a deterministic computation using a
specific software. If the software use is a 3D FEM program considering nonlinear constitutive
models, one single realisation typically lasts 30 minutes. In order get the probability of failure,
a total of 2,500 hours are therefore needed.
Note that a Ultimate Limit State (ULS) consideration will take a significant higher amount of
time to get an output. Clearly, the greatest limitation of this method is the large computation
time when advanced numerical modelling is used to obtain each deterministic output.
Table 2-1: Values of target probability of failure according to Eurocode 7 (REF)

Limit State Target Probability of Failure, Pf Napprox Eq. (2-20)

1 year 50 years 1 year 50 years
−6 −5
ULS 1 × 10 7.2 × 10 10,000,000 140,000
SLS 2 × 10−3 6.7 × 10−2 5,000 150

2-4-2 Importance sampling

The earliest proposals in this type of simulation are made by (Shinozuka, 1983) and (Harbitz,
1983). The basic concept of importance sampling, as its name suggests, is to sample from
an important region rather that in the whole x-space. Normally, small probabilities of failure
are of interest, implying that only a few amounts of the samples taken will fall in the failure
region. If only samples in the failure region or at least very close to the limit state would be
sampled, then the analysis could be extremely efficient. Therefore, the important region is the
one nearby the limit state, and should be big enough in order to sample more failures. The
non-important region would be considered either safe or it is too far from the mean values to
produce an impact in the probability of failure.
The sampling from the important region is performed from the importance sampling density
function, hX (x), instead of the original PDF. Thus, Eq. (2-7) is modified in order to include
the importance sampling density function as follows

Pablo A. Vásconez RESTRICTED Master of Science Thesis

2-4 Simulation Methods for determining the reliability 15

Z ∞
fX (x)
pf = I(x) hX (x) dx (2-21)
−∞ hX (x)

where I(x) is the same indicator function as defined in the original Monte Carlo method.
Therefore, the estimator of the probability of failure, according to the importance sampling
method:

N
1 X fX (xi )
p̂f,IS = I(Xi ) (2-22)
N i=1 hX (xi )

Comparing Eq. (2-22) with the estimator for Monte Carlo (Eq. (2-13)), it is observed that
importance sampling weights each realisation by the factor hfX (xi )
X (xi )
. Note that this factor is not
necessarily a constant value. It will be if a hX (x) is chosen to be proportional to the original
PDF fX (xi for the whole x-space. Since the utility of hX (x) is to sample in regions close to the
limit state, more failures will normally be accounted in comparison with the original Monte
Carlo implementation. This leads to the conclusion that, if importance sampling succeeds,
the factor hfX (xi )
X (xi )
will be smaller than 1 for most of the realizations.
The major challenge about this method is to provide a good selection for hX (x). It is
recognized that it is not possible to find a general selection that will be efficient for all
cases (Swiler and West, 2010). In addition, there is the restriction that hX (xi ) must not be
0, unless either fX (xi ) or I(Xi ) are zero. The following are recommendations presented by
Swiler and West (2010), to select a good importance sampling function:

• hX (xi ) > 0 if I(Xi )fX (xi ) 6= 0

• hX (xi ) must be nearly proportional to |I(Xi )fX (xi )|

Several authors have attempted selections for the importance sampling density function, and
some of them have even proposed more advanced procedures by modifying the basic concept
here presented. A good summary was made by Engelund and Rackwitz (1993), in which
basically four ramification of importance sampling are presented: direct methods, updating
methods, adaptive schemes and spherical schemes.
The aim in this section is to keep the concept of importance sampling in a more general
view. In this sense, only two selections that Engelund and Rackwitz (1993) defined as direct
methods are presented in this document. They are presented graphically in Figure 2-4: (a)
the increase of variance of the density function and (b) the shifting of the function towards
the failure domain. To the left of the figure, the original variance is increased by a factor
called the variance factor, γσ which modifies the ratio between the standard deviation of
the original density function and the standard deviation of the importance sampling density
function. With the increase of variance, the contours of hX (x) cover a bigger portion of the
x-space than the ones for fX (x). In this manner, the samples falling in the failure region will
be more frequent, reducing the computation time in principle.
For the ease of the mathematical computations, the ratio fX (xi )/hX (xi ) is derived for the
case of n uncorrelated random variables, all with a standard normal distribution. The x-space

Master of Science Thesis RESTRICTED Pablo A. Vásconez

16 A review on the reliability methods

(a) (b)

Figure 2-4: Two variations of importance sampling: (a) Increase of variance of the density
function, (b) Shifting towards failure domain (Waarts, 2000)

for this particular case is called u-space in literature. Hence, the u-space is a set of all the
possible combinations of uncorrelated random variables that have a mean of 0 and a standard
deviation of 1. Beware that it is possible to transform an x-space with any correlation and
distributions to the u-space via the Rosenblatt transformation (Rosenblatt, 1952).
The assumption of uncorrelated random variables allows the decoupling of the probability
density functions fU (u) and hU (u), reducing them into f (u) and h(u), respectively. The
ratio f (u)/h(u) is found by using the equations of the PDF of the two distribution. Both are
normal with a mean of 0, but with different standard deviations: 1 for f (u) and γσ for h(u).
This is written in the expression:

2
√1 exp( −u )
f (u) 2π 2
= (2-23)
h(u) √ 2 exp( 2γ 22 )
1 −u
2πγσ σ

Simplifying the equation for the ratio is:

2
f (u) exp( −u
2 )
= γσ −u2 (2-24)
h(u) exp( 2 ) 2γσ

Given that for uncorrelated random variables the probability densities can be multiplied in
order to obtain the probability of the joint density, the equation for the ratio fU (u)/hU (u)
can be expressed by:

n
exp( 12 u2k )
P
fU (ui ) k=1
= γσn n (2-25)
hU (ui ) exp( 1 P
u2k ))
2γσ2
k=1

Pablo A. Vásconez RESTRICTED Master of Science Thesis

2-4 Simulation Methods for determining the reliability 17

Back to Figure 2-4, the right plot shows the shift of the original probability function to a
region located at failure. Mathematically, this can be simply representing in a general form
by hX (x) = fX (x − xs ), where the vector xs is the new center towards the function is shifted.
The problem about this selection is that it needs prior knowledge of the location of the failure,
which normally will require additional computations. For instance, the method FORM can
be applied to find the most probable failure point (see Section 2-5-1 for more details) .

Advantages and Limitations

The best characteristic of the direct methods mentioned in the previous sections is their
robustness (Engelund and Rackwitz, 1993). Further, if a proper selection for the importance
sampling density function is made, they also are likely to be very efficient.
Nevertheless, without prior knowledge of the problem, the proposed importance sampling
methods are not guaranteed to be efficient. The saving in computation time made in an
initial estimation using particular parameters of the methods (e.g. a variance factor of 2)
may be diminished or totally disappear by the recalculation with different parameter in order
to achieve convergence of the same result.

2-4-3 Directional sampling

Another method to reduce the computation time from Monte Carlo is Directional Sampling,
first proposed by Deak (1980).
This method attempts to take advantage of the rotational symmetry of the joint PDF fU (u)
in the u-space, which is always a multinormal distribution (Lemaire, 2010). Recall that
regardless the individual distributions of the random variable and their correlation, a trans-
formation to the u-space is possible (Rosenblatt, 1952) . The proposed procedure is to sample
in a radial manner, i.e. to generate random directions α(i) instead of random values for the
uncertain parameters (x). The random directions are generated in the u-space to be uniformly
distributed on the unit hypersphere centred at the origin. Figure 2-5 shows three random
directions α(1) to α(3) for two random variables in the u-space.
The next part of the procedure is to find, for each direction α, the radial distance (with
respect to the origin) for which the limit state is reached, here called as ρ(α). The integral
to compute pf can be now expressed in polar coordinates as:

"Z #
Z ∞
pf = fR (r|α)dr fA (α)dα (2-26)
unit sphere ρ(α)

in which fR (r|α) is the conditional probability density function for the radius r, given that
a specific direction α occurs; and fA (α) is the probability density function of the random
directions α. Note that R and A are the random-variable representation of the variables
r and α, respectively. Being the sampling uniformly distributed from the unit sphere, the
density fA (α) is the uniform distribution. The density fR (r|α) is not free to be chosen. It is
derived from the relationship:

Master of Science Thesis RESTRICTED Pablo A. Vásconez

18 A review on the reliability methods

u2
α(1) α(2)

g(U) < 0

(3)
α
g(U) > 0

u1
Figure 2-5: Schematic representation of the Directional Sampling. Dashed lines are the random
directions α

U = RA (2-27)

The square of the modulus of U results in:

|U |2 = R2 |A|2 = R2 (2-28)

because A is a unitary vector. Expressing R2 in terms of the components of U gives:

R2 = U12 + U22 + . . . Un2 (2-29)

It is obtained that R2 follows the well-known chi-squared distribution, i.e . R2 ∼ χ2n . The
PDF and CDF of this distribution are expressed as fχ2n and χ2n , respectively. Therefore, an
explicit expression for the inner integral in Eq. (2-26) is obtained:

Z ∞ Z ∞
fR (r|α)dr = fχ2n (r2 |α)dr = 1 − χ2n (ρ2 (α)) (2-30)
ρ(α) ρ(α)

Substituting the new expression for the integral in Eq. (2-26), the probability of failure can
be computed by:

Z h i h i
pf = 1 − χ2n (ρ2 (α)) fA (α)dα = E 1 − χ2n (ρ2 (α)) (2-31)
unit sphere

The third term of the previous equation suggests that sampling uniformly from the unit
sphere, allows to compute the probability as the expected value of 1−χ2n (ρ(α)). An estimator

Pablo A. Vásconez RESTRICTED Master of Science Thesis

2-4 Simulation Methods for determining the reliability 19

of the probability of failure can be computed accurately by a sufficiently large number of the
total amount of sample directions Nd with the expression:

Nd
1 X
p̂f,DS = Pi where Pi = 1 − χ2n (ρ(α(i) )) (2-32)
Nd i=1

where i corresponds to one sample direction. The coefficient of variation COV determines
when the amount of sampled directions Nd is sufficient. As explained for Monte Carlo, a
value of COV = 0.3 may be considered enough. An approximate expression for the coefficient
of variation for directional simulation is Ditlevsen et al. (1990):

v
u dN
1u 1 X
COVp̂f,DS = t (pi − pf )2 (2-33)
pf Nd (Nd − 1) i=1

It is observed that the coefficient of variation depends on the value of pi and subsequently in
ρ(αi ). This implies that the number of sampled directions ultimately depends on the shape
of the limit state function g(x) = 0. For instance, if the shape of the limit state function
is the one of the hypersphere, the summation term would be 0 from the second direction
already, regardless the probability of failure. Therefore, the closer the limit state function is
to a hypersphere, the lower the computation time required.
Note that the total amount of realizations N of the performance function is higher than
the total amount of sample directions Nd , because several g(x)-realizations are required per
sample direction to find the radius ρ(αi ). To do this, a so-called linear search method is used.
This is an iterative procedure depicted in Figure 2-6 for two random variables in the u-space.
Points numbered from 1 to 5 refer to successive performance function g(x) = 0 realizations
in one single direction. After the first 2 samples the location a value for ρ(αi ) can be first
estimated. This can be checked by a new realisation, i.e. point 3. If the tolerance is satisfied,
only 3 samples are necessary. If not, more realisations are needed as in the fictional case
presented in Figure 2-6 to refine the search. The linear search procedure can be also observed
in an elevation view along the sample direction. The x-axis is the modulus |u| of the vector
of the random variables in the u-space. The vertical axis shows the value of the performance
function. It is observed, that sample number 5 is the value of the searched radius ρ(αi ).

Advantages and limitations

The main advantage of directional simulation is the reduction of computation time. This
derives from the computation of the probability of failure per direction by only finding the
distance between the origin and the limit state.
Nevertheless, as pointed out by (Ditlevsen and Madsen, 1996), this method is not efficient
when the failure region is a half-space and number of random variables n is not small. The
reason is that the outcomes of ρ(A) are very frequently excessively large and they become
larger with the number of dimensions n.

Master of Science Thesis RESTRICTED Pablo A. Vásconez

20 A review on the reliability methods

u2

g(U) < 0
g(U) > 0

α(i)
3
ρ(α(i)) 5
4

u1
Figure 2-6: Schematic representation of the Directional Sampling. Dashed line is a random
direction α(i) and the numbers represent realisations for the linear search method in chronological
order

2-5 Methods approximating the performance function

Another variant of reliability methods, which attempt to reduce the computation time of
Monte Carlo, are the methods which approximate the performance function g(X). But this
approximation is not performed for the whole x-space. Rather, its focus is the limit state
contour g(X) = 0 around the most probable failure point, taking advantage of the fact that,
the further from this point, the lower the effect on the probability of failure introduced by
inaccurate prediction of g(X).

The most widely used from this family are First-Order Reliability Method (FORM) and
Second-Order Reliability Method (SORM). Their names correspond to the order of the
approximation they use for g(X): first-order (linear) in FORM and second-order (quadratic)
in SORM. Figure 2-7 shows the geometrical representation for a space with n = 2 random
variables. It can be observed that the approximation by FORM and SORM produce exactly
the same value of g(X) = 0 at the most probable failure point or design point x∗ or u∗
depending on the space used as a basis (x-space or u-space, respectively). Further away from
this point, the prediction is inaccurate for both methods, although the loss of accuracies in
generally lower for SORM.

Once, the approximate performance function ĝ(X) is known, it is used to compute the prob-
ability of failure.

SORM is not described in this thesis. References for this method are Der Kiureghian et al.
(1987); Hohenbichler et al. (1987).

Pablo A. Vásconez RESTRICTED Master of Science Thesis

2-5 Methods approximating the performance function 21

Limit State
X2
g(X) = 0

Failure domain
g(X) < 0
SORM
Safe domain
g(X) > 0

X1
FORM

Figure 2-7: FORM and SORM approximation of the performance function g(X)

2-5-1 First-Order Reliability Method (FORM)

This section focus on the theory behind FORM, probably the most important reliability
method developed as an alternative to Monte Carlo. One reason is the great reduction time
that it produces for a number of problems. But also, and maybe equally in importance, it
introduces some useful concepts for the understanding of the n-dimensional space, such as
the design point and the reliability index.
FORM was proposed by Hasofer et al. (1974). The development is presented for the case in
which the variables are uncorrelated and have the standard normal distribution. But it must
be known that this method is applicable to any distribution type and cross-correlation of
interest, via the Rosenblatt transformation (Rosenblatt, 1952), to convert the variables from
the x-space to the u-space.
In this study, also the u-space is used for the explanation of the method. Therefore, in this
section, the performance function is referred as g(U ) instead of g(X).
The performance function is approximated by using a first-order Taylor expansion, given by
the equation:

g(U ) ≈ ĝFM (U ) = g(u∗ ) + ∇g(u∗ )(U − u∗ )T (2-34)

where ĝFM (U ) is the first-order approximation of the performance function, u∗ = u∗1 , u∗2 , . . . u∗n
is the point for the Taylor expansion, T is the transpose operation, and ∇g(u∗ ) is the gradient
of g(U ) at u∗ . The latter has the expression:

Master of Science Thesis RESTRICTED Pablo A. Vásconez

22 A review on the reliability methods

∂g(U ) ∂g(U ) ∂g(U )

 
∇g(u) = , ,..., (2-35)
∂u1 ∂u2 ∂u3 u∗

Note that the point u∗ in the previous section is named the most probable failure point or the
design point and here is recognized as the expansion point for the Taylor expression. Indeed,
the selected point in FORM to apply the Taylor expansion is the most probable failure point.
This is a natural selection, since the highest contribution for the probability of failure in
Eq. (2-7) come from the highest values of the integrand fX (x) (φU (U ) in the u-space). With
the integration going away from this point, the values of φU (U ) quickly diminishes.
To find the most probable failure point is mandatory for this method. Mathematically, this is
done by solving the following constrained optimization problem:
(
max φU (u)
u such that (2-36)
subject to g(u) = 0

where φU (u) is the standard multinormal density function of equation:

1 1
φU (u) = n/2
exp(− uT u) (2-37)
2π 2

Given that the argument of the exponent in Eq. (2-37) is a function of the square of the
modulus of u, i.e. |u|2 and that it will always be negative value, the optimization could be
restated as a minimization problem as follows:
(
min |u|
u such that (2-38)
subject to g(u) = 0

The solution to this problem gives the most probable failure point u∗ and also the modulus
of |u|. Both are presented in Figure 2-8. The latter of the two outputs, is also known as the
reliability index, β. In other words, the β provides the shortest distance from the origin to
g(u) = 0 in the u-space.
Returning to Eq. (2-34), and substituting g(u∗ ) = 0, the approximation of the performance
function is

n
ĝFM (U ) = ∇g(u∗ )(U − u∗ )T = −∇g(u∗ )u∗ T + ∇g(u∗ )U T = a0 +
X
ai Ui (2-39)
i=1

where is a constant value resulting from the inner product a0 = −∇g(u∗ )u∗ T and ai are
the components of the gradient ∇g(u∗ ). The last expression in Eq. (2-39) is used to show
that ĝFM (U ) is a normally distributed random variable, because it results from a linear
combination of normally distributed random variables. Its mean and standard deviation can
be computed by the following equations:

µĝFM = a0 = −∇g(u∗ )u∗ T (2-40)

Pablo A. Vásconez RESTRICTED Master of Science Thesis

2-5 Methods approximating the performance function 23

u2

u*
g(U) < 0

β=|u*| g(U) > 0

g(U) = 0

u1
Figure 2-8: Most probable failure point u∗ and reliability index β. Colours are PDF contours.

v
u n
= t a2i = |∇g(u∗ )|
uX
σĝFM (2-41)
i=1

Having the distribution of ĝFM (U ), the probability of failure can be directly computed. It
is only necessary to substitute ĝFM (U ) = 0 in the cumulative distribution of ĝFM (U ). This
operation is performed in the following equation:
!
0 − µĝFM ∇g(u∗ ) ∗ T
   
p̂f ;FM =Φ =Φ u = Φ αu∗ T (2-42)
σĝFM |∇g(u∗ )|

where α is the unit vector of the gradient ∇g(u∗ ). The operation αu∗ T is an inner product,
which geometrically represents the projection of u∗ in the direction α. This projection is the
reliability index β. The previous statement can be demonstrated geometrically. Figure 2-9
shows that the vector u∗ is perpendicular to the limit state function g(U ) = 0, because
it follows the shortest path between the origin and the limit state. The vector α is also
perpendicular to g(U ) = 0, because it follows the direction of the gradient ∇g(u∗ ). This
means that u∗ and α lie in the same line, although with opposite signs. Hence, the projection
of u∗ in the direction α is the modulus of the former, i.e. −|u∗ |. From Eq. (2-38), |u∗ | = β,
resulting in the final expression of the probability of failure according to FORM:

p̂f ;FM = Φ(−β) (2-43)

The previous equation literally states the probability of failure is computed from FORM is
the cumulative value for the standard normal distribution corresponding to the distance to
the most probable failure point (β) from the origin (mean-value point). The name reliability
index for β comes from the fact that the higher its value, the higher the reliability:

Master of Science Thesis RESTRICTED Pablo A. Vásconez

24 A review on the reliability methods

R̂e;FM = 1 − p̂f ;FM = 1 − Φ(−β) = Φ(β) (2-44)

u2
g(u*)
Δ

α
g(U) < 0

u* g(U) > 0
β=|u*| g(U) = 0

u1
Figure 2-9: Vectors u∗ and α lie in the same line with opposite signs

A final remark is that the reliability Re and the reliability index β should not be mixed. The
former is the actual probability that a system or component do not fail (hence ranging from 0
to 1) and the latter is the distance to the design point in the u-space (hence ranging from 0 to
∞). As observed in Eq. (2-44), there is a positive correlation between both, if β approximates
to ∞, then Re approximates to 1.

Search of the most probable failure point

In the previous section, Eq. (2-43) indicates that the main target of the FORM calculation
is to find β, which ultimately is to find the most probable failure point u∗ . The purpose
of this section is to show how this target can be achieved. Note that the derivatives of the
performance function g(U ) are used. This already suggest that, when complex models or
definition of failure are evaluated, an analytical solution is not feasible. These kind of models
are of interest for this study, thus a numerical implementation is to be presented.
The underlying algorithm used by FORM in this study, is fully described by Brinkman (2012).
Not being a public document, the procedure is briefly described in this report. Five steps are
required to find the most probable failure point :

1. Linearisation of the performance function g(U ) in uk , where uk is the the starting

location of iteration k. In other words, to build up ĝFM (U ) of Eq. (2-39) at uk instead
of u∗ .

2. Normalisation of the ĝFM (U ) in uk .

3. Estimation of the location of the most probable failure point, based on ĝFM (U )

Pablo A. Vásconez RESTRICTED Master of Science Thesis

2-5 Methods approximating the performance function 25

4. Selection of location uk+1 , the starting point for iteration k + 1.

5. Verification of the convergence of iteration procedure.

In Step 1 the point uk is generally selected arbitrarily and it is known as the start point or start
vector for FORM. The origin (0, 0, . . . , 0) is a common selection. Knowing uk , the performance
function is computed at this point g(uk ). Then, the components of the gradient ∇g(uk ) are
computed by obtaining the values of g(U ) for small positive and negative increments of each
random variable ui , one at a time, having as a reference the point uk .
3

−0
0.05

.1
2

−0
1 .0
5

Δu1
0
Δu2
U1

0
uk
0.1

0.
−1 05

−2

−3 0.1
−3 −2 −1 0 1 2 3
U2

Figure 2-10: Step 1 of the search of the most probable point for two random variables. Colours
are real contours of the performance function. Star represents the mean-values point.

Figure 2-10 shows the selection of the starting vector uk in a point different than the mean-
values point (represented by a star), for a case with two random variables. The point selected
is (0,-2). At distances ∆u1 and ∆u2 realisations of the performance function are taken to
compute the gradient of the performance function. They correspond only to the positive
increments of the random variables. Obtaining also the negative increments, the following
equation is used to compute the gradients (known as the two-sided discretisation method):

k
∂g(U ) g(uk1 , uk2 , . . . , uki + 0.5∆ui , . . . , ukn ) − g(uk1 , uk2 , . . . , uki − 0.5∆ui , . . . , ukn )

≈
∂ui ∆ui
(2-45)
The term on the left-hand represent the values ai in Eq. (2-39), rewritten here as aki to
consider the iteration number. The only unknown for ĝFM (U ) is the coefficient ak0 , obtained
from the expression:

n
X
ak0 = ĝFM (uk ) − aki Ui (2-46)
i=1

Master of Science Thesis RESTRICTED Pablo A. Vásconez

26 A review on the reliability methods

At the end of step 1, the approximation of the performance function is given as:

n
X
ĝFM (U ) = ak0 + aki Ui (2-47)
i=1

In step 2, the normalisation is performed by dividing each term of Eq. (2-47) by the modulus
of the vector ak , which is the same vector as α introduced in the previous section. This gives

n
X
ĝFM (U ) = β k + αik Ui (2-48)
i=1

in which β k = ak0 /|ak | and αik are the components of αk for the iteration k. Note that β k is
the estimation of the reliability index for the iteration k.
In step 3, the design point is estimated using the equation:

u∗k = −β k αk (2-49)

In step 4, the location of uk+1 for the next iteration step is selected. The design point from
Eq. (2-49) could be used as uk+1 . Nevertheless, it may not be a good decision because this
estimation is derived from the approximated performance function rather than the actual.
For this reason, uk+1 is chosen to lie in the same direction towards u∗k is located, but with
a smaller step than the obtained. This is achieved by

uk+1 = ru∗k + (1 − r)uk (2-50)

where r is called the relaxation factor for FORM. Figure 2-11 shows the effect of the relaxation
factor. If a value of r = 1 is chosen, the starting point on the next iteration step would be u∗k ,
located at the extreme of the full line. This may be risky since the gradient at the current
location may be very high and the real contour g(u) = 0 skipped. Therefore, r < 1 is selected
in Figure 2-11 to reduce the size of the step. Note that the same direction is used.
Finally, in step 5 the iterations continues if the following criteria is not accomplish.

|ĝFM (U )| < 1 (2-51)

β k − 2 < u∗k < 1 (2-52)

The values of 1 and 2 are small numbers. Figure 2-12 shows all the iterations made for a
fictional case until the criteria is fulfilled and the design point is obtained.

Pablo A. Vásconez RESTRICTED Master of Science Thesis

2-5 Methods approximating the performance function 27

−2
2 0

uk* −1
1 1

uk+1
3
U1

0
uk
0

−1

1
−2 2
4 3

−3
−3 −2 −1 0 1 2 3
U2

Figure 2-11: Step 4 of the search of the most probable point for two random variables. Colours
are linear contours of the performance function considered by FORM. Star represents the mean-
values point.

g(U)>0
1
u* g(U)<0
U1

−1 g(U)=0

−2

−3
−3 −2 −1 0 1 2 3
U2

Figure 2-12: All iterations for the search of the most probable point for two random variables,
starting in a different point than the mean-values point. Star represents the mean-values point.

Remarks about the name most probable failure point and the geometrical interpretation
of β

So far, the name most probable failure point has been used to identify the point u∗ . If
the procedure to compute u∗ follows the theoretical development of the previous section,
indeed u∗ is the most probable failure point. Nevertheless, the correspondent point x∗ is not
generally the most probable failure point in the x-space. This is noted in (Lemaire, 2010).

Master of Science Thesis RESTRICTED Pablo A. Vásconez

28 A review on the reliability methods

The explanation in the fact that the density fX (x) is obtained in one way or another by
the multiplication of φU (u) and the Jacobian of the transformation ui → xi . For other than
normal distributions, the transformation is not linear producing values for the Jacobian which
vary in the x-space and subsequently producing differences in the shapes of fX (x) and φU (u).
Furthermore, in (Lemaire, 2010) it is argued that the use of the name design point is more
appropriate for this point, which is derived from the philosophy of partial coefficients currently
used for design purposes.
In addition to this, it is worth to mention that in the x-space, x∗ is not always the shortest
distance between the origin (means values, not necessarily 0) and the limit state function
g(X) = 0. When cross-correlation is used between the random variables, the contours of the
joint density fX (x) are elliptical as shown in Figure 2-13. The design point, obtained directly
in the x-space, will lead to the most probable failure point. This corresponds to the point
of tangency between the elliptical contours and the limit state function in the direction that
they increase the most, which depends on the correlation matrix. This direction does not
coincide any more with the perpendicular to g(X) = 0, being β a distance higher than the
shortest in the x-space. The comparison between the distances is depicted in Figure 2-13.
With this information at hand, the most general definition for the reliability index β is the
distance between the mean-value and the design point.
x2

g(X) > 0
x*
g(X) < 0
β
min|x*|

g(X) = 0

x1

Figure 2-13: Comparison between β and the minimum distance min|x∗ | from the mean-values
point to the limit state function. Colours are PDF contours.

Advantages and limitations

FORM is a method that promises a very low calculation effort to compute the probability of
failure. (Lemaire, 2010) affirms that this actually occurs for many practical cases.
Another positive point about FORM is that it provides additional information that may be
of interest for the designer. This information comprises the design point and the directions

Pablo A. Vásconez RESTRICTED Master of Science Thesis

2-6 Summary of the chapter 29

of its location. The design point is a point very close to the most probable failure point.
The direction of its location are indicators of the sensitivity of the random variables on the
response (Karamchandani and Cornell, 1992).
Obviously, it also has it disadvantages. It is identified that it can lead to large errors for cases
with high non-linearity or multiple design points. In addition, since it uses small increments for
each random variable to compute the gradient of the performance function, the computation
time increases with the number of random variables.

2-6 Summary of the chapter

The formal definition of reliability and probability of failure is given. The probability of failure
is simply the integral of the probability density function over the failure domain. However, as
an analytical solution is not feasible for practical problems. Only approximate solutions are
feasible. Numerical integration is the most intuitive solution for this kind of problems. But
the fact that practical problems usually will have more than 3 random variables, make this
method prohibitive.
As an alternative, a method of experimental nature called Monte Carlo can solve the integral
with the greatest accuracy in a considerably smaller computation time that in Numerical
Integration. However, its computation time is still very high for the small probabilities of
failure that are desired. This is even more pronounced when the underlying numerical model
is time consuming (e.g. finite element analysis). On the positive side, the number of model
realisations does depend on the number of random variables for Monte Carlo.
Improvements to Monte Carlo have been proposed by inducing sampling in specific regions or
directions, namely Importance and Directional Sampling. They both are capable of providing
good accuracies. Their disadvantage is that it is not guaranteed that they converge faster than
Monte Carlo. The convergence depends on the shape of the limit state function, implying
that some prior knowledge is usually necessary. In addition, Directional Sampling usually
needs more computation time with increasing amount of random variables.
A different approach is FORM, which attempts to find the location of the most probable failure
point, properly called design point. In this location, it makes a first-order approximation of the
limit state function, being able to compute the probability. A major advantage is efficiency,
only a few realisations of the model is required for cases with medium dimensionality and non-
linearities. Nevertheless, is not suitable for some situations, such as multiple design points or
very high non-linearities, which are less frequent in practical applications.

Master of Science Thesis RESTRICTED Pablo A. Vásconez

 Chapter 3

Response Surfaces on the Reliability

Computation

3-1 Introduction

In the previous chapter, the main reliability methods available in literature and under analysis
in this study are presented. They are classified in two types: simulation methods and methods
approximating the performance function. The former methods have an experimental nature,
whereas the latter have a more formal mathematical basis, but after some relatively strong
simplifications. Both, however, follow a procedure that uses always the performance function
built up directly from the time-consuming deterministic model.
In this chapter, an additional tool to reduce the computation time is introduced. They are
the so-called response surfaces. They are simple mathematical functions which attempt to
replace the use of the time-consuming deterministic model. In this context, the objectives of
the chapter are the following:

• Provide the formal definition of a response surface

• Define procedures to build up a response surface

• Present different alternatives of the use of a response surface available in literature

• Describe the general advantages and limitations of the use of response surfaces regarding
reliability computation.

3-2 Definition of Response Surface

A response surface (RS) is a k-th order polynomial ŷRS (x1 , x2 , . . . , xn ) which approximates
the actual response y(x1 , x2 , . . . , xn ) in a region of interest. For reliability analysis, the k-th

Pablo A. Vásconez RESTRICTED Master of Science Thesis

3-2 Definition of Response Surface 31

order polynomial, named ĝRS (X), attempts to approximate the performance function g(X)
(Chapter 2). The earliest application in this field known by the author is found in (Faravelli,
1989).
In principle, the order k of the polynomial could be any integer value, but in practice it is
common to use either first or second-order polynomials, i.e k ≤ 2. It owes to the exponential
increase of computation time required to build up the response surface as k increases and
the relatively small gain in accuracy compared to the actual response (Rajashekhar and
Ellingwood, 1993).
The mathematical formulation for the first-order and second-order response surface are rele-
vant to define. A first-order response surface with n parameters has the following equation:

n
X
ĝFO;RS (X) = a + bi Xi = a + X T b (3-1)
i=1

where a is a constant and bi are the coefficients of the first-order terms. The third expression
of Eq. (3-1) is the vectorial representation of the response surface, in which the vector b
groups all the coefficients bi .
The second-order response surface with n parameters is represented mathematically by

n
X n
X n X
X n
ĝSO;RS (X) = a + bi Xi + cii Xi2 + cij Xi Xj = a + X T b + X T cX (3-2)
i=1 i=1 i≥j j=1

where cii are the coefficients accompanying the squared second-order terms Xj2 ; and cij are
the coefficients of the so-called cross terms Xi Xj . Coefficients a and bi are exactly the same as
in Eq. (3-1). In the vectorial representation of the second-order response surface, the matrix
c contains all the coefficients of the second-order terms cii and cij /2:
 
c11 c12 /2 · · · c1n /2
 .. 
 . c22 · · · c2n /2 
c= (3-3)
 
.. .. .. .. 

 . . . .


sym · · · · · · cnn

Regardless the selected order for the response surface, its equation is ultimately defined by
the initially undetermined coefficients. Responses using the actual performance function g(x)
at different points x are obtained in order to compute the unknown coefficients in ĝRS (X)
such that the error of approximation is minimized in the region of interest. It should be clear
for the reader that it is unlikely that such an approximation works well outside the region of
interest. It will often work fine for a small region of it. This can be observed in Figure 3-1 and
Figure 3-2 for n = 2 parameters. Both response surfaces are good approximating the center
values of the MATLAB’s built-in function peaks. Nevertheless, as the x values go further
from the center the predictions become very inaccurate.
Some authors, use a second-order response surface without cross terms (Bucher and Bourgund,
1990). The reason is once more to reduce the computation time. Considering the cross terms,

Master of Science Thesis RESTRICTED Pablo A. Vásconez

32 Response Surfaces on the Reliability Computation

the minimum amount of samples to compute the coefficients is (n2 + 3n + 2)/2, whereas that
disregarding the cross terms, the number of samples is just 2n + 1. The latter selection means
that cij in Eq. (3-2) are 0 and, subsequently, matrix c is diagonal.

Figure 3-1: First-order response surface on function peaks from MATLAB

Figure 3-2: Second-order response surface on function peaks from MATLAB

Pablo A. Vásconez RESTRICTED Master of Science Thesis

3-3 Computation of a Response Surface 33

3-3 Computation of a Response Surface

In the previous section the concept of response surface is defined. It can be already intuited
some of the necessary steps to compute the response surface. The steps are explicitly stated
in this section.
The computation of the (undetermined) coefficients of the response surfaces requires the
following steps:

1. Selection of the order of the polynomial

2. Selection of the center of sampling and sampling technique

3. Computation of the coefficients

Only first and second-order response surfaces are addressed in this study. All the other points
are addressed in the next subsections.

3-3-1 Selection of the center of sampling and sampling technique

The center of sampling and the sampling technique together define the region of interest for
the response surface ĝRS (X). The center defines the point in the x-space of more interest for
the prediction. The sampling technique basically defines the size of the region. As is noted in
Section 3-2, the size of the region should be relatively small in order to have a good prediction
of actual response.
Recalling the reliability method FORM Section 2-5-1, it is intuitive that the point of more
interest to predict in the design point, because it is nearby the most probable failure point and
because of this contributes more to the probability of failure. Nevertheless, the design point
is not known a priori. If such a selection is attempted additional effort is required to find
it. For this reason, three different selections for the center of the sampling can be attempted
(Liu and Moses, 1994; Rajashekhar and Ellingwood, 1993):

• No center, meaning that the sampling will be totally random in the whole x-space,
until they complete a sufficiently big amount of samples to compute the coefficients of
the response surfaces. See Figure 3-3(a).

• Mean-values point, which correspond to the mean values of the parameters x. See
Figure 3-3(b)

• Extreme-values point, which correspond to unfavourable values of the parameters x,

for instance, the correspondent 95th-percentile for load or 5th-percentile for resistance.
See Figure 3-3(c)

• The design point, which requires first to obtain an estimation of the design point.
See Figure 3-3(d).

Master of Science Thesis RESTRICTED Pablo A. Vásconez

34 Response Surfaces on the Reliability Computation

u2 u2

u1 u1

(a) No center of sampling (b) Mean-values point

u2
u2
u*

u1 g(U)=0

u1

(c) Extreme-values point (d) Design point

Figure 3-3: Common choices for center of sampling (marked with a cross)

If either the mean or the extreme-values point are selected as the center of sampling, the
calculation can be started by selected a suitable sampling technique. The sampling techniques
that has proven to work well are the ones suggested by the theory of design of experiments
(Lemaire, 2010). The most commonly used experimental designs are:

• Star shape designs: in which the minimum number of samples required to build up the
response surface are taken. In this design, the surface will fit exactly the experimental
points. For first-order RS, the required number of samples is N = n + 1: one sample in
the center of sampling and one sample on the axis of each parameter at a certain distance
d, either in the positive or negative direction. This sampling is shown in Figure 3-4(a)
for 2 random variables. For second-order RS, a usual selection is to fit Eq. (3-2) without
cross-terms. In such a case, only N = 2n + 1 samples are necessary. They are taken in
the center of sampling plus two additional samples on the axis of each parameter at a
distance d, in both the positive or negative direction.

• Factorial designs: in which the samples cover all the possible combinations of the
levels of the parameters. The levels are the representative values that want to be
investigated. The most common of this sampling type is the 2n factorial design. The
base and the exponent represent the levels and the exponent, the number of parameters.
They are named arbitrarily low and high levels. Both the high and low levels are located

Pablo A. Vásconez RESTRICTED Master of Science Thesis

3-3 Computation of a Response Surface 35

at a distance d with respect to the center of sampling.

The left diagram of Figure 3-4(b) shows the samples required for a factorial design
with 2 levels and 2 parameters A and B: low-low, high-low, high-low and high-high.
Note that the interaction between the parameters is considered in this sampling by the
combination ab. For 3 parameters this is even more noticeable, as in the right diagram
of Figure 3-4(b), in which the samples ab, bc and abc represent the interactions between
parameters A, B and C. The total number of samples required is N = 2n .
Regardless the levels used for the factorial design, its applicability is constrained to a
very low amount of parameters. For instance, the 2n factorial design requires N = 1024
samples for 10 parameters, which is already a high percentage of Monte Carlo simulation
(if the final goal is to compute the probability of failure).

• Fractional factorial designs: As it name suggests, they use only a fraction of the
combinations provided by the factorial designs, saving computation time. They are
based on the principle that if several variables are present, a phenomenon would be
driven primarily by some of the variables and low-order interactions. For the common 2n
factorial designs, typical fractions are one half or one quarter. The formal nomenclature
for these specific designs are 2n−1 and 2n−2 , respectively. In Figure 3-4(c), it is observed
that one-half fraction of the complete 23 factorial design (4 out of 8 possible samples)
are selected for the sampling , i.e. combinations a,b,c and abc. More information about
these designs can be found in (Montgomery, 2008).

• Central composite design: which is either a factorial or fractional factorial design

with the addition of combinations in the center of sampling and on the axis of each
parameter. Figure 3-4(d) shows the central composite design for 2 and 3 parameters
with 2 levels of the parameters in each case. This method increases the number of
samples compared to the factorial or fractional factorial, hence it is applicability is also
for a relatively small amount of parameters

The aforementioned distance d, which separates the center of sampling with the low and high
levels of each parameter, is usually expressed in terms of the standard deviation σi of each
random variable. It is very common to use a number between 1 to 2 times σi .
In the case that the design point is selected as the center of sampling, an iterative procedure
needs to be followed. This process is described in more detail in Eq. (3-4) and was proposed
first by (Bucher and Bourgund, 1990). It is anticipated that the basic idea is to use the
mean-value point as the center of sampling to compute an initial response surface, estimate
the design point (using the RS) and choose it as a new center of sampling. In the new center,
any sampling technique can be chosen resulting in the final response surface, centred in an
estimation of the design point.

Master of Science Thesis RESTRICTED Pablo A. Vásconez

36 Response Surfaces on the Reliability Computation

High
High (+)
(+)
High

Variable C
Variable B
(+)

B
d

le
iab
Low Low

r
Low

Va
(-) (-)
(-)

Low High Low High

Variable A Variable A
(-) (+) (-) (+)

(a) Star-shaped design for 2 and 3 variables

bc
abc

b ab
High
(+)
High c
(+) ac
Variable B

d Variable C

b High
ab
(+)
Low
(-)

B
le
(1) a

iab
Low Low

r
(-)

Va
(1) a (-)
Low High
Variable A
(-) (+) Low High
Variable A
(-) (+)

(b) Full factorial design for 2 and 3 variables

abc

High c
(+)
Variable C

b High
(+)
e B
bl

Low Low
ria

(-)
Va

a (-)

Low High
Variable A
(-) (+)

(c) Fractional factorial design for

3 variables
Variable B

Variable C

e B
bl
ria
Va

Variable A

Variable A

(d) Central Composite design for 2 and 3 variables

Figure 3-4: Different experimental designs (or sampling techniques). Adapted from (Mont-
gomery, 2008)

Pablo A. Vásconez RESTRICTED Master of Science Thesis

3-3 Computation of a Response Surface 37

3-3-2 Computation of the coefficients

Once the order of the response surface and the required samples are obtained, the coefficients
can be computed by a multivariate regression technique. The most widely used is the least
squares method, which finds the coefficients that minimize the sum of squares of the error
between the responses obtained using the full model g(X) for the N samples taken and
the responses produced with the resulting response surface ĝRS (X). The background of the
procedure and the final solution are presented in the following paragraphs.
For computing the least squares method, it is necessary to have an expression that relates
the full model with the response surface, which is

g(X) = ĝRS (X) +  (3-4)

where the  represents the expected error between the full model and the response surface.
Note that actually the error is not a constant, it depends on the values for X. This means
that if N samples are imposed in the two models, then N values for the error are obtained.
The expression for this system is:

g(X) = ĝRS (X) +  (3-5)

The size of vectors g(X), ĝRS (X) and  is the number of samples N . Substituting Eq. (3-1)
or Eq. (3-2) a general expression for the system for the previous equation is

gX = W δ +  (3-6)
where in general δ is a vector of the constants a, bi and cij ; and W is a matrix of constant,
linear and quadratic combinations of X. In particular, for the first-order response surface, δ
includes a and b of Eq. (3-1), being the former the first element. The matrix W contains in
its rows vectors X T corresponding to each observation as follows
 
(1) (1) (1)
X T (1)
 
1 X1 X2 ... Xn
(2) (2) (2)
X T (1)   
1 X1 X2 ... Xn

   
W = = (3-7)

..   .. .. .. .. .. 
. .

  
 . . . . 

X T (N ) 1 X1
(N )
X2
(N )
. . . Xn
(N )

For second-order response surface, δ is a rearrange of a, b and c of Eq. (3-2). Likewise, matrix
W is a rearrange of the constant, linear and quadratic combinations of X. Both are given in
the following equations:
h iT
δ= a b1 b2 . . . bn c11 c12 . . . cnn (3-8)

 
(1) (1) (1) 2(1) (1) (1) 2(1)
1 X1 X2 ... Xn X1 X1 X2 ... Xn
 (2) (2) (2) 2(2) (2) (2) 2(2) 
 1 X1 X2 ... Xn X1 X1 X2 ... Xn 
W = (3-9)
 
.. .. .. .. .. .. .. .. .. 

 . . . . . . . . . 

(N ) (N ) (N ) 2(N ) (N ) (N ) 2(N )
1 X1 X2 . . . Xn X1 X1 X2 . . . Xn

Master of Science Thesis RESTRICTED Pablo A. Vásconez

38 Response Surfaces on the Reliability Computation

Returning to Eq. (3-6), it is observed that it is not possible to compute the coefficients in
vector δ unless a condition is established for the vector  that is also unknown. As mentioned
before, this condition is to minimize the error. In the least squares method, it is actually
the modulus of this vector that is minimized, or in other words, to minimize the sum of the
squares of its component. Letting L represent the sum of the squares of vector 

N
X
L = 2i = T  (3-10)
i=1

Solving Eq. (3-6) for  and substituting in the previous equation, the sum of squares can be
represented in terms of δ and W

L = g T g − 2δ T W T g + δ T W T W δ (3-11)

where g is the vector of responses corresponding to the samples, written without the depen-
dency of X (which still exists) for simplicity. Minimizing L implies to satisfy the condition:

∂L 
= −2W T g + 2W T W δe = 0 (3-12)
∂δ e δ

Note that this equation holds for a unique set of coefficients δ,

e which is the solution desired
for this method. Solving for δ, the equation
e

δe = (W T W )−1 W T g (3-13)

allows to compute the coefficients of the response surface either if it is of the first or second
order.

3-4 Response surfaces in the computation of reliability

Until this point, a response surface is defined and also the procedure to compute it. Several
selections that can be made such as the order of the response surface, the center of sampling,
the samples to take and the computation of the coefficients. In addition, the use of a response
surface promises the reduction of computation time. In this section, several alternatives of
its use are described to support this idea.
The summarized methods in this section do not attempt by any means to be a complete
review of all the different proposals made in literature. It rather shows several different ways
in which the response surface can be used in order to substitute the full model and reduces
computation time. These studies show that this is a possible goal.
In Section 3-2, it is mentioned that the first application of the response surface for structural
reliability problem is given in Faravelli (1989). Nevertheless, in the paper the aim is to apply
the response surface to model spatial variability. Bucher and Bourgund (1990) developed
a method which is the basis for all the others that preceded. The proposal is to use the
design point as the center of sampling, but, as indicated in Section 3-3-1, this requires an

Pablo A. Vásconez RESTRICTED Master of Science Thesis

 following analysis the points x, are chosen to be the mean values 2, and x, = 2, _+f,o,, in which
f, is an arbitrary factor ( ~ = 3 is employed in the numerical examples) and % are the standard
deviations. Using the 2n + 1 function values of g(x) at these points, the parameters a, b,, c, are
obtained from a set of linear equations.
It is noted that for cases where g(x) is not sufficiently smooth, e.g. discontinuous, the factors
f, should be chosen in a way that the interpolation points lie on or close to the failure surface
g (3-4
x ) Response
= 0 [5,6].surfaces in the computation of reliability 39
In the next step, the function ~,(x) is used along with the information on xi and o~ to obtain
an estimate of the design point x D. This estimate is based on the assumption of uncorrelated
iterative procedure. In the first iteration, the mean point is used as the initial center and from
Gaussian variables. It is not intended for use as design point but rather to obtain a new center
it, the
point forstar-shaped
the secondexperimental
interpolation.design
Once tox D build enoughg(xD)
is found, samples to build upand
is evaluated a second-order
the new center
response surface. The distance d
point x M for interpolation is chosen on a straight line from the mean vector x 3tostandard
between the samples and the center is selected as x D so that
deviations.
g(x) = 0 ∗fromOnce
lineartheinterpolation,
initial response
i.e., surface is obtained, the first estimation for the design
point x can be made. Since the estimated design point may not be in the limit state exactly,
interpolation is used tog(-~)
find the point for that produces g(X) = 0, which is the new center
xM = x + (XD-- X) g(~) _ g(xo) (3)
xc :
This strategy guarantees that the new center point is sufficiently close to the exact limit state
g(x) = 0. N o w the same interpolation using eqn. (2) isg(µ x)
repeated using x M as new center point.
xc = µx + (x∗ − µx ) ∗)
(3-14)
Hence the total number of g(x)-evaluations is 4n +g(µ ) − g(x
3.xThis procedure is shown schematically in
Fig. 1.
More realisations
The update of thearepolynomial
made aroundRS x~,(x)
c in the samethat
ensures way the
as in the first
critical iteration
domain to build upcovered
is sufficiently the
byfinal response
numerical ĝRS (X). The
experiments procedure
from is shown schematically
the full mechanical model, e.g. FinE Figure 3-5.Once
analysis. In the
~ ( xexample,
) is defined
the distance d for the second iteration is chosen to be a smaller value.

X2 X

"~(x) = o
g(x)=° ~D,~ ~-~.@ g(x) = o
;2

_ I '~ ~' Xl '~D1 l~ - xl

Xl
Fig. 1. Schematic sketch of suggested procedure.
Figure 3-5: Schematic representation of the iteration to center the response surface near the
design point (Bucher and Bourgund, 1990)

With the final response surface, the next step is to use any of the reliability methods presented
in Chapter 2: Monte Carlo, Importance Sampling, Directional Simulation, FORM or SORM.
Bucher suggested Monte Carlo for this purpose. Note that this procedure only requires
N = 4n + 3 realisations of the full model, which makes it a very fast method.
In Rajashekhar and Ellingwood (1993), a slight variation of Bucher response surface approach
is made. The design point is calculated repeatedly (more than once) until a desired tolerance is
achieved for two successive estimations of the design point. Also, cross terms are added to the
response surface. The results are more accurate, but with a huge increase in computational
cost.
A significant variation to the response surface approach appears in Kim and Na (1997), by
the introduction of the gradient projection method to produce sampling very close to the limit
state function. Figure 3-6 (a) represents the initial sampling which is made in exactly the
same manner as Bucher, but using a first-order response surface instead. Figure 3-6 (c) shows
that the sampling for the second iteration is indeed very close to the limit state, which does
not happen in Figure 3-6 (b) that correspond to Bucher approach.
To obtain the sampling points near the limit state function, a unit vector is obtained which
is perpendicular to the gradient of the surface and belongs to the hyperplane ĝRS (X). In
d from the design point is selected for each random variable.
that direction, a certain distance√
In that study is chosen as fi σXi n − 1, being fi a factor between 1 and 1.5. With the new

Master of Science Thesis RESTRICTED Pablo A. Vásconez

 from the limited information on the original limit state. Details will be discussed with numerical
examples.
Advanced Monte Carlo simulation methods for reliability evaluation are required to maintain the
accuracy that is preserved by adopting a higher-order response function. Therefore, the tedious
computing process in Monte Carlo simulation may diminish the advantage of the response surface
method.
40 Response Surfaces on the Reliability Computation

X2
<

%\- XI
(a) Samplingpoints selectedfromthe means
X2

(b) Sampling )ointsselectedfromdesignpoints

X2

~ ~ xl
(c) Samplingpoints selectedby vector projection
Fig. 1. Examplesof varioussamplingpoints; 0 Samplingpoint, - - Responsesurface, Limitstate.
Figure 3-6: Comparison between the experimental design and the vector projected sampling
(Kim and Na, 1997)

samples a new response surface is generated and the process is repeated until the reliability
index β converges according to a required tolerance. The method proves to be very accurate,
but unfortunately no information about the number of realisations of the full model is given.
In any case, it must be several times faster than the Rajashekar approach, due to the use of
first-order response surfaces.
In Das and Zheng (2000), the cumulative response surface method is developed. The key
features of this approach are the improvement of the computation of the unit vector tangent
to the limit state and the introduction of criteria to assess the need of using cross-terms. The
criteria includes the use of samples found in previous iterations, which was not done in any
of the previous approaches. The results showed good accuracies and also were very efficient.
In Gayton et al. (2003), another innovative use of the response surfaces is proposed under the
acronym C2QRS which stands for Complete Quadratic Response Surface with ReSampling.
The main innovation of this method is to consider the design point as a random variable U ∗ .
The distribution for U ∗ is derived from one single experimental design by the use of inferential
statistics and then the mean value for the design point µU ∗ is estimated. These two steps are
important to save computation time. Previous methods, regarded the computed u∗ as the
final design point, which is not robust because it depends on selected experimental design.

Pablo A. Vásconez RESTRICTED Master of Science Thesis

3-4 Response surfaces in the computation of reliability 41

The consideration of U ∗ made in this method is consistent with Eq. (3-6), which shows that
the error is random and being derived from it, the design point is also a random variable.
Figure 3-7 shows schematically the derivation of the empirical distribution for U ∗ from a
prior evaluated experimental design, by means of the jackknife technique (Miller, 1974). In
(a), the observation u(1) is not considered to fit a first response surface, used to obtain the first
estimation of the design point called u e∗(1) . Then in (b), observation u(2) is not considered to
fit a second response surface, used to obtain a second estimation of the design point u e∗(2) . In
(c), after repeating this process with all the observations and empirical distribution for U ∗ is
obtained. Notice that in order to do this, the sample size N should be higher than the required
for building up the response surface. Their suggestion is to use N = (n + 1)(n + 2)/2 + 2 as
sample size.
N. Gayton et al. / Structural Safety 25 (2003) 99–121 107
(a) (b) (c)

Figure 3-7: Derivation by of the empirical distribution for U ∗ using the jackknife technique
(Gayton et al., 2003)
Fig. 4. Step 2—experiments resampling.
Figure 3-8 depicts that the empirical distribution for U ∗ is used to define the re-sampling
area.
5. The
CQ2RS location
step 3:fordesign
the re-sampling falls in within the 90% confidence interval, considering
point estimation
that U ∗ is a Student type variable with N − 1 degrees of freedom. In (b) it is observed
This sampling
that the step is of is
main importance
selected since it consists
as star-shaped. Plotin(c)
determining
shows thetherepetition
experiments
of of
thethe next
jackknife
experimental design, if a new experimental design
∗ is required, or the final estimate
technique to obtain a new distribution for U . This process is repeated until the standard of the design
point, if the level of accuracy is met. Three sub-steps 3.1, 3.2, 3.3 are used.
deviation is smaller than 0.05. The results show good agreement in the reliability index and
the amount of realisations was reasonable: 49, 24, 14 and 40 for 7,2,2 and 6 random variables.
5.1. Sub-step 3.1
In Kaymaz and McMahon (2005),

ð1Þ  the use ofweighted
 regression rather than normal regression
, the mean value me  can be single value



2Þ ðpÞ
From theFor
is suggested. distribution e
u
this, Eq. (3-13) u ðmodified
e
; is ;...; e
u to u

defined, one problem being the unknown accuracy of this value, or another solution consists in
defining it through a confidence interval. T
δe = (W HW )−1 W T Hg (3-15)
Several ways
n exist
o to define a punctual estimate of the mean value, one of them is the arithmetic
where
meanH value
is a n×nue
.diagonal
But it ismatrix of weights.
often more realistic The
and weights aretoselected
interesting provideby
a man equation
 confidence
which
 eu

depends on how close
interval like a< me
the value of g(X) is zero, or in other words, the
 < b rather than m  ¼ c. In our case, the variance of the e limit

state function.
u distribution is
For most of the examplesu
given in the e

upaper, theneaccuracy


o
  obtained was significantly better
u m
eu

clearly unknown. Therefore, the random variable pffiffiffi is a Student type variable with  ¼
Master of Science Thesis RESTRICTED s= p Pablo A. Vásconez
m  1 degrees of freedom according to [16]. The confidence interval with the probability 1a is
defined about the arithmetic mean value using the Student distribution as follows:
 Example of a two variables problem
For a two variable problem, this sub-step is illustrated in Fig. 5. The rectangular confidence
area then becomes the new search domain.

5.2. Sub-step 3.2

42 The next task is then to define a set of new experiments
Response Surfaces
of small sizeon
inthe
the Reliability
new search Computation
domain,
to include them to the previous experimental design, and to restart the method from step 2.
(a) (b) (c)

Figure 3-8: Resampling area derived considering U ∗ as a Student type variable with N − 1
degrees of freedom (Gayton et al., 2003)

Fig. 5. Step 3—design point estimation.

than using normal regression, but in some cases this did not occur. The number of evaluations
is not provided unfortunately. However, since only one iteration is used in this method, it is
expected to be an efficient method.
So far, all the methods have in common the following characteristics:

• They all compute an estimation of the design point, either by interpolation or by re-
gression.

• They all update the response surface, therefore, at least 2 response surfaces are com-
puted in the procedure.

To update the response surface towards the design point appears to be important to define
the performance of a response surface. Nevertheless, as noted in Section 2-5-1, there could be
problems that do not have a unique design point. A method to overcome this problem with
the use of response surfaces is presented by Gupta and Manohar (2004). This method is not
described here because it is out of the scope of this study.

3-5 Advantages and Limitations of the Response Surface

In the previous section was shown that a number of approaches exist with the use of response
surfaces. In the following lines a summary of the advantages and limitations of the different
approaches is given in a general manner. Some of them are taken from (Lemaire, 2010)
Advantages:

• The number of realisations of the full model is significantly reduced

Pablo A. Vásconez RESTRICTED Master of Science Thesis

3-6 Summary of the chapter 43

• The numerical design of experiments can include the expertise of the designer
• Possibility of reusing previous realisations to make new designs

Limitations:

• The location of the center of sampling and the sampling technique to build up the
response surface may influence the results significantly.
• Low number of realisations limit the available responses for the performance of the
regression
• Absence of a convergence rule as a function of the number of realisations
• Most of the response surface approaches are not able to deal with multiple design points
problem.
• The number of computation increase with the number of random variables

3-6 Summary of the chapter

The role of the response surface in the reliability computation has been presented in this
chapter. First, the response surface is defined as a k-order polynomial which attempts to
approximate the actual response of a model in a region of interest. The polynomials used in
practice are of first and second-order, to save computation time.
The response surface requires several realisations of the geomechanical model in order to be
built. For this, sampling points are defined by selecting a center of sampling and a sampling
technique. The center of sampling ideally should be the so-called design point, but this is
an impossible choice a priori. Hence, mean values are the most common choice, but for the
computation of an initial response surface only. A sampling is used according to the theory
of experimental design. Once the responses are available, the undetermined coefficients of the
response surface are computed by multivariate regression.
In the available methods in literature, the response surface is updated. This means that more
than one response surface is computed. For instance, an initial response surface is used to
estimate the design point by applying FORM. The design point is used to select new samples
and to build a new response surface. This new response surface could be accepted or the
process could be repeated until the change in the reliability index or the design point is
smaller than a given tolerance. The reliability can be computed by means of any reliability
method using exclusively the response surface, being Monte Carlo the best option.
Many alternatives of the response surface approach are proposed in literature. The differences
include the selection of the experimental design, the computation of the design point by
regression or interpolation or the use of a weighted rather than normal regression.
The different alternatives have proven that, indeed, the response surface provides savings in
computation time. Nevertheless, in general the response surfaces have several limitations: the
dependency with the existence of a unique design point, the absence of a convergence rule
according to the number of samples and the increase of computation time with the number
of random variables.

Master of Science Thesis RESTRICTED Pablo A. Vásconez

 Chapter 4

Workflows under Evaluation

4-1 Introduction

The computation of the reliability requires several ingredients. The first is the performance
function, an explicit or implicit equation that indicates if failure occurs. This equation in-
cludes the relationship between the uncertain parameters and the response of interest, which
is ultimately analysed to determine if failure occurs or not. With the performance func-
tion defined at least conceptually, the next step is simply to compute the integral under the
probability distribution function in the region of failure. For this, a significant number of
methods are available. The question is which method to choose in order to provide accurate
and efficient results.
The intention of this chapter is to define the methods or workflows that are evaluated in
this thesis. First, the specific performance functions that are used in the workflows for
the evaluations of the subsequent chapters are derived, based on the particular definition
of failure for the examples under study. Then, the response surfaces, reliability methods and
the complete workflows are presented, highlighting any important difference with the studied
in Chapter 2 and Chapter 3.
Formally, the objectives of the chapter are the following:

1. Describe the workflows to be evaluated

2. Define the performance function for the problems under evaluation
3. Provide details about the response surfaces and the reliability methods that are part of
the workflows under evaluation

4-2 Definition of the specific performance functions for this study

As defined in Section 2-2 and stated specifically in the next section (Step A), the computation
of the reliability requires the definition of the performance function. This, in turn, depends

Pablo A. Vásconez RESTRICTED Master of Science Thesis

4-2 Definition of the specific performance functions for this study 45

on the definition of failure, which is usually not straight forward in some geotechnical appli-
cations and even less in reservoir engineering. This section explains what is the consideration
for failure in this study, arriving finally to the performance functions used in the workflow
evaluations of the subsequent chapters.
The definition of failure is connected to a specific activity under assessment. The focus activity
in this study is the depletion of a reservoir. Regarding this activity, there are several failure
mechanisms (Table 4-1) of concern for the geotechnical engineer, which can be group in 4
failure types:

1. Subsidence, occurrence of local or regional settlement of the ground larger than a

allowable value for the platform or the surroundings, respectively.

2. Formation failure, occurrence of a strain higher than the correspondent to reach

the peak stress of the formation surrounding the reservoir. The failure mechanisms
considered are in shear, tensile and compressive stresses. Each of them constitutes a
failure mechanism by itself. The peak shear stress is computed by a failure criterion,
e.g. Mohr-Coulomb, Hoek-Brown, etc.
In addition, the formation failure type includes the occurrence of a strain higher than
a certain limit (lower or higher that the one to reach the peak stress). Extension or
compaction are the failure mechanisms that belong to this definition.

3. Well failure, occurrence of a strain higher than the correspondent for the peak shear
stress of the casing material. Also, a limit for the axial elongation or compression strain
may be set, which may be lower or higher than the strain for the peak shear stress.

4. Fault failure, occurrence of a relative displacement between the two sides of the for-
mations separated by a fault, which is higher than the correspondent for the peak shear
stress of the fault. The peak shear stress of the fault is computed by a failure criterion,
e.g. Mohr-Coulomb.

Table 4-1: Failure mechanisms for each failure type

Failure Type Failure Mechanism

Subsidence Subsidence

Formation Shear
Tension
Compression
Extension
Compaction

Well Axial elongation

Axial compression
Shear

Fault Shear

Master of Science Thesis RESTRICTED Pablo A. Vásconez

46 Workflows under Evaluation

While the aforementioned failure types allow to focus the attention of the usual geomechanical
issues regarding reservoirs, they are not sufficient to compute the reliability. The reason is
that exceeding in one single material point, the maximum limit of either settlement or strain
correspondent to the peak stress, may not imply the inability of a geotechnical structure to
continue to serve its purpose. There is the need of defining operational failure rather than
a physical one. In order to help this assessment, but certainly not to be the final word, 4
indicators are used to define failure:

1. Maximum value of settlement, strain or stress for the region of the continuum and
for the failure mechanism of interest.
2. Average value of settlement, strain or stress for the region of the continuum and for
the failure mechanism of interest.
3. Failure area, i.e. the size of the subregion for which the limit value of settlement,
strain or stress is exceeded (for the failure mechanism of interest).
4. Percentage of the failure area, i.e. the ratio between the failure area and the region
of interest for a certain failure mechanism.

The thesis focuses on the aforementioned failure indicators for the problems of subsidence
and formation shear only. Figure 4-1 and Figure 4-2 show how the failure indicators look like
for the depletion of a reservoir. Figure 4-1 depicts a typical change in pressure vs. defor-
mation curve along the surface. The maximum in Figure 4-1(a) and the average subsidence
in Figure 4-1(b) are highlighted by the horizontal line. The failure area indicator results by
the computation of the shaded area on the surface in Figure 4-1(c). Note that a limit value
(represented by the dashed line) of subsidence needs to be defined in order to compute the
area that fails. The percentage of failure area results by the ratio of the failure area and the
total area under consideration. Both are shaded in Figure 4-1(d), being the failure area the
darker shade.
Before introducing the failure indicators for shear failure depicted in Figure 4-2, it is necessary
to introduced the concept of shear capacity utilization SCU. In this study, it is defined as
the ratio between the shear stress and peak stress in a given point. The shear strength is
computed according to the Mohr-Coulomb model. In an elasto-plastic geomechanical model
(GM), the shear strength could never be surpassed. But given that the models in the present
work are elastic only, the stress can reach infinite values. Hence, the SCU reports how much
shear capacity is available given a region of the continuum. Therefore, this does not consider
the redistribution of stresses that the plastic behaviour of the soil presents.
In this context, Figure 4-2(a) presents the maximum SCU. The dotted line shows a diagram
SCU vs. distance from the corner to the reservoir in a path oriented 60o with the horizontal.
The maximum SCU in this case occurs in the corner, due to stress concentration. The average
SCU is shown in Figure 4-2(b). The failure area SCU results from the computation of the
shaded area in Figure 4-2(c), which has a tubular shape because of the reduction of the stress
with distance from the corner of the reservoir. The percentage of failure area results by the
computation of the ratio between the shaded areas in Figure 4-2(d)
It is now possible to determine implicitly the performance function for the failure indicators
under evaluation. Following the notation of Eq. (2-1), the actual values of the failure indicators

Pablo A. Vásconez RESTRICTED Master of Science Thesis

4-2 Definition of the specific performance functions for this study 47

Surface Surface

Average
Maximum

Reservoir Reservoir

(a) Maximum Subsidence (b) Average Subsidence

Surface Surface
Percentage of Failure Area
Failure Area

Limit Limit

Reservoir Reservoir

(c) Failure Area Subsidence (d) Percentage of Failure Area Subsidence

Figure 4-1: Failure Indicators for Subsidence

Master of Science Thesis RESTRICTED Pablo A. Vásconez

48 Workflows under Evaluation

Surface Surface

Maximum
Average

Reservoir Reservoir

(a) Maximum SCU (b) Average SCU

Surface Surface Percentage of

Failure Area Failure Area

it
it

Lim
Lim

Reservoir Reservoir

(c) Failure Area SCU (d) Percentage of Failure Area SCU

Figure 4-2: Failure Indicators for Formation Shear Capacity Utilization (SCU)

Pablo A. Vásconez RESTRICTED Master of Science Thesis

4-3 General description of the workflows under evaluation 49

computed in Figure 4-2 and Figure 4-1 are named by L(x). The limit value in order to
consider them as failure is Rmax . Therefore, taking as maximum subsidence as example the
performance function is

g(x) = Rmax;sub − Lmax;sub (x) (4-1)

The same logic is applied to build up the performance functions in Table 4-2. Note that for
failure area and percentage of failure area, the value of L also depends on a limit value lim.
If this limit is changed, also the performance function changes.

Table 4-2: Performance functions for Subsidence and Formation Shear Failure

Failure Indicator Performance function

Subsidence
Maximum Subsidence g(x) = Rmax;sub − Lmax;sub (x)
Average Subsidence g(x) = Rave;sub − Lave;sub (x)
Failure Area Subsidence g(x) = Rarea;sub − Larea;sub (x, limsub )
Percentage Area Subsidence g(x) = Rper;sub − Lper;sub (x, limsub )

Formation Shear Failure

Maximum SCU g(x) = Rmax;SCU − Lmax;SCU (x)
Average SCU g(x) = Rave;SCU − Lave;SCU (x)
Failure Area SCU g(x) = Rarea;SCU − Larea;SCU (x, limSCU )
Percentage Area SCU g(x) = Rper;SCU − Lper;SCU (x, limSCU )

4-3 General description of the workflows under evaluation

In this section, a description of the workflows that are evaluated in the coming chapters is
given. First, a general classification of the workflows is introduced. Then, one of the group
of workflows is described in more detail due to the complexity of the procedure.
There are many specific workflows evaluated in this study. They can be classified in three
groups of workflows:

1. Workflows using a Response Surface only, Res: Perform a reliability method us-
ing exclusively a response surface to calculate the performance function, which actually
is an approximation of it, i.e. ĝ(x). The response surface is never updated during the
reliability computation.

2. Workflows using a Response Surface with updates according to a threshold,

Rit: Performs a reliability method using both a response surface and the geomechanical
model to calculate the performance function. The use of the response surface or the
geomechanical model depends on the threshold range. This is a range of performance
function values for which ĝ(x) is not accepted. If ĝ(x) falls within the threshold range,
the geomechanical model is used to compute the actual performance function g(x).

Master of Science Thesis RESTRICTED Pablo A. Vásconez

50 Workflows under Evaluation

Since a new value of the actual response is available, the response surface is updated
by regression and the threshold range is reduced. The threshold range is defined by the
expression |g(x)| < th , where th is the current threshold. When the threshold range is
updated, the new threshold is th = γt |ĝ(x)|, where γt is called the threshold relaxation
factor, taken arbitrarily as 2/3. The response surface is updated and the threshold
reduced until the end of the reliability method.

3. Workflows using the Geomechanical Model only, Mod: Performs a reliability

method using exclusively the geomechanical model to calculate the performance function
g(x).

Note that in each workflow group it is possible to define a response surface and a reliability
method of preference. The choices for this thesis are presented in Section 4-5. It is appropriate
to present a flow chart to show the steps required. This is done in Figure 4-3. As observed,
basically 5 generic steps can be described as components of the workflow (for Mod only steps
A, D and E apply):

• Step A: General inputs for the program, including: failure mechanism; geomechanical
engine to use to compute responses; geometry and material properties of the subsurface;
the probabilistic characteristics: distribution, parameters and correlation.

• Step B: Selection of a response surface type. This means that the shape of the response
surface is selected to be fitted through the data, but the specific coefficients are computed
by regression from scenarios hel. Five response surface types are available as discuss
later.

• Step C: Computation of the response surface. This includes the generation of scenarios,
the computation of the correspondent responses by the geomechanical engine and the
data fitting to build up the response surface.

• Step D: Input for the reliability, including: the limit value, the reliability method and
the required settings for its performance. Four reliability methods are implemented as
discussed later.

• Step E: Reliability computation. In general terms, in this step the computations strictly
follow a reliability method. Large difference in computation time is seen in this step
since each workflow includes no use, partial use or full use of the geomechanical model.
The final output is the probability of failure and thus the reliability.

An important remark about the group of workflows Res and Rit, is that they are not using
the response surfaces as proposed in literature (Section 3-4). Workflows Res and Rit lack the
step of computing the design point x. Rit offers an alternative to this, stimulating samples
nearby the limit state function g(x) = 0 in general, thanks to the use of the threshold. Res
does not offer alternative and hence it is questionable that it will provide good accuracies of
the probability of failure, given that the response surface is never updated.

Pablo A. Vásconez RESTRICTED Master of Science Thesis

4-3 General description of the workflows under evaluation 51

A.LGeneralLInputs
FailureLMechanism

GeomechanicalLEngine

BaseLCase

ProbabilisticLPropertiesL

Res Rit Mod

B.LSelectionLofLaLresponseLsurfaceLtype
ResponseLSurfaceLtype ResponseLSurfaceLtype

C.LComputationLofLtheLresponseLsurface
ScenarioLgeneration ScenarioLgeneration

ResponsesLfromLGML ResponsesLfromLGML

CoefficientsLcomputation CoefficientsLcomputation

ResponseLSurfaceLfunction ResponseLSurfaceLfunction

D.LInputsLforLtheLreliabilityLmethods

LimitLvalue LimitLvalue LimitLvalue

ReliabiltyLmethodLandL ReliabiltyLmethodLandL ReliabiltyLmethodLandL

settings settings settings

Threshold

E.LReliabilityLcomputation
PerformanceLfunction PerformanceLfunction PerformanceLfunction

ReliabilityLcomputationL ReliabilityLcomputationL
ReliabilityLcomputation (useLofLGM)

ReliabilityLvalue |PerformanceLfunction|L<LThreshold ReliabilityLvalue

yes no

NewLresponseLsurface KeepLresponseL
NewLThreshold surface
(useLofLGM)

ReliabilityLvalue

Figure 4-3: Diagrams representing the proposed workflows in the study: Res, Rit and Mod.
Colors indicate inputs (yellow), process (red) and outputs (green). GM = geomechanical model

Master of Science Thesis RESTRICTED Pablo A. Vásconez

52 Workflows under Evaluation

4-4 Graphical description of workflows using a Response Surface

with updates Rit

In the previous section, the groups of workflows under evaluation described in a general
manner. The author considers that no problem is understanding the workflows Res and Mod
should arise. However, worfklows Rit deserves further explanation given its complexity and
also that it has not been described previously in literature. The credits for this development
correspond to Deltares, company which suggested the application of this procedure to the
author.
A graphical explanation is given starting in Figure 4-4, considering a fictional case with only
one random variable x1 . The actual response surface L(x), the initial response surface L̂(x),
the limit value R and the threshold range are shown in the mentioned figure. The limit value
R is equal to 8. The threshold range is defined by the lines th;range;min and th;range;min , which
are located in R + 3 and R − 3. This means that the initial threshold is th;ini = 3. In other
words, the initial threshold range is initially expressed as |ĝ(x)| < 3.

25

20

15 Response Surface
L(x1)

Actual Response
10 R (lim.val.)
th;range;min
5
th;range;max

0
0 2 4 6 8 10
x1

Figure 4-4: Presentation of the threshold range in a fictional case

Suppose that the reliability method starts. Some samples are taken according to the density
function of x1 . As long as the samples fall outside the threshold range, ĝ(x) is computed.
This can be observed in Figure 4-5, where the crosses correspond to the response surface
rather than the actual response. But, when a sample produces a value inside the threshold
range, i.e. ĝ(x) < 3 an updating procedure is followed.
The updating procedure for the workflows Rit includes the computation of a new response
surface and the reduction of the threshold range. The whole procedure for one update is
depicted in Figure 4-6. First, the realisation of the response surface produces a value inside
the threshold range, for instance ĝ(x) = 1.84, presented with a cross symbol in Figure 4-
6(a). Second, this result is not accepted and for the same value of x1 the actual response
is computed. This means that the dot in Figure 4-6(b) is a new response available; the
knowledge of the actual response towards failure is increased. Third, taking advantage of
the new information, a new response surface is computed by regression considering all the
known responses so far. Figure 4-6(c) shows that the previous responses (squares) and the
new response (dot) are considered to build up the new response surface represented by the

Pablo A. Vásconez RESTRICTED Master of Science Thesis

4-4 Graphical description of workflows using a Response Surface with updates Rit 53

25

Actual Response
20

R (lim.val.)
15
L(x1)

Response Surface
10

Point outside
5 threshold range
PDF*50
0
0 2 4 6 8 10
x1

Figure 4-5: Sampling falling outside the threshold range. Only the response surface is used.

darker straight line. An important observation is that the new response surface improved
the prediction of the actual response towards the limit state, i.e. the line R = 8, but it also
diminished the prediction for low values of x1 . This is a favourable event for what reliability
concerns. Fourth, the threshold range is reduced as shown in Figure 4-6(d). As mentioned
earlier the reduction is calculated as th = (2/3)|ĝ(x)|. For this example, the new threshold is
th = (2/3)|1.84| = 1.23. The new threshold range is |ĝ(x)| < 1.23.
The reliability method continues with the use of the updated response surface and threshold
range. If again, the a certain value of x1 produces a response inside the threshold range, then
a new update is required. The result of this in shown in Figure 4-7. Note that the threshold
range is very small after a certain number of updates. It is likely that no further updates are
performed and the final response surface is used until the end of the reliability method.

Master of Science Thesis RESTRICTED Pablo A. Vásconez

54 Workflows under Evaluation

25 25

Actual Respon
20 20
Actual Response
15 15 R (lim.val.)

L(x1)
L(x1)

R (lim.val.)
10 10 Response Surf
Response Surface

5 5
Point inside New realizatio
threshold range actual respons
0 0
0 2 4 6 8 10 0 2 4 6 8 10
x1 x1
(a) A sample falls inside the threshold range (cross) (b) New realisation of the actual response (dot)

25
25
Actual Response
20
20
R (lim.val.)
15 15
L(x1)

Response Surface
L(x1)

Actual Res
10 10 R (lim.val.)
New realization of
actual response Response S
5 5
New Response Surface
0 0
0 2 4 6 8 10 0 Previous
2 realizations
4 of 6 8 10
x1 actual responses x1
(c) A new response surface is computed (d) The threshold range is reduced

Figure 4-6: Updating procedure of response surface and threshold when a samples falls inside
the threshold range. Legend from Figure 4-4 is applicable

25

20

15
L(x1)

Actual Response
10 R (lim.val.)
Response Surface (3)
5

0
0 2 4 6 8 10
x1

Figure 4-7: Next update of the response surface and threshold

Pablo A. Vásconez RESTRICTED Master of Science Thesis

4-5 Response Surfaces used in the workflows 55

4-5 Response Surfaces used in the workflows

In this section, the response surfaces used in the workflows under evaluation are described in
further detail.
Recalling Chapter 3, the computation of a response surface requires to define: the order k of
the polynomial, the center of sampling and a sampling technique. In this study, the center of
sampling is always the point corresponding to the mean values of the random variables. The
distance d with the sampling is always equal to 1.28 times the equivalent standard deviation
of each random variables in the u-space, i.e. d = 1.28σi . The reason for 1.28 is because it
correspond to the percentiles P10 and P90 of the distribution, once it is in transformed to
the standard normal space as shown in Figure 4-8.

0.45
P50
0.4
0.35
0.3
0.25
PDF

0.2
P10 P90
0.15
0.1
0.05
0
-6 -4 -2 0 2 4 6
x1

Figure 4-8: Sampling distance is d = 1.28σi , represented by selection of percentiles P10 and
P90 and low and high values of the random variables

The response surfaces, however, differ in the order k of the polynomial and the experimental
design used. Five response surfaces are used in this thesis:

• First-Order 10 (FO10): as it name indicates, the order of the polynomial k is 1.

Hence, it is a hyperplane fitted to the responses obtained by sampling the percentile
P10 value of the distribution of each random variable (keeping the other in the mean)
and also one sample for the mean-values point P50. The sampling is shown in Figure 4-
9(a) for two variables.

• First-Order 90 (FO90): also with an order k = 1 for the polynomial. The design
is also a half star, but sampling the percentile P90 of the distribution of each random
variable (keeping the other in the mean) and also one sample for the mean-values point
P50. This is shown in Figure 4-9(b) for two random variables.

• First-Order Double (FOD):a composite surface made from 2n hyperplanes (each

order k = 1). The responses are obtained by sampling the percentile P10 and P90 of
each random variable. An additional sample at the mean-values point is also taken.
Hence, the full star-star shaped design described in Section 3-3-1 and shown for two
random variables in Figure 4-9(c). With this sampling, 2 gradients are computed for

Master of Science Thesis RESTRICTED Pablo A. Vásconez

56 Workflows under Evaluation

each random variable: one for lower values than P50 and one for higher values than
P50.

• Second-Order (SO): a quadratic hypersurface, i.e. a polynomial of order k = 2. The

sampling is done according the 2n factorial design (Section 3-3-1) considering P10 and
P90 as the low and high values of each random variable. In addition, the mean-values
point is sample. All the samples are shown in Figure 4-9(d). Despite taking some many
samples, no coefficients for the cross terms are computed (see Eq. (3-2)).

• Experimental Design (ED): It is a first-order response surface (k = 1) which uses a

special design of experiment called a two-level fractional factorial design of Resolution
III, i.e. 2n−p
III . The letter p represents the fraction to be taken as explained in Section 3-
3-1) and is selected as the biggest integer for which 2n−p > n holds. This guarantees
that always there would be at least one more responses than the number of variables,
which allows to fit a first-order polynomial. More about this design can be found in
(Montgomery, 2008). As for the other response surfaces, the low and high values for
the response are P10 and P90 and also an additional sample is taken for the mean-
values point. All the samples taken for a case of two random variables are shown in
Figure 4-9(e).

After the samples are obtained, the multivariate regression presented in Eq. (3-13) is used
to compute the coefficients. For the initial response surface, a perfect fit results for response
surfaces FO10, FO90 and FOD. But if one of the workflows Rit is used, the regression is used
anyway to account for the new available samples.

4-6 Reliability Methods used in the workflows

The reliability methods under evaluation in this study are the following:

• Monte Carlo (MC): used exactly as described in Section 2-4-1.

• Importance Sampling (IS): the alternative of the increased variance is used, as

described in Section 2-4-2.

• Directional Simulation (DS): the procedure described inSection 2-4-3 is used, but
with an additional consideration when failure is not found (explained below).

• First-Order Reliability Method (FORM): used exactly as described in Section 2-

5-1.

For Directional Simulation, the linear search presented was shown for the case failure is found.
However, there will be directions in which failure is actually not found. In such a case, the
direction α is discretised in 20 intervals. The size of the intervals are ∆u = 1. This is shown
in Figure 4-10. Hence, if failure is not found in many directions, the method may be time
consuming.
In a direction α(i) , a total of 20 realisations of the performance function are made. This
implies that the method may be time consuming.

Pablo A. Vásconez RESTRICTED Master of Science Thesis

4-6 Reliability Methods used in the workflows 57

u2 u2

P50,P90

P50,P50 P50,P50
P10,P50 u1 P90,P50 u1

P50,P10

(a) First-Order 10 (FO10) (b) First-Order 90 (FO90)

u2 u2

P50,P90

P10,P90 P90,P90

P50,P50 P50,P50
P10,P50 P90,P50 u1 u1

P50,P10 P10,P10 P90,P10

(c) First-Order Double (FOD) (d) Second-Order (SO)

u2

P10,P90 P90,P90

P50,P50
u1

P10,P10 P90,P10

(e) Experimental Design (ED)

Figure 4-9: Sampling for the different response surfaces under evaluation, for a case with 2
random variables

Master of Science Thesis RESTRICTED Pablo A. Vásconez

58 Workflows under Evaluation

u2

α(i)
19 20

g(U) > 0
2 failure is not found
1
in this direction
g(U) < 0
u1

g(U) = 0

Figure 4-10: Discretisation of the direction α(i) in 20 intervals if failure is not found during the
linear search method

A remark on the reliability methods is that they need some settings from the user in order
to operate (as written on Step D, Figure 4-3). The reliability parameters are summarized in
Table 4-3. The default values for the reliability parameters used in this thesis (unless stated
explicitly the use of others) are shown in Table 4-4.

Pablo A. Vásconez RESTRICTED Master of Science Thesis

4-6 Reliability Methods used in the workflows 59

Table 4-3: Parameters of each reliability method

Reliability Method Parameter Description

Numerical Integration Umin, Umax Minimum and maximum value for u

Intervals Defines the discretisation of the
domain in the u-space

Crude Monte Carlo Seed

Minimum Samples
Maximum Samples
Variation Coefficient Failure Coefficient of Variation of the
probability of failure
Variation Coefficient No Failure Coefficient of Variation
reliability

Directional Simulation All MC parameters

Importance Sampling Variance Factor A factor to increase the

standard deviation of the
random variables
All MC parameters

FORM Maximum Iterations

Start Method Fast inputs to set the start vector
Start Vector User-defined value for the start vector
Relaxation Factor A value to control the step
of the iterations

Master of Science Thesis RESTRICTED Pablo A. Vásconez

60 Workflows under Evaluation

Table 4-4: Default Values for the parameters of the reliability methods used in this study

Method Reliability Parameter Default Value

Numerical Integration Umin, Umax -5, 5

Intervals 100

Crude Monte Carlo Seed 1

Minimum Samples 1000
Maximum Samples 100000
Variation Coefficient Failure 0.1
Variation Coefficient No Failure 0.1

Directional Simulation Minimum Samples 10

All other MC parameters

Importance Sampling Variance Factor 1.5

All other MC parameters

FORM Maximum Iterations 50

Start Method Start Zero
Start Vector (0,0. . . )
Relaxation Factor 0.75

Pablo A. Vásconez RESTRICTED Master of Science Thesis

4-7 Summary of the chapter 61

4-7 Summary of the chapter

The methods or workflows that are evaluated in this thesis are presented. A complete view
of the workflows clearly shows that 3 to 5 steps are needed in order to compute the reliability
of a structure:

• Step A: General Inputs to arrive to the specific definition of performance function

• Step B: Selection of a response surface type

• Step C: Computation of the response surface function

• Step D: Inputs for the reliability methods

• Step E: Formal start of the reliability method

The specific definition of performance function are presented explicitly, the definition of fail-
ure that is considered for the examples under analysis in this study. Two types of failure
mechanism are of interest: subsidence and formation shear failure. For each of them, there
are 4 important failure indicators, depending on the needs of a project: maximum value,
average value, failure area and percentage of failure area. Each of them, are analysed in the
following chapters.
Depending on the procedure follow between Steps B to E, the workflows under evaluation are
classified in three groups:

• Workflow Res performs a reliability method using exclusively a response surface (which
is never updated) to estimate the values of the performance function; the response
surface is never updated.

• Workflow Rit performs a reliability method using the response surface to estimate the
performance function, with the difference that the response surface is updated during the
execution of the reliability method. A threshold is used to defined a range that includes
the limit state, for which the estimation of the performance function is not accepted.
If this occurs, the geomechanical model is used to compute the actual performance
function, providing an additional point to update, by regression, the response surface.

• Workflow Mod performs a reliability method using exclusively the geomechanical model
to calculate the performance function.

Detailed information is provided about the response surfaces and reliability methods used in
all groups of workflows (with the exception of workflows Mod that does not use a response
surface). The response surface types include a specific order for the polynomial and sampling
technique and are named as: First-Order 10 (FO10), First-Order 90 (FO90), First-Order
Double (FOD), Second-Order (SO) and Experimental Design. The reliability methods are
the explained in Chapter 2: Monte Carlo, Importance Sampling and Directional Sampling.

Master of Science Thesis RESTRICTED Pablo A. Vásconez

 Chapter 5

Evaluation of Workflows for

Quasi-Linear Responses

5-1 Introduction

The investigation of the performance of the workflows is started for examples that are con-
sidered to have a quasi-linear responses, or in other words, cases in which there limited
non-linearity present in most of the domain for the random variables. This depends on the
inherent degree of non-linearity of a response with respect to the parameters, but also to the
magnitude of the variances assigned to them. These types of problems are evaluated first,
because they serve to identify the workflows with good potential to compute the reliability
that deserve further research.
The failure mechanisms of subsidence and shear failure of a reservoir due to the depletion of
a reservoir for a homogeneous subsurface can be classified as quasi-linear examples, as long as
the geomechanical model Geertsma (Chapter A) is used. Therefore, this examples are used
for the evaluations.
Initially, the terms of the evaluation are explicitly defined, in order to avoid ambiguity. Recall
that the thesis intends primarily to identify efficient methods that can make feasible the incor-
poration of reliability analysis to common geotechnical practice. For this, efficient methods
needs to be efficient, but also accurate. This terms, accuracy and efficiency is what is defined
to start with.
The first evaluation, which is the only including all the research workflows, is made exclusively
to the failure indicator of Maximum Subsidence. It is aimed to avoid the unnecessary evalu-
ation of workflows that do not have potential to compute the reliability for all the examples.
Therefore, only the workflows with good potential for Maximum Subsidence are used in the
whole study.
In addition to the discrimination of workflows with or without potential to compute the
reliability accurately and efficiently, explanations are given to support the success or the

Pablo A. Vásconez RESTRICTED Master of Science Thesis

5-2 Definition of efficient workflows 63

failure of them. The aim is to be able to recommend improvements to the procedures that
may lead to increase either their accuracy or efficiency.
Formally, the objectives for this chapter are:

1. Define criteria to consider a method as efficient.

2. Determine the methodology to evaluate the proposed workflows in terms of efficiency.

3. Evaluate all the workflows for an initial example.

4. Identification of the workflows with good potential to compute the reliability.

5-2 Definition of efficient workflows

The ideal workflow in terms of efficiency is the one which could produce an exact value of
reliability taking an infinitely small time to compute it. However, the assumptions about
the reliability methods and response surfaces, discussed in Chapter 2 and Chapter 3, make
impossible to obtain the exact value of the reliability (except of course for very specific cases).
A loss in accuracy must be tolerated. In general, this loss is negatively correlated with the
computation time.
The previous paragraph reveals that two important factors must be addressed about the
prediction of a particular method: the accuracy on the prediction and the computation time.
Accuracy is defined as the degree of closeness of a prediction to the exact value. Given in
practice small values of probability of failure are searched, a strict but logical definition of
accuracy must be given in terms of probability of failure, rather than reliability. In this study,
the accuracy of the probability of failure Apf is given by
  !
 f exact − p̂f 
p 
Ap̂f = max 1 −  ,0 (5-1)
 pf
exact


in which p̂f is the estimated probability of failure obtained by a particular workflow. With
this definition, the accuracy can be any real value from 0 to 1. A value of 1 means that the
prediction is exact, p̂f = pf exact . In order to understand what a value of 0 means, first note
that the subtracting term of Eq. (5-1) is the relative error of the prediction with respect to
the exact value. The accuracy takes a value of 0 when the relative error is equal to 1 or,
in other words, a difference between the prediction of the exact value at least equal to the
magnitude of the exact value. Bigger differences would be considered as an accuracy of zero
as well.
The computation time literally is the time that a computer takes in order to carry out a specific
procedure. This depends on the amount of the floating-point operations that the procedure
contains and the characteristics of the processor (or processors) used by the computer. Since
the characteristics of the processors may vary for different reasons, it is more meaningful to
represent the computation time by exclusively the amount of floating-point operations.
The amount of floating-point operations depend on both the probabilistic software and the
geomechanical model. The amount of floating-point operations made in the probabilistic

Master of Science Thesis RESTRICTED Pablo A. Vásconez

 Evaluation of Workflows for
64 Quasi-Linear Responses

software remain in the same order, regardless the geomechanical model used. The amount
of floating-point operations of the geomechanical engine may change significantly depending
on the level of sophistication considered. Compared to the geomechanical models used in
practice (and including the engines presented in Chapter A), the amount of floating-point
operations made by the probabilistic software are negligible. Each model realization takes in
average a similar amount of floating-point operations. Therefore, a good representation of
the computation time can be made exclusively by the number of model realizations.
Having all these considerations at hand, an efficient method is defined as the one that uses
a small number of geomechanical model realisations for a required level of accuracy. In this
thesis, the exact value is considered to be equal to the one computed by Monte Carlo using
exclusively the geomechanical model.

5-3 Methodology for the evaluation

In order to investigate the performance of the workflows for quasi-linear problems, two stages
of evaluation are proposed:

1. Apply all the workflows presented in Section 4-3 to a first quasi-linear example (i.e.
Maximum Subsidence) to identify specific or group of workflows with more potential of
efficiency, i.e. the potentially efficient workflows.

2. Apply the potentially efficient workflows for other quasi-linear examples.

The first stage uses the problem of maximum subsidence, whereas the second stage uses the
indicators of average and failure area subsidence as well as maximum and average formation
shear failure.
To identify workflows with potential of efficiency, a ranking for the accuracy levels is proposed
as follows:

• High: workflows capable to reach an accuracy higher than 0.70, Ap̂f > 0.70

• Medium: workflows capable to reach an accuracy between 0.40 to 0.70, 0.40 < Ap̂f <
0.70

• Low: workflows capable to reach an accuracy lower than 0.40, Ap̂f < 0.4

Note that this scale is arbitrarily selected by the author. For the target probability of failure
in this study, these values seem reasonable for the savings in computation time as it will be
seen later. As an example, if a particular workflow is identified to produce high accuracies
and the exact value of probability of failure is pf ;exact = 0.0020 it means that it is expected
to produce an outcome in the range 0.0014 < p̂f < 0.0029. It is seen that the high accuracies
provide still a good level of closeness to the exact value. For the same example, a medium
accuracy is considered to be in the ranges 0.0008 < p̂f < 0.0014 or 0.0029 < p̂f < 0.0050.
Finally, low accuracies would correspond to the ranges p̂f < 0.0008 or p̂f > 0.0050.

Pablo A. Vásconez RESTRICTED Master of Science Thesis

5-4 Evaluation for Subsidence with Homogeneous Subsurface 65

5-4 Evaluation for Subsidence with Homogeneous Subsurface

This section presents the first model used to evaluate the performance of the workflows to
compute the reliability. As stated earlier, all the workflows proposed in Section 4-3 are applied
to this model, in particular to the maximum subsidence indicator, to identify the potentially
efficient workflow.

5-4-1 Description of the model EWG and limit state values

First, the characteristics of the base case (i.e. the model using mean values) and then the
probabilistic distributions for the model EWG (Evaluation Workflows Geertsma) are pre-
sented. The geometry, parameters and distribution are likely values to find in practice, but
do not represent a specific case study.

D=2000m

R=2540m
h=100m

ν =0.40
cm=170E-06 MPa-1

Figure 5-1: Sketch of the model EWG

The model EWG represents a depleting reservoir at a mean depth of D = 2000m below the
surface. The shape of a reservoir is a cylinder with a mean radius R = 2540m and mean
thickness h = 100m, as depicted in Figure 5-1.
The subsurface, including the reservoir, is considered as an homogeneous material with linear
elastic properties: mean Poisson’s ratio ν = 0.40 and mean compaction coefficient cm = 170
× 10−6 MPa−1 . The mean depletion pressure ∆p = 25 MPa is used for production.
All the previous geometrical properties and model parameters are selected as random variables
with distributions summarized in Table 5-1. The values of vertical stresses, horizontal stresses
and pore pressures change with depth at fixed gradients of 20, 15 and 10 kPa/m, respectively.
Three of the failure indicators defined in Eq. (4-2) are analysed for subsidence: Maximum,
Average and Failure Area. The indicator Percentage of Failure Area is not analysed because
its results are exactly the same as failure area, as long as the reference area keeps constant.

Master of Science Thesis RESTRICTED Pablo A. Vásconez

 Evaluation of Workflows for
66 Quasi-Linear Responses

Table 5-1: Distribution of the random variables for model EWG

Random Variables Distribution Type Mean Standard Deviation COV

D [m] Normal 2000 30 0.015
R [m] Normal 2540 203.2 0.08
h [m] Normal 100 15 0.15
∆p [MPa] Normal 25 2 0.08
ν [-] Normal 0.40 0.02 0.05
cm [MPa−1 ] Normal 0.000170 0.000034 0.20

Taking as a reference the recommendations of the Eurocode about the target probability of
failure (Table 2-1), a value pf ≈ 0.002 is selected for the analyses. Prior performance of
Monte Carlo using the geomechanical model only, suggests the following limit values for the
different failure indicators: Rmax;sub = 0.40m for maximum subsidence; Rave;sub = 0.34m for
average subsidence; and Rarea;sub = 500m2 for failure area subsidence (this indicator requires
a limit selected as limsub = 0.38m, see Figure 4-1). The exact values of probabilities of failure
pf ;exact are 0.001618, 0.0018 and 0.002188, respectively.

5-4-2 Sensitivities with the Tornado Plot

A basic but insightful plot to learn about the sensitivities and the degree of non-linearity of
the response of interest is the Tornado Plot. Figure 5-2 shows the Tornado Plot built for the
model EWG using the sampling of the response surface FOD. The extreme values of this plot
are either P10 or P90 of the distribution of each random variable. The midpoint of the bars
correspond to P50 of each random variable. The colors of the bars indicate that only ν and
D have a negative correlation with the maximum subsidence, i.e. when they increase, the
response reduces. All the others are positively correlated.

cm

h
Random Variables

∆p

P10 - P50
D P50 - P90
0.14 0.16 0.18 0.2 0.22 0.24
Maximum Subsidence [m]

Figure 5-2: Tornado plot for maximum subsidence using the response surface FOD

To what sensitivities concerns, the tornado plot provides the random variables ranked from
most to least influential for maximum subsidence as: cm , h, ∆p, R, ν and D. This means
that a change is cm affects more the response and D affects less the response in this particular

Pablo A. Vásconez RESTRICTED Master of Science Thesis

5-4 Evaluation for Subsidence with Homogeneous Subsurface 67

case. Note that the sensitivity is connected to the variance of the variables in addition to the
real influence of a random variable on the response. For a random variable with a very small
variance, the bars in the tornado plot will have a very small length.
To what non-linearity regards, the tornado plots shows an almost equal variation of the
response for the same variation of the random variables (i.e. 1.28σi ) in opposite directions.
Only the random variable R shows a small non-linearity, because the change for in response
between P50-P90 is smaller thant P10-P50.
The tornado plot deserves some criticism. It does not show the interaction between the
random variables that actually exists. For instance, reducing the values of D and nu at the
same time could be thought to produce more settlement than the one-at-a-time variation.
Nevertheless, this is not true because a reduction of D decreases the effect of the Poisson’s
ratio, since there is a smaller vertical column for which the vertical strains increase and
therefore lower subsidence is produced. Another drawback is the use of only 3 points to try
to identify the characteristics of the response.

5-4-3 Results of the evaluation for Maximum Subsidence

This section presents the results for the evaluation of all the workflows proposed in Section 4-3
for maximum subsidence. The exact value of probability of failure (obtained by Monte Carlo
on the geomechanical model) is 0.001618.

Workflows using a Response Surface Res

The first group of workflows evaluated are the ones that use exclusively the response surface,
Res, as explained in Section 4-3.
Table 5-2 summarized the results for all the possible combinations under this type of workflow.
The rows present the response surface used and the columns the reliability method. Focusing
on the first part of the table, the estimated values of probability of failure are rather far from
the considered exact value of 0.001618. Correspondingly, the values of accuracy Ap̂f are very
low, regardless the initial sampling considered.
The conclusion for this subsection is that the reliability methods performed directly on a
response surface produces low accuracies for quasi-linear problems. The reason lies in the
sampling, which takes place in some predefined directions and at a predefined distance from
the mean, regardless the distance to which the limit state function g(x) = 0 is located.
When computing the probability of failure, a extrapolation takes place to other directions
and usually very far distances.

Workflows using a Response Surface and the Geomechanical Model Rit

The results of the evaluation of the group of workflows Rit, requires a graphical representation,
because of the possibility to change the initial threshold th;ini in the analyses. As it will be
observed, the selection of this value has an important effect on the accuracies.

Workflows Ritfor Monte Carlo: Results

Master of Science Thesis RESTRICTED Pablo A. Vásconez

 Evaluation of Workflows for
68 Quasi-Linear Responses

Table 5-2: Results of workflows Resfor model EWG

Response Reliability Method

Surface MC IS DS FM

Probability of Failure, pf [10−3 ]

FO10 0.130 0.075 0.060 0.072
FO90 0.110 0.068 0.055 0.064
FOD 0.110 0.066 0.054 0.064
SO 0.470 0.013 0.000 0.010
ED 0.320 0.243 0.191 0.246

Accuracy the Probability of Failure, Ap̂f

FO10 0.08 0.05 0.04 0.04
FO90 0.07 0.04 0.03 0.04
FOD 0.07 0.04 0.03 0.04
SO 0.29 0.01 0.00 0.01
ED 0.20 0.15 0.12 0.15

Number of model realisations, Nmod

FO10 7 7 7 7
FO90 7 7 7 7
FOD 13 13 13 13
SO 65 65 65 65
ED 9 9 9 9

Figure 5-3 shows the plots probability of failure, accuracies and number of geomechanical
model realisations vs. the initial threshold th;ini for different response surfaces. All the
results correspond to the group of workflows Rit{Monte Carlo}.
The first relevant observation is that the use of the threshold improves the accuracy on the
probability of failure. For instance, the response surface FOD for th;ini =0m results in an
accuracy as low as 0.08; whereas that for th;ini =0.12m, the accuracy obtained is as high as
0.98. An improvement occurs for all the response surfaces without exception.
The second observation is that, excluding response surface SO, there is a peak-residual shape
of the plot Ap̂f vs. th;ini . From 0 to a certain value of th;ini , there is an increase in accuracy
until a peak in accuracy is reached. For instance, for the response surface FO10 a th;ini =0.04m
is required to reach the peak; for response surface FO90 a th;ini = 0.12m is needed. For higher
initial thresholds, the value the accuracy reduces until a residual value. The th;ini for what
the residual accuracy occurs approximately at 40% of the limit value Rmax;sub , but any value
higher than the mentioned one would produce exactly the same the result. This means that
an infinite th;ini produces the residual accuracy.
A third observation, this time coming from the plot Nmod vs. th;ini , is that the number of
realisations increase with the initial threshold. For instance, response surface ED requires
around 12 model realisations for th;ini = 0.04m and 15 model realisations for th;ini = 0.20m.
But this has exceptions. Response surface FO90 shows a slight decrease in the number of
samples after between th;ini = 0.12m and 0.16m. However, in the great majority of cases, this

Pablo A. Vásconez RESTRICTED Master of Science Thesis

5-4 Evaluation for Subsidence with Homogeneous Subsurface 69

observation holds.
A fourth observation, that follows from all the previous, is that an increase in number of
realisations does not necessarily lead to an increase in accuracy. For a th;ini between 0 and
the one for which the peak accuracy occurs, there an increase in number of realisations derives
in increase of accuracy. But for th;ini between the one for the peak and the one for the residual,
an increase in number of realisations derives in a decrease of accuracy.
The fifth observation from these plots, is that all the response surfaces for any initial threshold,
underestimate the probability of failure. The only exception is response surface FOD, which
overestimated for th;ini = 0.04m, as observed in the plot pf vs. th;ini .

Workflows Ritfor Monte Carlo: Discussion

Five observations are relevant for workflows Rit{Monte Carlo}, which arise from Figure 5-3.
This subsections proposes an explanation for each of them.
The explanation for the first observation about the increase of accuracy with the use of the
threshold is straight-forward. Figure 4-6 shows how this type of workflow offers help in two
ways to the computation of the probability of failure. First, there is a range that contains
the limit value for which the geomechanical engine is used rather than the response surface.
And second, the response surface is updated with outcomes that are closer to the limit state,
improving the approximation of the response surface with respect to the geomechanical model
in a region closer to failure.
The peak-residual relation between the accuracy and initial threshold is explained by the
location of the new samples that are used in the regression to update the response surface. The
difficulty to represent graphically due to the high dimensionality of the example, forces to use
a fictional case to show how the interplay occurs. Figure 5-4(b), Figure 5-4(d) and Figure 5-
4(f) show the initial response surface (solid inclined line) with different initial threshold values
(represented by different distances between the horizontal solid lines). The density function
is represented in all the plots to highlight the likeliness for a specific value of x1 to be taken.
The samples taken to form the response surface are also shown, which are only 2 samples
(represented by crosses) in all cases.
For very small initial thresholds, the low accuracy on the probability of failure arises from
the far location of the new samples (respect to the design point) and the very few (or no)
updates during the Monte Carlo simulation. In Figure 5-4(b), it is observed that the value
of x1 ≈ 10, that will produce an update of the response surface, is located at a far distance
from the mean but also from the design point (x1 ≈ 8). This has a double negative impact
on the accuracy. First, this point has a small probability of occurrence, meaning that a
high number of samples are likely to be taken using the initial response surface and thus the
failures between the range 8 < x1 < 10 are not considered. Second, when x1 ≈ 10 occurs, the
new response surface will overestimate (in this case) even if a small non-linearity in the actual
response surface exists. The overestimation can be observed in Figure 5-4(c), which is actually
a response surface after 3 updates, but it is easy to imagine that when the sample for x1 ≈ 10
was taken, the overestimation of the actual response was even higher. This indicates that the
overestimation can be corrected with more sampling, but the almost negligible threshold after
very few updates constrains this. Furthermore, note that x1 ≈ 10 may never occur during
the simulation and thus no updates of the response surface.

Master of Science Thesis RESTRICTED Pablo A. Vásconez

 Evaluation of Workflows for
70 Quasi-Linear Responses

(a)
3.00
FO10

Probability of Failure, pf [10−3]

FO90
2.50 FOD
SO
2.00 ED
exact
1.50

1.00

0.50

0.00
0.00 0.04 0.08 0.12 0.16 0.20 0.24 0.28 0.32 0.36 0.40
Initial Threshold Maximum Subsidence [m]

(b)
1.00
Accuracy Probability of Failure, Ap̂f

FO10
0.90 FO90
0.80 FOD
SO
0.70 ED
0.60
0.50
0.40
0.30
0.20
0.10
0.00
0.00 0.04 0.08 0.12 0.16 0.20 0.24 0.28 0.32 0.36 0.40
Initial Threshold Maximum Subsidence [m]

(c)
80
FO10
70 FO90
FOD
60 SO
ED
50
Nmod

40
30
20
10
0
0.00 0.04 0.08 0.12 0.16 0.20 0.24 0.28 0.32 0.36 0.40
Initial Threshold Maximum Subsidence, th [m]

Figure 5-3: Performance of workflow Rit{Monte Carlo} for Maximum Subsidence of example
EWG. (a) Probability of Failure pf (b) Accuracy of the probability of failure Apˆf and (c) Number
of geomechanical model realizations Nmod for different initial threshold values th;ini and response
surfaces

Pablo A. Vásconez RESTRICTED Master of Science Thesis

 5-4 Evaluation for Subsidence with Homogeneous Subsurface 71

Actual Response R (lim.val.) New Response Surface

Previous sampling points PDF*20 New Sampling point

(a)

12 12

Actual Response Actual Response

10 10
R (lim.val.)

8 R (lim.val.) New Response Surface

8
Previous sampling points
L(x1)

L(x1)
6 6 Response Surface PDF*20
New Sampling point
2 4 6 8 4 10 4 Points used for
x1 sampling
2 2
PDF*20

0 0
0 2 4 6 8 10 0 2 4 6 8 10
x1 x1

(b) Initial RS for very small initial threshold (c) Final RS for very small initial threshold

12 12

Actual Response
10 10 Actual Response

R (lim.val.) R (lim.val.)
8 8

New Response Surface

L(x1)

L(x1)

6 6 Response Surface
Previous sampling points
4 4
Points used for
sampling
PDF*20
2 2
PDF*20 New Sampling point
0 0
0 2 4 6 8 10 0 2 4 6 8 10
x1 x1

(d) Initial RS for small initial threshold (e) Final RS for small initial threshold

12 12

Actual Response
10 10 Actual Response
R (lim.val.)
New Response Surface
8 8 R (lim.val.)
Previous sampling points
L(x1)
L(x1)

6 6 PDF*20
Response Surface
New Sampling point
4 4
Points used for
2 2 sampling

PDF*20
0 0
0 2 4 6 8 10 0 2 4 6 8 10
x1 x1

(f) Initial RS for big initial threshold (g) Final RS for big initial threshold

Figure 5-4: Resulting sampling and prediction of the response surface for different initial thresh-
olds

Master of Science Thesis RESTRICTED Pablo A. Vásconez

 Evaluation of Workflows for
72 Quasi-Linear Responses

For small initial threshold, the good accuracy on the probability of failure owes to the location
of the new samples near the design point and a sufficient amount of updates during the Monte
Carlo simulation. From Figure 5-4(d), a value of x1 ≈ 7 is required to update the response
surface, located between the mean and the design point. This produces that less samples
are taken with the initial response surface before the first update occurs, but also the range
of x1 for which failures occur is smaller than when using a smaller initial threshold. A very
favourable situation occurs in the case under analysis, because there is no range of x1 for which
failures occurs with the initial threshold. This means that, even if the response surface never
updates and the threshold range remains the same, the results will be perfectly accurate.
Nevertheless, this may be time consuming, since still many samples are going to be taken
inside the threshold range. The updates do occur and the final response surface represents
very accurately the actual response nearby the design point. Figure 5-4(e) shows this after 5
updates, possible thanks to the wider threshold range.
For a big threshold, the reduction in accuracy on the probability of failure is produced due to
the presence of samples very close to the mean during the Monte Carlo simulation. In Figure 5-
4(f), it is observed that any value of x1 will produce an update of the response surface. This
means that even a value close to the mean and far from the design point is used to update the
response surface, not producing a significant improvement of the response surface towards
the limit state. In despite of this, the threshold still reduces and may leave ranges of x1
for which failures are possible to occur. Because a normal regression is used considering all
the previous samples, the new samples that would be closer to the design point will have a
smaller influence to improve the response surface; in other words, more samples are required
to produce the same accuracy between the response surface and the actual response than if
the samples close to the mean would have not been taken. The samples close to the mean act
as an anchor for the subsequent response surfaces. In Figure 5-4(g) it can be observed the
presence of additional samples closer to the mean; it can be argued that the final response
surface is very accurate, but it has to considered that this occurred after 7 updates. Until
this occurs, there may be a certain amount of failures not counted, decreasing the accuracy
on the probability of failure.
The explanation of the peak-residual relationship between the accuracy and the initial thresh-
old, also explain the third and fourth observations made. The number of realisations increase
with the initial threshold because the threshold range is wider. But it was also observed a
local reduction in a small number of cases. They all occur right after the peak accuracy.
The more likely explanation is that the simulation produced a value of x1 very close to the
design point, being an exception rather than the rule. Concerning the fourth observation,
the explanation of why more realisations do not necessarily improve the accuracy lies on the
importance of the location the sampling. Big initial thresholds produce more samples, but
also more centred towards the mean rather the design point.
The fifth observation is fully explained by the fact that the response surface underestimate
the actual response at the design point. This is clear in Figure 5-4. For most of the density
function, the response surface predicts accurately the response surface; however, at the design
point, located on the tail of the distribution, the response surface underestimates significantly
the response. Subsequently, the probability of failure will be underestimated. The reader may
wonder why the fictional case is used to explain also this observation. The reason is that to
a certain extent, the actual response has this shape although in multiple dimensions. This
follows from the analytical solution for maximum subsidence (Geertsma, 1973):

Pablo A. Vásconez RESTRICTED Master of Science Thesis

5-4 Evaluation for Subsidence with Homogeneous Subsurface 73

!
D/R
Lsub (x) = 2 ∗ cm (1 − ν)∆p ∗ h 1 − p (5-2)
1 + (D/R)2

Let consider only two random variables for simplicity: cm and h. Increasing the values of
these two variables at the same time, gives and effect similar to a quadratic function. In
other words, travelling through the u-space in a direction other than the axes, will produce
a function either quadratic, cubic, etc. For instance, Figure 5-5 shows the variation of cm
and h in a specfic direction the u-space: α = (0.707, 0.707); hence, equal change for both
variables. By looking at this plot, it is clear that the shape of the fictional case and the
actual response surface for the case under analysis are very similar. The direction selected
is actually one of the most unfavourable according to the Tornado Plot, meaning that it is
likely that the failures counted in Monte Carlo appear in a significant number in this or a
similar direction. The CDF in this direction is presented (although scaled by 0.5) in order
to see that the underestimation appears already from points that have a probability of 0.20
(CDF = 0.80) to be taken.

0.50

0.45
Maximum Subsidence, Lsub (ucm , uh ) [m]

0.40

0.35

0.30

0.25

0.20

0.15

0.10
Actual Response
0.05 RS FO90
CDF*0.5
0.00
−5.0 −4.0 −3.0 −2.0 −1.0 0.0 1.0 2.0 3.0 4.0 5.0
±|u| at α = (0.707, 0.707)

Figure 5-5: Maximum subsidence for a specific direction

The conclusion form these results is that the workflows using threshold and Monte Carlo
can produce good accuracies with a small number of model realisations, making this group
of workflows very efficient for quasi-linear problems. All the first-order response surfaces
required less than 20 model realisations to get accuracies of more than 0.70. For second-order
response surfaces 72 samples are needed, but most of them correspond to the generation of the
initial response surface (65 samples), suggesting that the computation time may be reduced
by using a sampling technique other than full factorial design. An important selection is the
initial threshold, that for this example, required to be around 0.1 and 0.3 the limit value
Rmax;sub .

Master of Science Thesis RESTRICTED Pablo A. Vásconez

 Evaluation of Workflows for
74 Quasi-Linear Responses

Workflows Ritfor Importance Sampling: Results and Discussion

Figure 5-6 shows the probability of failure, accuracy and number of geomechanical model
realisations versus the initial threshold th;ini for different response surfaces. All the results
correspond to the group of workflows Rit{Importance Sampling}.
All the observations made previously for Rit{Monte Carlo} are valid for the workflows under
analysis in this subsection:

• The use of the threshold improves the accuracy on the probability of failure. For in-
stance, response surface FO90 the accuracy increases from 0.04 for th;ini =0m to 0.50
for th;ini =0.12m.

• There is a peak-residual shape of the plot Ap̂f vs. th;ini , excluding response surface
FO10 and SO.

• The number of realisations increase with the initial threshold. For instance, response
surface FO10 initially requires around 9 model realisations for th;ini = 0.04m and 12
model realisations for th;ini = 0.20m. There are also very few exceptions for the rule,
e.g. for response surface FOD between th;ini = 0.04 and 0.08m.

• The increase in number of realisations does not necessarily lead to an increase in accu-
racy. Response surface ED required 13 realisations to get an accuracy of 0.35, but if
the th;ini is increased to allow 14 realisations to occur, the accuracy is reduced to 0.20.

• All the response surfaces for any initial threshold, underestimate the probability of
failure.

The explanation for these observations are the same as for Monte Carlo, given the similarities
between these two procedures. Figure 5-7 reaffirm that the only addition in Importance
Sampling is the use of the importance sampling density function h(x), whose resulting CDF
is presented as a dotted line. By comparing it with the actual CDF of the random variables,
it is observed that it is more likely that more extreme values of |u| are taken, meaning that a
situation of an overestimation of the response surface as explained in Figure 5-4(c), is more
probable to occur. However, the same figure depicts that this is corrected with subsequent
updates of the response surface.
The reader may wonder, then, if overestimation is more likely, then why all the response
surface underestimate the result? The answer is connected to an outcome obtained later
(results for Mod) in this section: importance sampling using exclusively the geomechanical
model, underestimated the result. The latter, obtained a probability of failure pf = 0.001045.
Having, this result as the reference pf ;exact , it can be inferred from Figure 5-6 that response
surface FOD overestimate the probability of failure. Hence, more overestimation occurred for
Importance Sampling.
A relevant observation for these workflows is that the accuracies obtained (versus the Monte
Carlo result) are very sensitive on the variance factor used, whereas the number of model
realisations are not. For the variance factor of 1.5, used for this example, the accuracies
obtained are low. The only exception is FOD, but this is not consistent, because as explained

Pablo A. Vásconez RESTRICTED Master of Science Thesis

5-4 Evaluation for Subsidence with Homogeneous Subsurface 75

(a)
2.00
FO10
1.80

Probability of Failure, pf [10−3]

FO90
1.60 FOD
SO
1.40 ED
1.20 exact
1.00
0.80
0.60
0.40
0.20
0.00
0.00 0.04 0.08 0.12 0.16 0.20 0.24 0.28 0.32 0.36 0.40
Initial Threshold Maximum Subsidence [m]

(b)
1.00
Accuracy Probability of Failure, Ap̂f

FO10
0.90 FO90
0.80 FOD
SO
0.70 ED
0.60
0.50
0.40
0.30
0.20
0.10
0.00
0.00 0.04 0.08 0.12 0.16 0.20 0.24 0.28 0.32 0.36 0.40
Initial Threshold Maximum Subsidence [m]

(c)
70
FO10
60 FO90
FOD
50 SO
ED
40
Nmod

30

20

10

0
0.00 0.04 0.08 0.12 0.16 0.20 0.24 0.28 0.32 0.36 0.40
Initial Threshold Maximum Subsidence, th [m]

Figure 5-6: Performance of workflow Rit{Importance Sampling} for Maximum Subsidence of

example EWG. (a) Probability of Failure pf (b) Accuracy of the probability of failure Apˆf and (c)
Number of geomechanical model realizations Nmod for different initial threshold values th;ini and
response surfaces

Master of Science Thesis RESTRICTED Pablo A. Vásconez

 Evaluation of Workflows for
76 Quasi-Linear Responses

0.50

Maximum Subsidence, Lsub (ucm , uh ) [m] 0.45

0.40

0.35

0.30

0.25

0.20

0.15

0.10 Actual Response

RS FO90
0.05 CDF*0.5
CDF;IS*0.5
0.00
−5.0 −4.0 −3.0 −2.0 −1.0 0.0 1.0 2.0 3.0 4.0 5.0
±|u| at α = (0.707, 0.707)

Figure 5-7: Comparison between the actual response and response surface, including the impor-
tance sampling cumulative density

in the previous paragraph, it overestimate the probability of failure of Importance Sampling

using exclusively the response surface. If a the variance factor of 1.0 would have been used
instead, the importance sampling density h(x) to be equal to the actual density f (x), and
the same accuracies from Rit{Monte Carlo} are obtained for Rit{Importance Sampling}. It
follows that for intermediate values of variance factors between 1.0 and 1.5 the accuracies will
change. Regarding the number of model realisations, it is observed that there is negligible
difference between Rit{Monte Carlo} results and Rit{Importance Sampling} and thus the
change in variance factor does not represent an advantage in computation time.
The final conclusion is that these Rit{Importance Sampling} are not more efficient than
Rit{Monte Carlo}. Despite, good accuracies may be obtained, several variance factor need
to be investigated, requiring more computational time at the end.

Workflows Ritfor Directional Sampling: Results and Discussion

Figure 5-8 shows the probability of failure, accuracy and number of geomechanical model
realisations versus the initial threshold th;ini for different response surfaces. All the results
correspond to the group of workflows Rit{Directional Sampling}.
From the observation made previously for Rit{Monte Carlo} and Rit{Importance Sampling}
the only valid is that the number of realisations increase with the initial threshold. Remind
that this owes to the use of a threshold that reduces as new samples follow inside the threshold
range, because the bigger the initial threshold, the more new samples required to update the
threshold. Furthermore, it is observed that in average, Rit{Directional Sampling} requires
more model realisations than Rit{Monte Carlo}.
Furthermore, the accuracies are low for most of the cases. Only response surface ED and

Pablo A. Vásconez RESTRICTED Master of Science Thesis

5-4 Evaluation for Subsidence with Homogeneous Subsurface 77

(a)
3.00
FO10

Probability of Failure, pf [10−3]

FO90
2.50 FOD
SO
2.00 ED
exact
1.50

1.00

0.50

0.00
0.00 0.04 0.08 0.12 0.16 0.20 0.24 0.28 0.32 0.36 0.40
Initial Threshold Maximum Subsidence [m]

(b)
1.00
Accuracy Probability of Failure, Ap̂f

FO10
0.90 FO90
0.80 FOD
SO
0.70 ED
0.60
0.50
0.40
0.30
0.20
0.10
0.00
0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00
Normalized Threshold Maximum Subsidence, R th
max;sub

(c)
80
FO10
70 FO90
FOD
60 SO
ED
50
Nmod

40
30
20
10
0
0.00 0.04 0.08 0.12 0.16 0.20 0.24 0.28 0.32 0.36 0.40
Initial Threshold Maximum Subsidence, th [m]

Figure 5-8: Performance of workflow Rit{Directional Sampling} for Maximum Subsidence of

example EWG. (a) Probability of Failure pf (b) Accuracy of the probability of failure Apˆf and (c)
Number of geomechanical model realizations Nmod for different initial threshold values th;ini and
response surfaces

Master of Science Thesis RESTRICTED Pablo A. Vásconez

 Evaluation of Workflows for
78 Quasi-Linear Responses

response surface FOD medium accuracies, but only for one value of initial threshold for
ED and 2 values of initial threshold for FOD. All the other response surfaces and initial
thresholds produced accuracies lower than 0.30. Moreover, response surfaces FO10 and SO
did not achieved did not produce accuracies higher than 0.03.
Also, notice that all the response surfaces (excluding SO) overestimated the probability of
failure for several values of initial threshold.
The reason for two previous observations owes to the linear search method employed by
Directional Sampling. Recall that the goal of the linear search method is to find the distance
to the limit state (Section 2-4-3 ) in a random direction α(i) , which means that the estimate
of the performance function ĝ(x) would be approximately 0 already for the first direction
that finds failure and subsequently, the threshold would be approximately 0. As a result, the
response surface will align strongly towards this direction, because it will have several sampling
points in this point. Given that the direction is random in the uniform hypersphere, it is more
likely that a direction whose α(i) is relatively big occurs during simulation; producing that
sampling points, from a region indeed close to the limit state but with a practically zero
chance of occurrence, are used to update the response surface. In addition, the far location
of these samples also increase the chance to produce overestimation over the regions of with
more probability of occurrence.
0.50

0.45

0.40 3 4
Maximum Subsidence, Lsub (u) [m]

2
0.35
1
0.30

0.25

0.20

0.15
Actual Response
0.10 RS FO10 (1)
RS FO10 (4)
0.05 CDF*0.5
New Samples
0.00
0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 5.5 6.0 6.5 7.0
|u| at α(i)

Figure 5-9: Comparison between the actual response and response surface at the direction
α(i) = (D, R, h, ∆p, ν, cm ) = (0.3178, 0.3445, 0.4846, 0.6702, 0.3103, 0.0059), for a simulation
with Directional Sampling

Figure 5-9 serves to reaffirm the assertions from the previous paragraph. The actual response
surface (curved solid line), response surface FO10 (inclined solid lines) and the cumulative
density function (dash-dot curve line) are plotted versus the distance to the mean values |u|
in specific direction α( i). This direction is actually the fifth direction (all the others did
not produce failure) taken during the directional simulation for response surface FO10 with

Pablo A. Vásconez RESTRICTED Master of Science Thesis

5-4 Evaluation for Subsidence with Homogeneous Subsurface 79

th;ini = 0.12m. The numbered points lying on the actual response, correspond to the new
samples considered to update the response surface FO10, showing that a total of 4 updates
occur in this direction. Response surfaces passes from response surface FO10 (1) to FO10 (4);
note that the latter represents better the actual response, having a perfect prediction around
|u| = 5, which is a point with a CDF of 0.9997, meaning that there is a very low probability
(0.0003) this or a higher distance from the mean in this direction is taken. Note that the
new sample 4 is very close to the limit value Rmax;sub , being already the threshold very small
and not allowing the other directions to take more than 1 update. Therefore, this direction
will have a significant influence in the response surface. Unfortunately, the prediction of
this response surface for more likely values, e.g. |u| = 1 to 3, are slightly overestimated (as
observed in the plot), leading to the low accuracy and overestimation of the probability of
failure.
The higher number of model realisations compared of Rit{Directional Sampling} comes also
from the linear search method, that explicitly attempts to find the limit state, increasing the
likeliness to get points inside the threshold range in comparison to Monte Carlo.
The explanations given in this subsection makes the implementation of Directional Sampling
and threshold unreliable to compute the probability of failure.

Workflows Ritfor FORM: Results and Discussion

Figure 5-10 shows the probability of failure, accuracy and number of geomechanical model
realisations versus the initial threshold th;ini for different response surfaces. All the results
correspond to the group of workflows Rit{FORM}.
The peak-residual relation between Apf and th;ini observed in Monte Carlo, also appears in a
limited extent in for FORM. Only two response surfaces FO10 and FO90 present this relation,
whereas response surfaces SO and ED show an increase of accuracy with the initial threshold
for all cases. The reason for this may lie in the original prediction that ED and SO in the
regions close to the design point, that is corrected with the widening of the initial threshold as
more samples are available, positive effect that is constrained by the closing of the threshold.
For all the response surfaces and initial thresholds, the probability of failure is underesti-
mated. The reason is the shape of the response, that as observed before, grows faster as the
distance from the mean-values increase. Recall that FORM performs a gradient-based search,
producing that the sampling is done through a specific path of the u-space, seeking a point
close to most likely failure point. When the search starts from a point in the safe domain,
the path in most cases produces values smaller than the limit value and, subsequently, when
samples fall in the threshold range, they are prone to produce underestimation of the response
and the probability of failure.
The reader may have noticed that the plot corresponding to response surface FOD does not
appear in Figure 5-10. The reason is that FORM did not converge for this response surface,
thus no result for Rit{FO10,FORM} is obtained. This has to do with the fact that the
response surface FOD is discontinuous, even when only the initial sampling is used to build
it. The discontinuity is enhanced when more sampling is available far from the mean, because
of the slight non-linearity that this example contains. To prove this, the prediction of mean
for th;ini = 0.40m using the coefficients for values lower than the mean is compared with the
prediction using the coefficients for values higher than the mean. The comparison shows that

Master of Science Thesis RESTRICTED Pablo A. Vásconez

 Evaluation of Workflows for
80 Quasi-Linear Responses

(a)
2.00
FO10
1.80

Probability of Failure, pf [10−3]

FO90
1.60 FOD
SO
1.40 ED
1.20 exact
1.00
0.80
0.60
0.40
0.20
0.00
0.00 0.04 0.08 0.12 0.16 0.20 0.24 0.28 0.32 0.36 0.40
Initial Threshold Maximum Subsidence [m]

(b)
1.00
Accuracy Probability of Failure, Ap̂f

FO10
0.90 FO90
0.80 FOD
SO
0.70 ED
0.60
0.50
0.40
0.30
0.20
0.10
0.00
0.00 0.04 0.08 0.12 0.16 0.20 0.24 0.28 0.32 0.36 0.40
Initial Threshold Maximum Subsidence [m]

(c)
80
FO10
70 FO90
FOD
60 SO
ED
50
Nmod

40
30
20
10
0
0.00 0.04 0.08 0.12 0.16 0.20 0.24 0.28 0.32 0.36 0.40
Initial Threshold Maximum Subsidence, th [m]

Figure 5-10: Performance of workflow Rit{FORM} for Maximum Subsidence of example EWG.
(a) Probability of Failure pf (b) Accuracy of the probability of failure Apˆf and (c) Number of
geomechanical model realizations Nmod for different initial threshold values th;ini and response
surfaces

Pablo A. Vásconez RESTRICTED Master of Science Thesis

5-4 Evaluation for Subsidence with Homogeneous Subsurface 81

the difference for the first update of the response surface is as small as 0.1cm, whereas for the
second update the difference is 1cm. These differences ultimately are translated to jumps in
the gradient that cannot be treated by FORM. With no updates present, given the linearity
for the initial sampling points, the workflow Res{FORM} has no problem to converge as
shown in Table 5-2.
The conclusion for this subsection is that Rit{FORM} is a workflow capable to produce good
accuracies at a very low computation time. Nevertheless, there is still a problem with the
selection of the response surface and the initial threshold in order to maximize the accuracy.
From the response surfaces evaluated, the most successful response surfaces are FO10 and
FO90; on the other hand, FOD must be disregarded due to its discontinuity.

Workflows using the Geomechanical Model, Mod

Table 5-3 shows the probability of failure, accuracy and number of geomechanical model
realisations for the evaluation of workflows Mod. All the four proposed reliability methods
are used.
Table 5-3: Results of the evaluation of workflows Mod for all the proposed reliability methods

Workflow Probability of failure, Accuracy, Number of

pf Ap f model realisations
Nmod
Mod{MonteCarlo} 0.001618 1.00 61809
Mod{Importance Sampling} 0.001043 0.64 54853
Mod{Directional Sampling} 0.001205 0.74 102748
Mod{FORM} 0.001503 0.93 28

The first observation is that the number of model realisations differs from each simulation
method. Recall that it is of interest the efficiency of the methods, meaning that the comparison
looks at both accuracy and computation time. For this reason, the methods are not compared
for the same number of simulations, but for the same convergence criteria. Recall that the
convergence for the simulation reliability methods, as stated in Table 4-4, is that the COV
of the probability of failure is reaches 0.1 or that the number of samples reaches 100,000;
whichever occurs first. This produces the mentioned difference in realisations.
Another observation, is that Directional Sampling reaches good accuracy, but requires more
realisations than Monte Carlo. The reason for this comes from the discretisation in 20 intervals
that this methods does for directions in which the limit state is not reached (Figure 4-10). The
actual number of sampled directions is around 6000, meaning that if the discretisation would
contain 4 intervals the number of simulations would have been reduced to around 24000, which
is still a high number. Nevertheless, the current implementation must be modified in order to
reduce the computation time. The good accuracy could be improved by being more strict on
the tolerances of the linear search method, with the cost of increasing the computation time.
Importance Sampling, using the increase variance technique, required smaller computation
time than Monte Carlo but a just a medium accuracy is obtained. The reason for this low
accuracy may lie on the convergence criteria. Figure 5-11 shows the Monte Carlo and the
Importance Sampling, using a variance factor of 1.5. It is observed that Monte Carlo has

Master of Science Thesis RESTRICTED Pablo A. Vásconez

 Evaluation of Workflows for
82 Quasi-Linear Responses

remained between 0.00145 and 0.00175 for nearly 50,000 samples, with a tendency to keep
the result within the same range. Importance Sampling indeed converged according to the
criteria used, but its result has also a slight tendency to increase, suggesting that with more
sampling both simulations may converge to the same value. This value is more likely to
be higher than 0.00145, though. Hence, the convergence criteria to compare Importance
Sampling and Monte Carlo should not be at the same COV for the probability of failure and
rather continue the simulation until a true convergence for each method separately. If this is
done for this case, the computation time for Importance Sampling would be increased.

0.0030
MC
IS
0.0025
Probability of failure, pf

0.0020

0.0015

0.0010

0.0005

0.0000
0 10 20 30 40 50 60 70
3
Number of samples, N [10 ]

Figure 5-11: Probability of failure as a function of the number of samples for a Monte Carlo (MC)
and Importance Sampling (IS) simulation. The variance factor used for Importance sampling is
1.5

The most important observation is the good performance of FORM, which obtained 0.93 for
accuracy using only 28 model realisations. The only remark is that the slightly reduction in
accuracy owes mainly to the approximation of the limit state function as a hyperplane and
in a lower extent to the approximation of the design point.

Being the error small for FORM, suggests that the limit state function can be approximated
to a great extent as a hyperplane for this example. Furthermore, the obtained probability
of failure of 0.00153 provides more support to the assertion that Importance Sampling may
reach a higher value if more sampling is allowed.

The conclusion for workflows Mod is that only FORM is able to provide both accurate and
efficient results for this example. Both Directional and Importance Sampling, as implemented
in this study, required a prohibitive number of realisations and still did not provide an out-
standing accuracy.

Pablo A. Vásconez RESTRICTED Master of Science Thesis

5-4 Evaluation for Subsidence with Homogeneous Subsurface 83

Efficient Workflows for Maximum subsidence

Three are the workflows that computed accurately and efficiently the probability of fail-
ure for the Maximum Subsidence of example EWG: Mod{FORM}, Rit{Monte Carlo} and
Rit{FORM}. The only reason to place first Mod{FORM} is that it can accomplish the task
using the default values from Table 4-4, which is a good advantage compared to the other
two.
Nevertheless, if the response surface and initial threshold are selected in a proper manner,
workflows Rit{Monte Carlo} and Rit{FORM} are prone to be more efficient Mod{FORM}.
From the plots for peak accuracies in Figure 5-18 and Figure 5-19, Rit{Monte Carlo} for
response surfaces FO10 and FO90, required only 9 realisations to obtain the perfect accuracy
and, Rit{FORM} obtained very high accuracies, requiring a similar amount of realisations.
This is an important reduction compared to the 28 realisations from Mod{FORM} and a
major reduction to the 61,809 realisations by Mod{Monte Carlo}.
A final remark is that, although response surface SO takes a significant higher amount of
realisations compared to the first-order response surfaces, its accuracy is not higher than
them. Moreover, response surface SO requires even more realisations than Mod{FORM}.
Given that the initial sampling is what takes most of the computation time, alternatives to
the use of full factorial must be investigated.

5-4-4 Results of the selected workflows for Average Subsidence

This section presents the results of the selected workflows for the failure indicator Average
Subsidence. The selected workflows are the ones that resulted efficient for maximum subsi-
dence: Mod{FORM}, Rit{Monte Carlo} and Rit{FORM}. For the last two, different initial
thresholds are used, given that no specific combination of initial threshold and response sur-
face has proven to be a robust alternative.
In Figure 5-12(a), the results for Rit{Monte Carlo} show that qualitatively the results for
average subsidence are similar than for maximum subsidence: there is a peak-residual relation
between the accuracy on the probability of failure and the initial threshold for the first-order
response surfaces (except ED), although with a residual accuracy slightly lower than the peak.
This may be driven by a slightly more linearity in the response due to the averaging of the
subsidence, rather than considering only one point, which ultimately is the case for maximum
subsidence. Nevertheless, what is concerning is the performance of the response surface FOD,
because it reached a good accuracy at around th;ini = 0.15m, but then if diminished to an
accuracy of only 0.24 for th;ini =0.20m. What cannot be observed from this plots is that the
probability of failure decreases exponentially with the initial threshold after th;ini =0.04m,
from 0.00654 to 0.00043.
In Figure 5-12(b), the results for Rit{FORM} are also similar to the obtained for maximum
subsidence: the peak-residual relation is slightly more pronounced in this example for the
first-order response surfaces. Note that the initial threshold for the peak accuracy is around
10% of the limit value Rave;sub = 0.34m, except response surface SO, which required around
60%. The peak and the residual accuracies are good for all response surfaces.
Workflow Mod{FORM} proved to be accurate and relatively efficient as well. It obtained
a probability of failure of 0.00203, corresponding to an accuracy of 0.88. The number of

Master of Science Thesis RESTRICTED Pablo A. Vásconez

 Evaluation of Workflows for
84 Quasi-Linear Responses

(a) Rit{Monte Carlo}

1.00
Accuracy Probability of Failure, Ap̂f

FO10
0.90 FO90
0.80 FOD
SO
0.70 ED
0.60
0.50
0.40
0.30
0.20
0.10
0.00
0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.34
Initial Threshold Average Subsidence [m]

(b) Rit{FORM}
1.00
Accuracy Probability of Failure, Ap̂f

FO10
0.90 FO90
0.80 SO
ED
0.70
0.60
0.50
0.40
0.30
0.20
0.10
0.00
0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.34
Initial Threshold Maximum Subsidence [m]

Figure 5-12: Accuracy of the probability of failure Apˆf for different initial threshold values th;ini ,
response surfaces and group of workflows

Pablo A. Vásconez RESTRICTED Master of Science Thesis

5-4 Evaluation for Subsidence with Homogeneous Subsurface 85

realisations required are 28 (Figure 5-18 and Figure 5-19), being a great reduction compared
to the 55,510 realisations that Monte Carlo required.
Therefore, the selected workflows show have the potential to compute the probability of failure
accurately and efficiently, keeping the need of a proper selection the combination of response
surface and initial threshold.

5-4-5 Results of selected workflows for Failure Area Subsidence

The results for the efficient workflows for the indicator Failure Area for Subsidence are pre-
sented in Table 5-4. No workflow other than Mod{Monte Carlo}, produce an outcome of the
reliability.

Table 5-4: Evaluation efficient workflows in Failure Area Subsidence

Workflow Probability of Accuracy

Failure probability of
(pf ) of failure
(Ap̂f )
Mod{Monte Carlo} 0.002188 1
Mod{FORM} N/C N/C
Rit{Any RS, Monte Carlo} 0 0
Rit{Any RS,FORM} N/C N/C

The problem is that the actual response contains a region of zero responses and a region of
non-zero responses as shown in Figure 5-13. In the case of Rit{FORM}, the no convergence
(N/C) is produced because the initial point for the iterations (mean-values point) lies on
the region of zero responses, meaning that a gradient of 0 is obtained and, subsequently, a
no continuation of the iterations. For workflows Rit, the problem is that all the samples to
build the response surface fall in the region of zero responses as observed for percentiles P10,
P50 and P90 in Figure 5-13, producing a response surface with zero coefficients. Hence, for
Rit{Monte Carlo} the response surface will give zero responses for the whole x-space and
thus a probability of failure of 0. For workflows Mod{FORM}, the response surface with zero
coefficients will not allow to the gradient-based search to continue.
Fortunately, for Mod{FORM} a simple solution is to change the initial point of FORM in
the reliability method parameters. The initial point must be located a region of non-zero
responses to work. The tornado plot helps to guide the selection of an initial point. Formally,
a tornado plot for this failure indicator should be used. But this is not possible given that P10
and P90 fall in the region of 0 response in this particular example. A way forward, however,
is to use the tornado plot for maximum subsidence (Figure 5-2). A point that produces a
high value of subsidence must be selected.
Several choices can be made for the initial point recalling that the limit value for failure area is
set Rarea;max = 500m2 . For instance, the point (cm , h, ∆p, R, ν, D) = (0.000213, 119, 27.56,
2800, 0.425, 2038) in the x-space is selected, which is equivalent to (1.28, 1.28, 1.28, 1.28,
1.28, -1.28, -1.28) in the u-space. Note that the direction towards failure for each random
variable is followed, i.e. P90 for the positively correlated variables and P10 for the negatively
correlated variables. The maximum subsidence is 0.33m for this point, implying that it still

Master of Science Thesis RESTRICTED Pablo A. Vásconez

 Evaluation of Workflows for
86 Quasi-Linear Responses

2000

1800 P10 P50 P90

1600

1400

1200
Failure Area

Response Surface
1000
Actual Response
800 Rfai;sub

600

400

200

0
0.E+00 1.E-04 2.E-04 3.E-04 4.E-04 5.E-04

Cm

Figure 5-13: Typical shape of the response for Failure Area

falls in the region of zero responses, because limsub = 0.38m is not exceeded. The next point
is obtained by setting the positively correlated variables to P95 and the negatively correlated
to P10. In the x-space, this point is (0.000226, 125, 28.29, 2874, 0.43, 2038), whereas in the
u-space this represents (1.64, 1.64, 1.64, 1.64, -1.28, -1.28). The failure area for this point is
584m2 , meaning it falls in the region of non-zero responses.

FORM is performed again with the new initial point, obtaining this time convergence after
3 iterations and requiring 21 geomechanical model realisations Nmod . The probability of
failure is 0.0024 and a corresponding accuracy Ap̂f of 0.89. This result provides an enormous
efficiency respect than Monte Carlo, which required 100,000 realisations. This information is
summarized in Table 5-5.

Table 5-5: Efficient workflows for Failure Area Subsidence

Workflow Probability of Accuracy Number of

Failure probability of model realizations
(pf ) of failure
(Ap̂f ) Nmod
Mod{Monte Carlo} 0.002188 1 100,000
Mod{FORM} 0.002434 0.89 21

Unfortunately, it is not possible to apply a simple solution to the workflows using threshold,
not even for Rit{FORM}. To select a different initial point for the iterations it does not solve
the issue, given that the response surface has zero coefficients. The only possible solution,
which is out of the scope of this thesis, is to propose a procedure to change the initial sampling
that currently is fixed at P10, P50 and P90.

The final conclusion for Failure Area Subsidence, is that only workflow than Mod{FORM}
can be used efficiently, provided that the initial point is set in the region of non zero responses
and considering linear or quasi-linear problems.

Pablo A. Vásconez RESTRICTED Master of Science Thesis

5-5 Evaluation for Shear with Homogeneous Subsurface 87

5-5 Evaluation for Shear with Homogeneous Subsurface

This section presents the evaluation of selected workflows for the second model with quasi-
linear responses. This is the case of shear failure in homogeneous subsurface. The selected
workflows are Rit{Monte Carlo}, Rit{FORM} and Mod{FORM}, because they resulted to
have potential to compute the reliability of failure for Subsidence Section 5-4-1.

5-5-1 Description of the model EWG02

The model EWG02 represents a depleting reservoir and serves to compute the shear capacity
utilisation SCU on the corner nearby the corner of the reservoir (see more details in Section 4-
2), presented by the red dotted line in Figure 5-14. The characteristics of the bases case, i.e.
the mean values of the distribution are written also in this figure. Compared to the model
EWG, there are two additional parameters the friction angle phi and the cohesion c, used
for the computation of the strength (maximum shear stress according to the Mohr-Columb
failure criterion) that a point of the subsurface can sustain.

Surface

D=2000m Grid for

outputs

R=2540m
Δp = 10 MPa h=100m

ν =0.25
cm=170E-06 MPa-1
Reservoir
φ = 33ο
c = 3 MPa

Figure 5-14: Sketch of the model EWG02 with properties of the base case

The probabilistic distributions used for this example are given in Table 5-6. Note that the
parameters that have to do with geometry, i.e. D, R and h are not considered as random
variables and subsequently do not appear in this table.
The failure indicators for which the evaluations are made are Maximum SCU and Average
SCU. The limit values selected for them are Rmax;SCU = 9.2 to obtain a probability of failure
of 0.00164 for Maximum SCU and Rave;SCU = 0.75 to obtain a probability of failure of 0.00219
for Average SCU.

5-5-2 Sensitivities with the Tornado Plot

The Tornado plot from Figure 5-15is built up for model EWG02 in order to comment about
the most influential variables and the non-linearity of the problem. Regarding sensitivity, the

Master of Science Thesis RESTRICTED Pablo A. Vásconez

 Evaluation of Workflows for
88 Quasi-Linear Responses

Table 5-6: Distribution of the random variables for model EWG02

Random Variables Distribution Type Mean Standard Deviation Min. Max. COV
∆p [MPa] Normal 25 2 - - 0.08
ν [-] Normal 0.40 0.02 - - 0.05
cm [MPa−1 ] Normal 0.000170 0.000034 - - 0.20
φ [o ] Truncated Normal 33 3 0.001 50 0.09
c [MPa] Truncated Normal 3 0.3 0.001 10 0.10

random variables ranked from most to least influential for Maximum SCU as: ν, ∆p, φ, c,
cm . Actually, the compaction coefficient cm does not have any influence in the value of SCU.
Regarding non-linearity, an almost equal variation of the response corresponds to the same
variation of the random variables (i.e. 1.28σi ) in opposite directions. A higher non-linearity
it is observed for variables φ and c, but it is still limited.

∆p
Random Variables

P10 - P50
cm P50 - P90
4.4 4.8 5.2 5.6 6 6.4 6.8
Maximum SCU [-]

Figure 5-15: Tornado plot for Maximum SCU using the sampling for response surface FOD

5-5-3 Results and Discussion for the Selected Workflows

In this section, the results from the evaluation of the workflows Rit{Monte Carlo}, Rit{FORM}
and Mod{FORM} are presented for the indicator of Maximum SCU. The example analysed
is EWG02, described in the previous section. The observations and conclusions for maximum
subsidence are also applicable to Average SCU. All the results for average subsidence are pre-
sented in the Section B-1, but the summary of relevant results can be observed in Figure 5-18
and Figure 5-19.
Figure 5-16 shows the probability of failure and the corresponding accuracies vs. the initial
thresholds for the evaluation of workflows Rit{Monte Carlo} for Maximum SCU.
Note that the results deserve the same observations that those made for Maximum Subsidence
for model EWG, the most relevant are: the peak-residual relationship between the accuracies
on the probability of failure and the initial threshold th; ini and that most of the workflows
underestimate the probability of failure. An example of the peak-residual relationship occurs

Pablo A. Vásconez RESTRICTED Master of Science Thesis

5-5 Evaluation for Shear with Homogeneous Subsurface 89

(a)
2.50
FO10
Probability of Failure, pf [10−3]

FO90
2.00 FOD
SO
ED
1.50 exact

1.00

0.50

0.00
0.00 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 10.00
Initial Threshold Maximum SCU [-]

(b)
1.00
Accuracy Probability of Failure, Ap̂f

FO10
0.90 FO90
0.80 FOD
SO
0.70 ED
0.60
0.50
0.40
0.30
0.20
0.10
0.00
0.00 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 10.00
Initial Threshold Maximum SCU [-]

Figure 5-16: Performance of workflow Rit{Monte Carlo} for Maximum SCU of example EWG02.
(a) Probability of Failure pf (b) Accuracy of the probability of failure Apˆf

Master of Science Thesis RESTRICTED Pablo A. Vásconez

 Evaluation of Workflows for
90 Quasi-Linear Responses

for the response surface FO90, in which the accuracies rise from 0.20 to 0.90 and then it
diminishes again to 0.50 when increasing th; ini from 0.70 to 3.00. The explanation for this
behaviour the reader is referred to Figure 5-4 in Section 5-4-3. For other response surfaces,
it appears that a smaller value of th;ini should have been used to obtain the peak accuracy.
Regarding the underestimation, it is observed that for all the response surfaces the probability
of failure is lower than the exact value of the probability of failure, with the only exception
of response surface FO90 with th;ini = 1.2.
An additional observation from Figure 5-16 is that there is a reduction in the number of
combinations of response surface and initial threshold that are capable to provide high accu-
racies. The most likely reason for this may lie in the increase of non-linearity observed for
parameters φ and c, which was limited until the P90 values, but recall than further from P90
the importance of the non-linearity will increase.
In Figure 5-17, it is observed that the workflows Rit{FORM} for all combinations of response
surfaces and initial thresholds investigated, obtained high accuracies (more than 0.80). This
suggests that the quasi-linearity in the initial sampled domain, produced that all the response
surfaces were initially the same and were capable to follow a similar path to the design point.

(a)
2.50
FO10
Probability of Failure, pf [10−3]

FO90
2.00 SO
ED
exact
1.50

1.00

0.50

0.00
0.00 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 10.00
Initial Threshold Maximum SCU [-]

(b)
1.00
Accuracy Probability of Failure, Ap̂f

FO10
0.90 FO90
0.80 SO
ED
0.70
0.60
0.50
0.40
0.30
0.20
0.10
0.00
0.00 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 10.00
Initial Threshold Maximum SCU [-]

Figure 5-17: (a) Probability of Failure pf (b) Accuracy of the probability of failure Apˆf .
Workflow: Rit{FORM}

For both Rit{Monte Carlo} and Rit{FORM}, the number of realisations are small (see Fig-
ure 5-19), even sligthly smaller that for the cases of maximum and average subsidence of

Pablo A. Vásconez RESTRICTED Master of Science Thesis

5-6 Summary of evaluations for quasi-linear examples 91

model EWG. The main reason for is that number of random variables are 5 instead of 6,
requiring less samples to build up the initial response surfaces. Despite the rather small re-
duction of number of random variables, there is a high reduction for response surface SO,
because of the full factorial design that requires 2n samples.
Workflow Mod{FORM} proved to be accurate and relatively efficient as well. It obtained
a probability of failure of 0.00172, corresponding to an accuracy of 0.93. The number of
realisations required are 30 (Figure 5-18 and Figure 5-19).
The important conclusions for this section is that the selected workflows still have the poten-
tial to compute the probability of failure accurately and efficiently. Mod{FORM} still can
be applied with the default values. Workflows Rit{FORM} worked well for almost all the
combinations of response surface and initial threshold, but Rit{Monte Carlo} requires the
proper selection of the combination of response surface and initial threshold.

5-6 Summary of evaluations for quasi-linear examples

Figure 5-18 and Figure 5-19 summarize the accuracies and the number of model realisations
obtained by Mod{FORM}, Rit{Monte Carlo} and Rit{FORM} for the indicators of maxi-
mum, average and failure area of examples regarding subsidence (EWG) and shear failure
(EWG02).
Regarding the accuracy on the probability of failure, all the workflows are capable to provide
high accuracies. Worklow Mod{FORM} obtained values of Apf ranging for 0.80 to 0.93
for all the examples evaluated including EWG Failure Area Subsidence. The accuracies for
workflows Rit{FORM} ranged from Apf = 0.68 to 1.0 as peak accuracies and 0.65 to 0.90
as residual accuracies, suggesting that there are a wide range of combinations of response
surface and initial threshold that can be chosen to obtain an accurate value of probability of
failure. Workflows Rit{Monte Carlo} obtained accuracies Apf between 0.45 to 1.0 as peak
accuracies and between 0.24 to 0.98 as residual accuracies. Clearly, the latter workflows are
more sensitive to the selection of initial threshold and response surface, but there are still
quite a number of options that make more likely their success.
The example EWG for Failure Area Subsidence was successfully computed by Mod{FORM}.
The reason lies on the shape of the actual response for this failure indicator, which is divided
in a region of zero responses and a region of non-zero responses. For Mod{FORM}, the
procedure can be started for any point in the domain, allowing to select one from the region
of non-zero responses and obtaining a result for the probability of failure.
Regarding number of model realisations, Rit{Monte Carlo} and Rit{FORM} are more efficient
than Mod{FORM} for the first-order response surfaces. For cases with 5 to 6 random variables
(EWG02 and EWG, respectively), workflows Rit required only between 9 to 23 realisations,
compared to the 21 to 30 realisations that the workflow Mod{FORM} required. Response
surface SO used 37 to 75 realisations to compute the probability of failure, being less efficient
than Mod{FORM}. In any case, the number of realisations is just a small fraction of the
around 60,000 samples that Mod{Monte Carlo} took.
Therefore, all the workflows shown in Figure 5-18 and Figure 5-19 have the potential to
compute accurately and efficiently the probability of failure.

Master of Science Thesis RESTRICTED Pablo A. Vásconez

 Evaluation of Workflows for
92 Quasi-Linear Responses

FO10 FO90 FOD SO ED Mod{FORM}

Workflow Rit{Monte Carlo} Workflow Rit{FORM}
1.00
Peak accuracies Apf

0.80

0.60

0.40

0.20

0.00
1.00
Residual accuracies Apf

0.80

0.60

0.40

0.20

0.00
Max.Sub Ave.Sub Fai.Sub Max.She Ave.She Max.Sub Ave.Sub Fai.Sub Max.She Ave.She
EWG EWG02 EWG EWG02

Figure 5-18: Peak and residual accuracies on the probability of failure obtained in the evaluation
of quasi-linear examples (EWG and EWG02). Failure indicators of maximum (Max), average
(Ave) and Failure Area (Fai) are presented. Workflows Rit for different response surfaces are
represented by bars and workflow Mod{FORM} by dashed lines.

Pablo A. Vásconez RESTRICTED Master of Science Thesis

5-6 Summary of evaluations for quasi-linear examples 93

FO10 FO90 FOD SO ED Mod{FORM}

Workflow Rit{Monte Carlo} Workflow Rit{FORM}
80
Nmod for peak accuracies

70
60
50
40
30
20
10
0
80
Nmod for peak accuracies

70
60
50
40
30
20
10
0
Max.Sub Ave.Sub Fai.Sub Max.She Ave.She Max.Sub Ave.Sub Fai.Sub Max.She Ave.She
EWG EWG02 EWG EWG02

Figure 5-19: Number of geomechanical model realisations Nmod for the peak and residual
accuracies on the probability of failure obtained in the evaluation of quasi-linear examples (EWG
and EWG02). Failure indicators of maximum (Max) and average (Ave) are presented. Workflows
Rit for different response surfaces are represented by bars and workflow Mod{FORM} by dashed
lines.

Master of Science Thesis RESTRICTED Pablo A. Vásconez

 Evaluation of Workflows for
94 Quasi-Linear Responses

5-7 Conclusions

The conclusions obtained after the evaluation of the workflows for quasi-linear responses are
the following:

• Workflow Mod{FORM} is capable to compute efficiently and accurately the probability

of failure for all the quasi-linear examples evaluated.

• Workflows that use the response surface and the geomechanical model Rit according
to the threshold range, have the potential to obtain efficiently good accuracies on the
probability of failure for quasi-linear responses (e.g. maximum subsidence and average
subsidence). The successful reliability methods within this workflow are Monte Carlo
or FORM.

• The accuracy of the worfklows that use the response surface and the geomechanical
model Rit according to the threshold range may fail is the initial threshold is not selected
properly. This means that these workflows must be improved in order to ease its use. In
the meanwhile, for the examples analysed, the most appropriate selections are response
surfaces FO10 and FO90 with initial threshold around 10% of the limit value.

• There is a strong tendency for workflows Rit to underestimate the probability of failure.
For a few initial thresholds, response surfaces FOD and SO overestimate the probability
of failure. Response surfaces FO10, FO90 and ED always underestimated the probability
of failure.

• For the failure area indicator, the only workflow capable compute the probability of
failure is Mod{FORM}. The reason is that this response has a region of zero responses
and a region of non-zero responses. Other methods should be study in order to have
more alternatives for this failure indicator.

• The second-order response surface proposed in this study is not more accurate and
significant less efficient that the first order response surfaces. The initial sampling must
be reduced in order to reduced its computation time.

• The proposed implementation of Importance Sampling and Directional Sampling do

not lead to efficient computation of the reliability. They either take too much computa-
tion time (Mod ) or have low accuracy (Res, Rit). For importance sampling, a further
complication is the decision about the variance factor, which must require the trial of
different variance factors increasing even more the computation time.

• The workflows using only the response surface Res produce very inaccurate results for
the levels of probability of failure adopted in this study. This applies even for quasi-
linear problems.

Pablo A. Vásconez RESTRICTED Master of Science Thesis

 Chapter 6

Evaluation of Workflows for

Non-Linear Responses

6-1 Introduction

In the previous chapter, the workflows or procedures to compute the reliability proposed in
this study, were evaluated under 2 examples from the oil and gas industry: subsidence and
shear due to a depletion of a reservoir. Under the geomechanical model used (Geerstma) these
example produced the so-called quasi-linear responses, or in other words, responses that are
fairly linear for the most probable region of the parameter space, but present slight non-
linearity for regions at the tails of the distribution. For these examples, workflows Res were
inaccurate and the proposed RitImportance Sampling and RitDirectional Sampling were inef-
ficient. Workflows Rit{Monte Carlo}, Rit{FORM} and Mod{FORM} proved to be accurate
and efficient for the mentioned examples.
In this chapter, the applicability of these workflows is under evaluation for examples that
already show non-linearity in the most likely region of the parameter space. The two examples
are related with the subsidence due to a depletion of a reservoir, but using a more advanced
geomechanical model, implemented in the software Quickblocks. One example considers a
homogeneous subsurface and the other a layered subsurface. Both produce an increase of
non-linearity of the actual response, but in the case with layered subsurface the non-linearity
is higher.
After obtaining the results for the non-linear examples, the effect of the target probability
of failure is studied. It is performed in order to highlight that the high or low accuracies
mentioned in this study are valid for target probability of failure of around 0.002, used in all
the examples.
Finally, in view of the results for examples with quasi-linear responses and with non-linear
responses, a reason to explain any low accuracies of the workflows will be given.
The objectives of the chapter are:

Master of Science Thesis RESTRICTED Pablo A. Vásconez

96 Evaluation of Workflows for Non-Linear Responses

1. Evaluate the workflows Rit{Monte Carlo}, Rit{FORM} and Mod{FORM} for exam-
ples with non-linear responses. The selected examples addressed: (1) subsidence in
homogeneous subsurface and (2) subsidence in layered subsurface.

2. Re-evaluate the workflows for the subsidence in layered subsurface for a higher target
of probability of failure.

3. Identify the reason for any low accuracies obtained in an example of non-linear response
(if it occurs or can be foreseen).

6-2 Evaluation for Subsidence with Homogeneous Subsurface

This section presents the evaluation for the first of the two non-linear responses analysed
in this chapter. This is the case of subsidence in homogeneous subsurface analysed in the
Chapter 5. The difference is that another geomechanical model is used (Quickblocks), which
increases the non-linearity. First, the example is described, including the probabilistic proper-
ties and limit values. Then the results obtained for the workflows are analysed and discussed.

6-2-1 Description of the example EWQ01

The example named EWQ01 (Evaluation of Workflows for Quickblocks 01), for the purposes of
this study, models the depletion of a reservoir. The subsurface is homogeneous, meaning that
both the reservoir, the upper and the lower layers have the same geomechanical properties.
The base case, i.e. the case with the mean values for each random variable is presented in
Figure 6-1. Note that the shape of the reservoir is changed to a parallelepiped (previously
it was a cylinder), but its thickness and volume are the same. Also, the upper and lower
layers are drawn, but just for reference, because the software requires the input of each layer
separately. But in essence, the example EWQ01 is the same as example EWG.

Grid for results

z1=0m

Upper: R=2540m

m

E=2745 MPa, ν=0.25

0

Δp= -25 MPa

50
4
y =

Reservoir: z2=-2000m x = 4500m

E=2745 MPa, ν=0.25 h=100m

Lower:
E=2745 MPa, ν=0.25
z3=-8000m

Figure 6-1: Sketch of the example EWQ01

The probabilistic distributions used for this example are given in Table 6-1. It is noticed
that the geomechanical parameters have the same distribution for all layers and that they

Pablo A. Vásconez RESTRICTED Master of Science Thesis

6-2 Evaluation for Subsidence with Homogeneous Subsurface 97

are (almost) fully correlated, which is expressed by the ρ = 0.99. This means that for any
realisation, the geomechanical parameters of each layer will be exactly the same value.

Table 6-1: Distributions of the Random Variables for example EWQ01

Parameter Distribution Mean Standard COV Correlation

Deviation ρ with Parameters
Upper.E [MPa] Normal 2745 549 0.20 0.99 Reservoir.E, Lower.E
Upper.ν [-] Normal 0.400 0.020 0.05 0.99 Reservoir.nu, Lower.nu
z2 [m] Normal 2000 30 0.02 - -
Reservoir.E [MPa] Normal 2745.000 549.000 0.20 0.99 Upper.E, Lower.E
Reservoir.ν [-] Normal 0 0 0.05 0.99 Upper.nu, Lower.nu
Reservoir.h [m] Normal 100.000 15.000 0.15 - -
Lower.E [MPa] Normal 2745 549 0.20 0.99 Upper.E, Reservoir.E
Lower.ν [-] Normal 0.400 0.020 0.05 0.99 Upper.nu, Reservoir.nu
Reservoir.x [m] Normal 4500 360 0.08 - -
Reservoir.y [m] Normal 4500.000 360.000 0.08 - -
∆p [Mpa] Normal -25 2 0.08 - -

The relevant difference compared with the sample EWG, is that the elastic modulus E is used
instead of the compaction coefficient cm . A coefficient of variation COV = 0.20 is assigned to
E as an attempt to keep the same uncertainty level that the model EWG has. In despite of
this, as observed in Section 6-2-2, the non-linearity of the response increased, producing an
example suitable for this chapter.
Because this non-linearity increased, it is necessary to increase the limit values to Rmax;sub =
0.55m to obtain a value of 0.00145 of probability of failure for maximum subsidence and to
Rave;sub = 0.42m to obtain a value of 0.001901 for average subsidence.

6-2-2 Sensitivities with Tornado Plot

The sampling corresponding to response surface FOD is performed to build up the Tornado
Plot for maximum (Figure 6-2). The main observation is that in both plots the elastic
modulus E presents a non-linear relation with the response between the range P 10 − P 90
of its distribution, since for the same change in E a difference change in maximum and
average subsidence is obtained. The most influential variable is the elastic modulus of the
reservoir Reservoir.E followed by the elastic modulus of the reservoir. This implies that the
non-linearity has an important effect on the response.
The increase in non-linearity is driven by the change of geomechanical parameters that the
two different software considers as an input. The geomechanical parameters for Geertsma are
the Poisson’s ratio ν and the compaction coefficient cm , whereas for Quickblocks are ν and
E. These three parameters are related by the equation:

(1 + ν)(1 − 2ν)
cm = (6-1)
E(1 − ν)

While, for a deterministic computation the use of one or other software would provide the same
result, for probabilistic purposes this does not hold. Recall that the maximum subsidence
can be computed from the Eq. (5-2), which means that a linear relation holds between the

Master of Science Thesis RESTRICTED Pablo A. Vásconez

98 Evaluation of Workflows for Non-Linear Responses

Reservoir.E
Reservoir.h
Lower.E

Random Variables
Reservoir.ν
∆p
Upper.E
Reservoir.y
Reservoir.x
Upper.ν
Lower.ν
P10 - P50
z2 P50 - P90
0.12 0.14 0.16 0.18 0.2 0.22 0.24 0.26 0.28 0.3
Maximum Subsidence [m]

Figure 6-2: Tornado plot for EWQ01 using sampling from response surface FOD

response if only cm is varied and slightly non-linear if several parameters are varied Geertsma
(1973). Substituting Eq. (6-1) into Eq. (6-2), the following equation expresses the relation
between the maximum subsidence and its parameters:

!
(1 + ν)(1 − 2ν) D/R
Lsub (x) = 2 ∗ cm ∆p ∗ h 1 − p (6-2)
E 1 + (D/R)2

Clearly, to vary E produces a non-linear response of the maximum subsidence.

From this relevant observation, it is concluded that the non-linearity of a certain response
of interest is not only induced by the mechanical non-linearity, i.e. by a non-linear relation
between the stress and strains.

6-2-3 Results and Discussion of selected workflows

In this section, the results from the evaluation of the workflows Rit{Monte Carlo}, Rit{FORM}
and Mod{FORM} are presented for the indicator of maximum subsidence. The example
analysed is EWQ01, described in the previous section. The observations and conclusions for
maximum subsidence are also applicable to average subsidence. All the results for average
subsidence are presented in the Section B-2, but the summary of relevant results can be
observed in Figure 6-9 and Figure 6-10.
Figure 6-3 shows the probability of failure and accuracies vs. the initial thresholds for the
evaluation of workflows Rit{Monte Carlo} for Maximum Subsidence.
The two main findings are the reduction of the accuracies (compared to the quasi-linear
examples) and the increase of the initial threshold to obtain the peak accuracy. The reduction
of the accuracies is in amount and in level: there is less amount of combinations between
response surfaces and initial thresholds that can obtain good accuracies and also the overall
peak accuracy is 0.80. Response surfaces SO and ED do not reach a good accuracy for any
value of initial threshold, whereas FO10, FO90 and FOD do so for a reduced number of
initial thresholds, producing that only one of the used values of th;ini worked in each case.

Pablo A. Vásconez RESTRICTED Master of Science Thesis

6-2 Evaluation for Subsidence with Homogeneous Subsurface 99

(a)
4.50
FO10
4.00
Probability of Failure, pf [10−3]

FO90
3.50 FOD
SO
3.00 ED
exact
2.50
2.00
1.50
1.00
0.50
0.00
0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.55
Initial Threshold Maximum Subsidence [m]

(b)
1.00
Accuracy Probability of Failure, Ap̂f

FO10
0.90 FO90
0.80 FOD
SO
0.70 ED
0.60
0.50
0.40
0.30
0.20
0.10
0.00
0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.55
Initial Threshold Maximum Subsidence [m]

Figure 6-3: (a) Probability of Failure pf (b) Accuracy of the probability of failure Apˆf . Workflow:
Rit{Monte Carlo}

Master of Science Thesis RESTRICTED Pablo A. Vásconez

100 Evaluation of Workflows for Non-Linear Responses

In addition, this portion of the domain is located closer to 50% of the limit value. For the
quasi-linear problems, it was more close to the 10% of the limit value.
For Rit{FORM} (Figure 6-4), the observations about the accuracies and the increase of the
threshold are also valid, with the addition that the peak-residual relation between Apf and
th;ini (found for quasi-linear examples) practically disappear. The peak accuracies still can
reach values of 0.90, but the issue is that this only occurred for response surface FO10 (for a
high portion of the initial threshold domain) and for response surface FOD (for a very small
portion of the initial threshold domain). Response surface, on the contrary did not reach an
accuracy of 0.50 for any value of initial threshold.
(a)
4.50
FO10
4.00
Probability of Failure, pf [10−3]

FO90
3.50 SO
ED
3.00 exact
2.50
2.00
1.50
1.00
0.50
0.00
0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.55
Initial Threshold Maximum Subsidence [m]

(b)
1.00
Accuracy Probability of Failure, Ap̂f

FO10
0.90 FO90
0.80 SO
ED
0.70
0.60
0.50
0.40
0.30
0.20
0.10
0.00
0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.55
Initial Threshold Maximum Subsidence [m]

Figure 6-4: (a) Probability of Failure pf (b) Accuracy of the probability of failure Apˆf . Workflow:
Rit{FORM}

From the analysis of the previous observation, it follows that there is a high contrast between
the results of response surface FO10 and FO90, which was not observed for quasi linear
problems. The higher accuracy of FO10 owes to the fact that for this particular example, the
parameters with more influence are negatively correlated with the response, i.e. low values of
the parameters area closer to failure.
For both Rit{Monte Carlo} and Rit{FORM}, the number of realisations are small, but higher
than for the quasi linear problems (Figure 6-4(b)). This comes to the fact that this model
uses 11 random variables, which implies a higher amount of initial sampling (Table 6-1).
Also Rit{Monte Carlo} and Rit{FORM}, there is a significant majority of cases in which the

Pablo A. Vásconez RESTRICTED Master of Science Thesis

6-3 Evaluation for Subsidence with Layered Subsurface 101

probability is underestimated. Most of the cases that overestimate the probability of failure
occur when the response surface FOD is used and very few cases for response FO10 and FO90.
Workflow Mod{FORM} proved to be accurate and relatively efficient. It obtained a proba-
bility of failure of 0.001737, corresponding to an accuracy of 0.80. The number of realisations
required are 72 (Figure 6-9 and Figure 6-10).
The conclusions for this section is that the selected workflows have the potential to compute
the probability of failure accurately and efficiently. Mod{FORM} can be applied with the
default values. On the contrary, for Rit the selection of response surface and initial threshold
becomes crucial, aspect that was of less importance for quasi-linear problems.

6-3 Evaluation for Subsidence with Layered Subsurface

This section presents the evaluation for the second example with a non-linear response anal-
ysed in this chapter. It is a case of subsidence due to reservoir depletion for a layered subsur-
face. It introduces the contrast between the geomechanical parameters of the layers, because
they are not correlated anymore. First, the example is described, including the probabilistic
distributions of the geomechanical parameters and limit values. Then the results obtained for
the workflows are analysed and discussed.

6-3-1 Description of the model EWQ02

The example presented in Figure 6-5, it is named EWQ02 in this study. It models the depletion
of a reservoir with a shape of a parallelepiped of 113m of thickness and 4500m times 4500m
in area. The subsurface is characterised by 6 layers, whose thickness and properties for the
base case (i.e. the mean values of each random variable) are shown in Figure 6-5 .

Grid for results

z = -1002m

Upper1:
E=2137 MPa, ν=0.398 z = -1986m
Upper2:
E=2474 MPa, ν=0.408
z = -2529m

0m

Upper3: Δp= -25 MPa

50
4

E=2744 MPa, ν=0.356

y =

Reservoir: z = -2768m x = 4500m

E=2932 MPa, ν=0.284
z = -2881m
Lower1:
E=3645 MPa, ν=0.330

z = -2942m
Lower2:
E=3645 MPa, ν=0.330
z = -10000m

Figure 6-5: Sketch of the example EWQ02

Master of Science Thesis RESTRICTED Pablo A. Vásconez

102 Evaluation of Workflows for Non-Linear Responses

The probabilistic distributions used for this example are given in Table 6-2. Note that the
random variables correspond to the geomechanical properties E and ν for each layer and also
the change in pressure ∆p. Geometry is not selected as random variable for this example.
Having 6 layers, a total of 13 random variables are available in this model.

Table 6-2: Distributions of the Random Variables for example EWQ02

Parameter Distribution Mean Standard Min Max COV

Type Deviation
Upper1.E [MPa] Normal 2138 186 - - 0.09
Upper1.ν [-] Truncated Normal 0.398 0.014 0.366 0.421 0.03
Upper2.E [MPa] Normal 2474 272 - - 0.11
Upper2.ν [-] Truncated Normal 0.408 0.023 0.289 0.449 0.06
Upper3.E [MPa] Normal 2744 415 - - 0.15
Upper3.ν [-] Truncated Normal 0.356 0.055 0.125 0.433 0.16
Reservoir.E [MPa] Normal 2933 541 - - 0.18
Reservoir.ν [-] Truncated Normal 0.284 0.063 0.001 0.428 0.22
Lower1.E [MPa] Normal 3465 31 - - 0.01
Lower1.ν [-] Truncated Normal 0.331 0.012 0.300 0.364 0.04
Lower2.E [MPa] Normal 3465 31 - - 0.01
Lower2.ν [-] Truncated Normal 0.331 0.012 0.300 0.364 0.04
∆p [MPa] Normal -25 2 - - 0.08

The limit values selected are Rmax;sub = 1.20m to obtain a value of 0.00185 of probability
of failure for maximum subsidence and to Rave;sub = 0.87m to obtain a value of 0.00196 for
average subsidence.

6-3-2 Sensitivities with Tornado Plot

The sampling corresponding to response surface FOD is performed to build up the Tornado
Plot for maximum subsidence (Figure 6-6).
The first observation is that in both plots of Figure 6-6, the only relevant layer is the one
containing the reservoir. Therefore, the elastic modulus E and the Poisson’s ratio ν from this
layer are the most influential random variables of this example. The next random variable of
importance is the change in pressure ∆p, which also occurs within the reservoir. Therefore,
it can be concluded that for this particular example, what occurs in the reservoir is what
determines the probability of failure.
The other observation about the Tornado plots, is the non-linear relation that both E and
ν have with the average and maximum subsidence between the range P 10 − P 90 of their
distributions, since the same change in E or ν produces a different change in maximum and
average subsidence.

6-3-3 Results and Discussion for selected workflows

In this section, the results from the evaluation of the workflows Rit{Monte Carlo}, Rit{FORM}
and Mod{FORM} are presented for maximum subsidence. The example analysed is EWQ02,
described in the previous section. The observations and conclusions for maximum subsidence
are also applicable to average subsidence. All the results for average subsidence are presented

Pablo A. Vásconez RESTRICTED Master of Science Thesis

6-3 Evaluation for Subsidence with Layered Subsurface 103

Reservoir.E
Reservoir.ν
∆p
Random Variables Upper1.ν
Upper2.E
Lower2.ν
Upper3.E
Upper2.ν
Lower2.E
Upper1.E
Upper3.ν
Lower1.ν
P10 - P50
Lower1.E P50 - P90
0.3 0.34 0.38 0.42 0.46 0.5 0.54 0.58 0.62
Maximum Subsidence [m]

Figure 6-6: Tornado plot for EWQ02 using sampling from response surface FOD

in the Section B-3, but the summary of relevant results can be observed in Figure 6-9 and
Figure 6-10.
Figure 6-7 shows the probability of failure and accuracies vs. the initial thresholds for the
evaluation of workflows Rit{Monte Carlo} for Maximum Subsidence.
The most relevant observation is that there is a reduction on the accuracies. Only for one
combination of response surface and initial threshold a good accuracy is obtained. This is the
case of response surface FOD and th;ini = 0.30m. All the rest of response surfaces produced
mostly very low accuracies.
Another observation is the increased amount of overestimation of the probability of failure
compared to the quasi-linear examples. Response surface FO10 overestimates the probability
of failure for very small, small and big values of initial threshold th;ini . What occurs, regardless
the initial threshold, is the situation depicted in Figure 5-4(c), the initial underestimation of
the actual response forces to take a sample far the mean-values point, which produces a
high value of actual response due to the non-linearity, leading to a new response surface
with overestimation for an important region of the x-space. The reason why for response
surface FO10 overestimate regardless the initial threshold is that the initial response surface
underestimate in a great extent, therefore even when using high th;ini , the threshold th reduces
at a higher pace of which the response surface can reduce the underestimation. This confirms
that a some point in the process of updates, the situation of Figure 5-4(c) occurs, producing
a strong overestimation instead that cannot be corrected fully with subsequent sampling.
For response surface FO90 there is an important difference: for medium high values of th;ini
there could be under or overestimation of the probability of failure. The reasons are the
degree of underestimation of the actual response and the locations of subsequent sampling.
The degree of underestimation for the initial response surface FO90 is higher than for response
surface FO10, because the former is built up from samples coming from P 50 − P 90 values
of E and ν of the reservoir, which are in the opposite direction towards the limit state. The
locations of the subsequent sampling can be close to the mean values, producing a higher
degree of underestimation of the actual response, reducing the probability that an update

Master of Science Thesis RESTRICTED Pablo A. Vásconez

104 Evaluation of Workflows for Non-Linear Responses

(a)
4.50
FO10
4.00
Probability of Failure, pf [10−3]

FO90
3.50 FOD
SO
3.00 ED
exact
2.50
2.00
1.50
1.00
0.50
0.00
0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 1.10 1.20
Initial Threshold Maximum Subsidence [m]

(b)
1.00
Accuracy Probability of Failure, Ap̂f

FO10
0.90 FO90
0.80 FOD
SO
0.70 ED
0.60
0.50
0.40
0.30
0.20
0.10
0.00
0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 1.10 1.20
Initial Threshold Maximum Subsidence [m]

Figure 6-7: (a) Probability of Failure pf (b) Accuracy of the probability of failure Apˆf . Workflow:
Rit{Monte Carlo}

Pablo A. Vásconez RESTRICTED Master of Science Thesis

6-3 Evaluation for Subsidence with Layered Subsurface 105

occurs. The probability of failure is then underestimated and also a high value of threshold
remains. The locations of the subsequent sampling can be closer to the limit state, reducing
the underestimation and leading to a response surface similar to the initial one for FO10 and,
as explained in the previous paragraph, end in overestimation of the probability of failure.
Similar reasoning can be extended for response surface FOD and also for response surfaces
SO and ED, which end with relatively high values of threshold.

(a)
4.50
FO10
4.00
Probability of Failure, pf [10−3]

FO90
3.50 SO
ED
3.00 exact
2.50
2.00
1.50
1.00
0.50
0.00
0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 1.10 1.20
Initial Threshold Maximum Subsidence [m]

(b)
1.00
Accuracy Probability of Failure, Ap̂f

FO10
0.90 FO90
0.80 SO
ED
0.70
0.60
0.50
0.40
0.30
0.20
0.10
0.00
0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 1.10 1.20
Initial Threshold Maximum Subsidence [m]

Figure 6-8: (a) Probability of Failure pf (b) Accuracy of the probability of failure Apˆf . Workflow:
Rit{FORM}

For Rit{FORM} (Figure 6-8), the observation is that only good accuracies are obtained
by response surface FO10. The reason is that response surface FO10 is the one with less
underestimation of the response and this is properly corrected during the performance of
the gradient-based search. The other response surfaces are not able to correct the response
surface sufficiently and finally obtain only medium to low accuracies. The gradient-based on
a response surfaces that underestimate the actual response lead to mainly underestimations
of the probability of failure.
For both Rit{Monte Carlo} and Rit{FORM}, the number of realisations are relatively small.
There is a slight increase compared to example EWQ01, mainly driven by the number of
random variables, which requires more sampling to form the initial response surface.
Workflow Mod{FORM} proved to be accurate and relatively efficient. It computed a prob-
ability of failure equal to 0.00189, equivalent to an accuracy of 0.98 and requiring 98 model

Master of Science Thesis RESTRICTED Pablo A. Vásconez

106 Evaluation of Workflows for Non-Linear Responses

realisations. These results are observed in Figure 6-9 and Figure 6-10.
Once more it is observed that the selected workflows have the potential to compute the
probability of failure accurately and efficiently. Mod{FORM} can be applied with the default
values, but for workflows Rit is crucial the selection of the response surface and the initial
threshold.

6-4 Summary of evaluations for examples with non-linear response

Figure 6-9 and Figure 6-10 summarize the accuracies and the number of model realisations
obtained by Rit{Monte Carlo} and Rit{FORM} for the indicators of maximum and average
subsidence for examples EWQ01 and EWQ02. Both the peak and residual accuracies are
presented (recall that the peak accuracy is the maximum accuracy for all the initial the
initial thresholds and the residual accuracy is the accuracy for a high or infinite value of
initial threshold).
Comparing the peak and residual accuracies for workflows Rit{Monte Carlo}, it is clear
that the selection of the response surface and initial threshold is key in the success of these
procedures. For instance, for any of the examples analysed, response surface SO leads to
low accuracies in all cases; or if response surface ED and the appropriate th;ini are selected
for EWQ02 Average Subsidence, an accurate probability of failure is obtained (Apf = 0.83),
but if the residual th;ini is chosen, an inaccurate value of probability of failure is obtained
(Apf = 0.0).
For workflows Rit{FORM}, the most important selection is the response surface. Both resid-
ual and peak accuracies are good for response surface FO10 in all the examples analysed.
For response surface ED the accuracies are also good, with the exception of example EWQ02
Maximum Subsidence. All the other response surfaces provide low accuracies. It is still
necessary to select a th;ini of around 25% of the limit value at least (Figure 6-8).
From the previous observation it follows that workflow Rit{FO10, FORM} is the advisable
to be compute the probability of failure accurately for subsidence, among the proposed work-
flows. As explained before, this owes to the fact the elastic modulus and Poisson’s ratio are
negatively correlated with the subsidence. Nevertheless, care must be taken for lower values
of probability of failure, in which the non-linearity may produce that the response surface
does not update sufficiently.
Regarding number of model realisations for the all the workflows presented in Rit{Monte
Carlo} and Rit{FORM}, it is observed that they are relatively low for the first order response
surfaces (Figure 6-10). For 11 to 13 random variables, as used in the examples, the methods
required only between 17 to 35 realisations. This does not apply for response surface SO,
whose bar goes beyond the upper limit of the plot, requiring more than 2049 for EWQ01 and
more than 8193 for EWQ02. This amount of samples is produced by the 2n full factorial
design to build up the initial response surface (Section 3-3-1 and Section 4-5).
For the non-linear examples EWQ01 and EWQ02, response surface SO cannot be considered
as accurate and efficient compared to the other response surfaces, because is not the most
accurate and requires a high computation time. There could have been the presumption that
this response surface would perform better than the linear response surfaces for these type of
examples (non-linear). This proves to be wrong at least for the sampling technique applied.

Pablo A. Vásconez RESTRICTED Master of Science Thesis

6-4 Summary of evaluations for examples with non-linear response 107

FO10 FO90 FOD SO ED Mod{FORM}

Workflow Rit{Monte Carlo} Workflow Rit{FORM}
1.00
Peak accuracies Apf

0.80

0.60

0.40

0.20

0.00
1.00
Residual accuracies Apf

0.80

0.60

0.40

0.20

0.00
Max.Sub Ave.Sub Max.Sub Ave.Sub Max.Sub Ave.Sub Max.Sub Ave.Sub
EWQ01 EWQ02 EWQ01 EWQ02

Figure 6-9: Peak and residual accuracies on the probability of failure obtained in the evaluation
of examples (EWQ01 and EWQ02) and for failure indicators of maximum (Max.Sub) and average
subsidence (Ave.Sub). Workflows Rit for different response surfaces are represented by bars and
workflow Mod{FORM} by dashed lines.

Master of Science Thesis RESTRICTED Pablo A. Vásconez

108 Evaluation of Workflows for Non-Linear Responses

FO10 FO90 FOD SO ED Mod{FORM}

Workflow Rit{Monte Carlo} Workflow Rit{FORM}
Nmod for residual accuracies Nmod for for peak accuracies

100

80

60

40

20

0
100

80

60

40

20

0
Max.Sub Ave.Sub Max.Sub Ave.Sub Max.Sub Ave.Sub Max.Sub Ave.Sub
EWQ01 EWQ02 EWQ01 EWQ02

Figure 6-10: Number of geomechanical model realisations Nmod for the peak and residual
accuracies on the probability of failure for the evaluation of examples with non-linear responses
(EWQ01 and EWQ02). Workflows Rit for different response surfaces are represented by bars and
workflow Mod{FORM} by dashed lines.

Pablo A. Vásconez RESTRICTED Master of Science Thesis

6-5 Influence of the target probability of failure on the accuracy 109

Figure 6-9 and Figure 6-10 show that Mod{FORM} can be used to obtain an accurate and
efficient result. The accuracies ranged between 0.80 to 0.93 and the number of realisations
range between 72 to 98. The last number start to appear high, but note that the example
EWQ01 considers 11 random variables already. The number of realisations could be reduced
by not selecting as random variables the least influential parameters.
Rit{FORM} shows a significant advantage compared to Mod{FORM} in efficiency. When
the response surface is used and updated according to the threshold methodology, the number
of model realisations is reduced by a factor of 3 to 5. For instance, Rit{FORM} required only
22 realisations versus the 98 realisations of Mod{FORM}.

6-5 Influence of the target probability of failure on the accuracy

So far in this chapter, it has been proved that the workflows using response surface and
updates according to the threshold methodology (Rit) have the potential to provide accurate
and efficient results. Nevertheless, there is the inconvenience to find the appropriate response
surface and initial threshold th;ini to make these procedures to work. This is a problem that
requires further attention.
This section highlights that the performance of the workflows in a non-linear problem is also
a function of the target probability of failure. In order to show this, the example EWQ02 is
used exactly as described in Section 6-3-1 and with the probabilistic properties from Table 6-
2, but this time for a limit for maximum subsidence of Rmax;sub = 0.60m. This results in a
probability of failure of pf = 0.12345 by applying Mod{Monte Carlo}, corresponding to an
increase by a factor of 50 times the target of probability of failure used so far in this study,
i.e. pf = 0.002.
The comparison between the accuracies obtained for the two different targets of probability of
failure shows a dramatic increase for the case with pf = 0.12435 (Figure 6-11). For workflows
Rit{Monte Carlo}, the peak accuracies are around 0.90 for the latter case, regardless the
response surface, whereas a peak accuracy of 0.80 is only reached by response surface FOD
for the case with pf ≈ 0.002. The residual accuracies are around 0.80 for pf = 0.12435 and
all response surfaces, whereas for pf ≈ 0.002 they are all lower than 0.20. The comparison of
Rit{FORM} only differs in that response surface FOD results in a good accuracy of around
0.80.
The reason for the increase in performance for the case with pf = 0.12435 is that the design
point is closer to mean values. When the design point is closer to the mean values, the initial
response surface is able to predict better the design the point, being relatively accurate for
pf = 0.12435. For the case with pf = 0.002, the initial response surface is very inaccurate,
lowering the accuracy even after several updates.

6-6 Limitation of the workflows Rit

The evaluations performed in this study, especially in this chapter for the so-called non-
linear responses, leads to the conclusion that the application of the workflows Rit has a the
limitation to select the proper combination of response surface and initial threshold to ensure

Master of Science Thesis RESTRICTED Pablo A. Vásconez

110 Evaluation of Workflows for Non-Linear Responses

FO10 FO90 FOD SO ED

Workflow Rit{Monte Carlo} Workflow Rit{FORM}
1.00
Mod{FORM}
Peak accuracies Apf

0.80

0.60

0.40

0.20

0.00
1.00
Residual accuracies Apf

0.80

0.60

0.40

0.20

0.00
pf = 0.002 pf = 0.12 pf = 0.002 pf = 0.12
EWQ02 EWQ02 EWQ02 EWQ02

Figure 6-11: Peak and residual accuracies on the probability of failure obtained in the evaluation
of examples EWQ for two different targets of probability of failure pf = 0.002 and 0.12. Workflows
Rit for different response surfaces are represented by bars and workflow Mod{FORM} by dashed
lines.

an accurate result. The best recommendation that can be given for examples similar to those
evaluated in this study is to choose the response surface FO10 with an initial threshold of at
least 30% of the limit value and FORM as the reliability method.
Furthermore, the limitation of obtaining inaccurate results if a proper combination of re-
sponse surface and initial threshold is selected, can get to the point that inaccurate results
are obtained regardless the combination of response surface and initial threshold selected.
The latter case did not occur for any of the examples evaluated. Nevertheless, departing
from the finding that the target probability of failure affects the accuracy (Section 6-5), it
can be foreseen that if a probability of failure lower than pf = 0.002 is targeted for the ex-
ample EWQ02, all the combination of response surfaces and initial thresholds will obtain low
accuracies.
It is important to identify which part of the procedure is limiting the acquisition of good
results. Three are the main candidates:

1. Order of the polynomial or function type used for fitting the initial sampling.
2. Sampling technique
3. Procedure to update the threshold and hence the response surface.

It has been observed before that first-order response surfaces provide more accurate results
than second-order for the particular sampling and examples used in this thesis, thus the first
point main may small impact on the workflow.

Pablo A. Vásconez RESTRICTED Master of Science Thesis

6-6 Limitation of the workflows Rit 111

The sampling technique is likely to improve more significantly the outcomes of this workflows,
by either changing the percentiles for sampling or the design of experiments. This is due to
the fact that the lower the target of the probability of failure, the further it is located with
respect of the mean values. Thus, an empiric relation between the percentiles to use as a basis
and the probability of failure could be built. Other design of experiments can be attempted,
but certainly the results will tend to be more successful if they are focused in the region
where failure is located as for example EWQ02 using response surface FO10. This can be
only identified by the elaboration of Tornado plot or a similar technique that allows to know
some information about the sensitivities.
It is believed in this thesis that the limiting part of the procedure is the update of the
threshold, because the update occurs in disconnection with the closeness of the prediction of
the actual response by the response surface has with respect to the actual response. Recall
from Section 4-3 that the threshold is updated by multiplying the estimated value of the
performance function |ĝ(x)| times the threshold relaxation factor γt . The estimated value of
the performance function |ĝ(x)| is partially related with the prediction of the actual response,
because it results from regression of samples on the actual response, but the strength of this
relation disappears with the increase of non-linearity. The threshold relaxation factor γt has
no relation at all with the prediction of the actual response, given that is always a constant
value of 2/3.
The results for Table 6-3 reaffirms that the disconnection of the closeness on the prediction
of the actual response (represented by the relative error) and the update of the threshold
produces a final low accuracy. The relative error is calculated as:

g(x) − ĝ(x)
Rel. Err. = (6-3)
g(x) + R

where R is the limit value. The disconnection can be observed for the three cases, because for
almost every update, the new threshold is lower than the absolute error. Because the relative
error is higher than the threshold, there could be responses that fall outside the threshold
range, but are actually failing.
Despite the disconnection between the relative error on the prediction of the actual response
and the new threshold, the three cases in Table 6-3 reach a high accuracy. The reason for
this lies on the cycles of over and underestimation observed by the negative and positive sign
of the absolute error. For the particular case of Monte Carlo, this means that the number of
failures that actually do not occur but are counted when overestimation of the actual response
exists, are compensated in great extent with the number of failures that occur but are not
counted when underestimation exists.
Table 6-3 also indicates that two controls could be added to the workflow in order to produce
more accurate results. These are the normalised threshold and the relative error between the
actual response and the predicted response for the new available samples. It is observed that
a normalised threshold of at least 0.10% and a relative error of at least 2% are a possible
target to be set in order to produce accurate probabilities of failure.
It can be argued that a relative error of 18% was still able to obtain a good accuracy for
example EWQ02, but such a selection increases the chances of being inaccurate. This is
supported by observing in Table 6-4 that the level of relative error of 19% lead to an accuracy

Master of Science Thesis RESTRICTED Pablo A. Vásconez

112 Evaluation of Workflows for Non-Linear Responses

Table 6-3: Relative Error and New Threshold versus update number for selected cases with high
accuracies

Update New Abs. Normalized Rel. Accuracy

Number Thres. Error Thres. Error Apf

Example: EWG02
Rit{FO90, Monte Carlo, th;ini = 1.20 SCU}
1 0.800 -0.07 9% -1%
2 0.460 0.68 5% 8%
3 0.040 -1.27 0.4% -12%
4 0.008 -0.16 0.1% -2% 0.91

Example: EWQ02
Rit{FOD, Monte Carlo, th;ini = 0.30m}
1 0.200 -0.84 17% -135%
2 0.113 0.85 9% 45%
3 0.074 0.36 6% 22%
4 0.045 0.40 4% 24%
5 0.026 0.37 2% 23%
6 0.009 -0.20 1% -20%
7 0.000 0.25 0.03% 17%
8 0.000 0.27 0.01% 18% 0.83

Example: EWQ02
Rit{FO10, FORM, th;ini = 0.90m}
1 0.600 0.00 50% 0%
2 0.341 -0.78 28% -126%
3 0.085 0.31 7% 19%
4 0.055 0.35 5% 22%
5 0.022 0.28 2% 19%
6 0.013 0.14 1% 11%
7 0.006 0.07 0.5% 5%
8 0.004 0.04 0.3% 3% 0.94

Pablo A. Vásconez RESTRICTED Master of Science Thesis

6-6 Limitation of the workflows Rit 113

of 0. More over for example EWG02 even a relative error of 4% could not be sufficient to
surpass an accuracy of 0.50.
Therefore, it is recommended that an improvement of the method considers a gradual reduc-
tion of the threshold is performed considering that the end situation should provide a relative
error of lower than 2%. This value could be revised, but it is considered to provide good
accuracies mainly in problems with one design point, which apparently it is the case for the
examples analysed in this study.
The conclusion for this section is that there is a limitation on the methodology for workflows
Rit, which ultimately for certain values of probability of failure may not produce accurate
results for any of the responses surfaces and initial thresholds attempted. The explanation is
found to be a disconnection between the prediction of the actual response and the threshold.
More detailed study of the process of update of the response surface suggests the relative
error on the prediction of the actual response could be used as a basis for the gradual closing
of the threshold. Some evidence supports as target a relative error of less than 2% for the
last threshold used.
Table 6-4: Relative Error and New Threshold versus update number for selected cases with low
accuracies

Update New Abs. Normalized Rel. Accuracy

Number Thres. Error Thres. Error Apf

Example: EWG02
Rit{FO90, Monte Carlo, th;ini = 9.20 SCU}
1 1.667 -0.17 18% -1%
2 1.005 0.13 11% 1%
3 0.370 0.76 4% 8%
4 0.073 -0.39 1% -4% 0.50

Example: EWQ02
Rit{FOD, Monte Carlo, th;ini = 0.30m}
1 0.600 -0.06 50% -3%
2 0.368 -0.23 31% -15%
3 0.230 -1.00 19% -189%
4 0.145 0.54 12% 33%
5 0.049 0.49 4% 29%
6 0.009 0.44 1% 27%
7 0.003 0.25 0.26% 17%
8 0.002 0.28 0.13% 19% 0.00

Example: EWQ02
Rit{FO10, FORM, th;ini = 0.90m}
1 0.100 -2.30 8% 218%
2 0.021 0.65 2% 35%
3 0.002 0.56 0.19% 32% 0.00

Master of Science Thesis RESTRICTED Pablo A. Vásconez

114 Evaluation of Workflows for Non-Linear Responses

FO10 FO90 FOD SO ED

Quasi-Linear Non-Linear
1.00
Peak accuracies Apf

0.80

0.60

0.40

0.20

0.00
Max.Sub Ave.Sub Max.She Ave.She Max.Sub Ave.Sub Max.Sub Ave.Sub
EWG EWG02 EWQ01 EWQ02

Figure 6-12: Quasi-linear vs Non-linear for Rit{Monte Carlo} peakAcc

FO10 FO90 FOD SO ED

Quasi-Linear Non-Linear
100
Nmod for peak accuracies

80

60

40

20

0
Max.Sub Ave.Sub Max.She Ave.She Max.Sub Ave.Sub Max.Sub Ave.Sub
EWG EWG02 EWQ01 EWQ02

Figure 6-13: Quasi-linear vs Non-linear for Rit{Monte Carlo} peakNmod

Pablo A. Vásconez RESTRICTED Master of Science Thesis

6-6 Limitation of the workflows Rit 115

FO10 FO90 FOD SO ED

Quasi-Linear Non-Linear
1.00
Residual accuracies Apf

0.80

0.60

0.40

0.20

0.00
Max.Sub Ave.Sub Max.She Ave.She Max.Sub Ave.Sub Max.Sub Ave.Sub
EWG EWG02 EWQ01 EWQ02

Figure 6-14: Quasi-linear vs Non-linear for Rit{Monte Carlo} resAcc

FO10 FO90 FOD SO ED

Quasi-Linear Non-Linear
1.00
Peak accuracies Apf

0.80

0.60

0.40

0.20

0.00
Max.Sub Ave.Sub Max.She Ave.She Max.Sub Ave.Sub Max.Sub Ave.Sub
EWG EWG02 EWQ01 EWQ02

Figure 6-15: Quasi-linear vs Non-linear for Rit{FORM} peakAcc

Master of Science Thesis RESTRICTED Pablo A. Vásconez

116 Evaluation of Workflows for Non-Linear Responses

FO10 FO90 FOD SO ED

Quasi-Linear Non-Linear
100
Nmod for peak accuracies

80

60

40

20

0
Max.Sub Ave.Sub Max.She Ave.She Max.Sub Ave.Sub Max.Sub Ave.Sub
EWG EWG02 EWQ01 EWQ02

Figure 6-16: Quasi-linear vs Non-linear for Rit{FORM} peakNmod

FO10 FO90 FOD SO ED

Quasi-Linear Non-Linear
1.00
Residual accuracies Apf

0.80

0.60

0.40

0.20

0.00
Max.Sub Ave.Sub Max.She Ave.She Max.Sub Ave.Sub Max.Sub Ave.Sub
EWG EWG02 EWQ01 EWQ02

Figure 6-17: Quasi-linear vs Non-linear for Rit{FORM} resAcc

Pablo A. Vásconez RESTRICTED Master of Science Thesis

6-6 Limitation of the workflows Rit 117

FO10 FO90 FOD SO ED

Quasi-Linear Non-Linear
1.00

0.80
Accuracies Apf

0.60

0.40

0.20

0.00
Max.Sub Ave.Sub Fai.Sub Max.She Ave.She Max.Sub Ave.Sub Max.Sub Ave.Sub
EWG EWG02 EWQ01 EWQ02

Figure 6-18: Quasi-linear vs Non-linear for Mod{FORM}

FO10 FO90 FOD SO ED

Quasi-Linear Non-Linear
100

80

60
Nmod

40

20

0
Max.Sub Ave.Sub Fai.Sub Max.She Ave.She Max.Sub Ave.Sub Max.Sub Ave.Sub
EWG EWG02 EWQ01 EWQ02

Figure 6-19: Quasi-linear vs Non-linear for Mod{FORM}

Master of Science Thesis RESTRICTED Pablo A. Vásconez

118 Evaluation of Workflows for Non-Linear Responses

6-7 Conclusions

This chapter attempted to investigate de performance of selected workflows for examples with
non-linear responses. The conclusions drawn are the following:

• The non-linearity of a given response of interest is not only induced by the mechani-
cal uncertainty, i.e. by a non-linear relation between the stress and strains, but also
from the change of a geomechanical parameter. For the analysis made using the soft-
ware Quickblocks, the elastic modulus E has a signficant non-linear relation with the
subsidence.

• For subsidence using the geomechanical model Quickblocks, the workflow Rit{FORM}
with response surface FO10 provides the highest accuracy on the probability of failure.

• Workflow Mod{FORM} proved to be accurate and efficient workflow for the non-linear
examples evaluated in this chapter.

• Workflow using response surface with updates according to the threshold (i.e. Rit{Monte
Carlo} and Rit{FORM}) have the potential to provide good accuracies for the examples
and target of probability of failure analysed in this study. Nevertheless, the reaching
of a good accuracy depends on the proper selection of the response surface and initial
threshold.

• In more general terms, the non-linearity of the example under analysis and the target of
probability of failure determine if there would be a combination of response surface and
initial threshold for workflows Rit that can actually provide high accuracies. The lower
the non-linearity or the higher target probability of failure, the higher the number of
the mentioned combinations available. The opposite leads that all combinations provide
low accuracies.

• The reason for the limited applicability of Rit is that there is a disconnection between
the prediction of the actual response and the threshold. More detailed study of the
response surface updates suggests that the relative error on the prediction of the actual
response could be used as a basis for the gradual closing of the threshold. Some evidence
supports as target a relative error of less than 2% for the last threshold used.

Pablo A. Vásconez RESTRICTED Master of Science Thesis

 Chapter 7

Conclusions and Recommendations

7-1 Conclusions

The study presented in this report is a contribution to encourage the incorporation of this
design tool to the common geotechnical practice. The focus is on finding reliability methods
or procedures (named workflows in this study), others than the widespread Monte Carlo, with
potential to compute efficiently and accurately the probability of failure for geomechanical
applications.
In order to achieve this objective, the methodology taken in this study was to evaluate several
workflows for specific geomechanical examples. The terms of the evaluation was the accuracy
on the probability of failure, measured as the degree of closeness to the exact result, and
efficiency, measured in the number of geomechanical model evaluations that the method
required. It is considered that the exact result comes from a Monte Carlo method and
that one evaluation of the geomechanical model takes a significant amount of time compared
to all the calculations correspondent exclusively to the probabilistic procedures. The first
assumption can be questioned, but still it is the best result that could be obtained given the
time constrains of the project itself. The second assumption is correct even for the simplest
geomechanical model used in this study.
The workflows evaluated are divided in three different groups:

• Workflow Res performs a reliability method using exclusively a response surface (built
up at the start of the process) to estimate the values of the performance function; the
response surface is never updated.
• Workflow Rit performs a reliability method using the response surface to estimate the
performance function, with the difference that the response surface is updated during the
execution of the reliability method. A threshold is used to defined a range that includes
the limit state, for which the estimation of the performance function is not accepted.
If this occurs, the geomechanical model is used to compute the actual performance
function, providing an additional point to update, by regression, the response surface.

Master of Science Thesis RESTRICTED Pablo A. Vásconez

120 Conclusions and Recommendations

• Workflow Mod performs a reliability method using exclusively the geomechanical model
to calculate the performance function.

Five response surfaces, differing in the order of the polynomial and sampling technique, are
used: First-Order 10 (FO10), First-Order 90 (FO90), First-Order Double (FOD), Second-
Order (SO) and Experimental Design. The reliability methods used are: Monte Carlo, Im-
portance Sampling and Directional Sampling.
The outcomes and observations derived from the examples analysed, made necessary a sep-
aration between the examples between quasi-linear and non-linear responses for reporting
purposes only. Actually, all the examples evaluated in the study are non-linear from the
probabilistic point of view. The difference is the higher degree of non-linearity for the so-
called non-linear examples, which makes the non-linearity observable even for variations very
close to the mean values.
Analysing the results of the study, it is shown that reliability based design (RBD) is possible
for the selected geomechanical examples, because one of the studied workflows was capable to
compute accurately and efficiently the probability of failure for all the analysis: Mod{FORM},
i.e. the reliability method FORM applied exclusively on the geomechanical model. The
accuracies obtained by this workflow ranged between 0.80 and 0.97 (using a scale from 0 to 1,
in which 1 is the exact result) and the number of geomechanical model realisations are around
30 for a 6-random-variable analysis and around 100 for a 11-random-variable analysis. This
method provides a strong reduction of model realisations when compared to the Monte Carlo
analysis that took 20,000 to 60,000 realisations to obtain a convergence result.
Another promising workflows are Rit{FORM} and Rit{Monte Carlo}, i.e. the reliability
method FORM or Monte Carlo is applied to a response surface that updates. For the examples
analysed it was always feasible to obtain accuracies of 0.70 or higher requiring roughly only
1/3 the number of evaluations the ones for Mod{FORM}. Nevertheless, it is identified that
the higher the non-linearity of the response or the lower the target probability of failure,
the lower is the accuracy obtained. The reason is that there is a disconnection between the
accuracy on the prediction of the actual response and the updates of the threshold. When the
response is highly non-linear or the target probability of failure is very low, the prediction
made by the initial response surface is very inaccurate and it is not possible for the procedure
to update the response surface until a good level of prediction is reached. By providing a
connection between the accuracy on the prediction of the actual response and the updates
of the threshold, it is expected that this workflow could produce higher accuracies for the
selected examples, but with a correspondent increase in the number of geomechanical model
realisations.
Some remarks can be already made based about the applicability of the proposed methods
in future scenarios. One of them could be the use of a higher number of random variables
may be required. Such a case will diminish the efficiency of the workflows, because FORM
computes numerically partial derivatives and the response surfaces need sampling for each
random variable. Thus, for high number of random variables it is suggested to select as
random variables only the most influential random variables, setting the other parameters
to conservative values. Nevertheless, the reduction of random variables should be studied
thoroughly and also the interaction among them, which was not the focus of this thesis.

Pablo A. Vásconez RESTRICTED Master of Science Thesis

7-1 Conclusions 121

Another scenario is the use of non-linear constitutive models to represent more realistically
the behaviour of the subsurface. As explained before, the current implementation of workflows
Ritis sensitive to the degree of non-linearity, suggesting that for this scenario, their application
could lead more likely to inaccurate probabilities of failure. Workflow Mod{FORM} is thought
to be able to obtain accurate results, but reducing in efficiency because a higher amount of
iterations are needed in presence of a high change in gradient. Nevertheless, the use of
complex constitutive models could lead to inaccurate results for Mod{FORM}, specifically if
they induce multiple design points.
It is advised that for every new type of problem addressed, prior validation should be per-
formed to the reliability methods of preference by contrasting their results to a Monte Carlo
analysis, as done in this thesis. The methods that performed well in the examples studied,
may be used as the aforementoned reliability methods of preference as a starting point.
The specific conclusions of the study, linked to the evaluations performed, are the following:

• Among the workflows that use the geomechanical model directly (here named Mod),
workflow Mod{FORM} computed accurately and efficiently values of the probability of
failure for the examples analysed. This holds true even for the indicator of failure area.
The proposed Mod{Directional Sampling} and Mod{Importance Sampling} can lead to
medium to good accuracies, but the computation time is very high, even comparable to
Mod{Monte Carlo}.

• Worfklows using the researched response surfaces with threshold updates (named Rit)
have the potential to obtain good accuracies for examples with quasi-linear and non-
linear responses. Nevertheless, their performance is very sensitive to the combination
of response surface with initial threshold selected. The degree of non-linearity of the
example under analysis and the target of probability of failure determine if there would
be a combination that provides a high accuracy.

• For the examples analysed and the target probability of failure taken in this study,
it was possible to find a combination of response surface and initial threshold to ob-
tain good accuracies with workflows Rit. The most successful is Rit{FORM} using the
combination of response surface FO10 and an initial threshold of 30% the limit value.
The preference of FO10 owes that for Geerstma the examples are quasi-linear (initial
response surface is less important) and that for Quickblocks the most influential param-
eters are unfavourable for their lower values of the distribution (location of the samples
taken to build FO10).

• Workflows Rit cannot be applied universally. For examples with high non-linearity
and/or low target probabilities of failure than the examples analysed no combination
of response surface with initial threshold will provide good accuracies. The reason for
the limited applicability of Rit is that there is a disconnection between the prediction
of the actual response and the threshold.

• Worfklows using the researched response surfaces with or without threshold updates
(i.e. workflows Res or Rit) have limited applicability for computing accurately the
probability of failure for examples with a region of zero responses and a region of non-
zero responses. For the indicator of failure area and the target probability of failure

Master of Science Thesis RESTRICTED Pablo A. Vásconez

122 Conclusions and Recommendations

used in this study, the initial response surface was built on samples from the region of
zero responses, producing a probability of failure of 0 for Monte Carlo and no value (no
convergence due to zero gradient) for FORM.

• Workflows using the researched response surfaces without updates (named Res) provide
very inaccurate results for the target probability of failure adopted in this study. This
holds true even for quasi-linear problems.

7-2 Recommendations

The recommendations regarding the applicability of the methods and minor improvements
are the following:

• Workflows using the researched response surfaces without updates should not be used
to compute the probability of failure of any of the examples evaluated and in more
complicated ones.

• Worflows Rit{Monte Carlo and FORM} should be used with care for similar examples
than the analysed in this study. The care prior the execution of the workflow is taken
by checking the non-linearity of the responses and the distance of the extreme responses
from the Tornado plot (or a similar diagram) and the desired limit value: if the degree of
non-linearity and the distance are both small, then any response surface but preferable
FO10 or FO90 and an initial threshold of around 10-30% can be used to compute the
probability of failure; if either the degree of non-linearity or the distance are high,
then the response surface which points towards the unfavourable direction for the most
influential parameters must be chosen and an initial threshold of 30%. The care taken
after the execution of the workflow is taken by checking the final normalised threshold
and the relative error on the prediction of the actual response must be checked to be
preferable to be 0.10% and 2% respectively (higher values may be accepted for the
relative error).

• Create a new response surface based on the samples that build up the tornado plot
(i.e. response surface FOD). First, the tornado plot is formed and then a first-order
response surface can be computed regarding P50 and the most unfavourable sample for
each parameter (either P10 or P90). With this change, the initial response surface will
predict better the response in the region of the limit state function.

• Use a factor to increase the estimated probability of failure by any of the workflows
used, given that in most cases the probability of failure was underestimated. This
factor should be 1.3 (corresponding to an accuracy of 0.70) and be applied if and only
care was taken in the application for workflows Rit. For Mod{FORM} a factor of 1.2
should be used instead.

• Before attempting the use of the proposed workflows for new failure mechanisms, they
should be evaluated in advance, as it was performed in this study for selected examples.

• For Directional Sampling, the linear search must be changed for the case when failure
is not found. Currently, it continues until a distance of 20 units in the standard normal

Pablo A. Vásconez RESTRICTED Master of Science Thesis

7-3 Future research 123

space, starting from the mean. This implies 20 model realisations per sampled direction.
A good suggestion is to select this distance in accordance with the target probability of
failure. For instance, for the target probability of failure of 0.002 taken in this study, a
distance of 3 units in the u-space may suffice to get the same accuracy than using 20
units (being each unit one realisation in the current implementation).

• Workflow Mod{FORM} is recommended for any of the examples evaluated in this study.

7-3 Future research

This section intends to contribute with suggestions for future research in order to eliminate the
limitations found during the evaluations of the workflows. They have the following objectives:

1. Eliminate the need of selecting properly the combination of response surface and an
initial threshold by improving the procedure that workflows using response surface with
threshold updates Rit follow.

2. Update of the response surface without the use of the threshold.

3. Diversifying the alternatives to compute efficiently and with good accuracy the proba-
bility of failure for the indicator of failure area subsidence.

Eliminating the selection of a proper combination of response surface and initial

threshold for Rit
In order to eliminate the need of selecting properly the response surface and the initial thresh-
old, there are several relevant alternatives which deserve investigation:

• Select a different initial sampling depending on the target probability of failure.

• Modify the manner the threshold is updated to be related to the accuracy of the pre-
diction of the actual response by the response surface.

For the first alternative, an investigation could be carried out for the same examples in this
study, but increasing the distances of sampling for the non-linear cases in order to increase
the accuracies until good levels regardless the response surfaces and hopefully the initial
threshold. Also, for all the examples, the target probability of failure could be reduced and
also the distance for sampling to achieve the same previous goal. The whole study considered
the sampling taken in percentiles P10 and P90, which had no relation to the target probability
of failure. Recall that this percentages represent a distance of 1.28 standard deviations from
the mean.
The recommended investigation is based on the fact that the lower the target probability of
failure, the further is the location for the samples that produce failures (with respect to the
mean-values point). If the initial sampling is located sufficiently far from the mean-values
point, the underestimation of the actual response may not be significant nearby the failure

Master of Science Thesis RESTRICTED Pablo A. Vásconez

124 Conclusions and Recommendations

regions, which allow the subsequent updates to produce an increase of the prediction of the
actual response and ultimately, a more accurate probability of failure.
Beware that this solution would be case-specific, because the relation distance of sampling vs.
target probability of failure will depend on the degree of non-linearity of the actual response.
The second alternative is an attempt for a universal application of Rit, or at least, to widen its
range of applicability. In Section 6-6 it is observed that the prediction of the actual response
needs to be related with the threshold update, which does not occur for their proposed
procedure. By controlling the normalised threshold and the relative error on the prediction of
the new sampled point, good accuracies on the probability of failure are feasible. A suggestion
for the controls, based in a few evidence from this study, is that the updates must stop if
the relative error on the prediction of the last sampled point is 1% as long as the normalised
threshold is already 0.10%.
The alternative has to be completed by selecting an initial threshold and the manner it is
reduced as more samples are available. The initial threshold becomes less importance because
the controls added, therefore an arbitrary value can be selected, for instance, 30% of the limit
value. The most important part of the procedure is the way that this threshold is reduced. A
possibility is to reduce the initial threshold by multiplying the performance function estimated
by the response surface by an arbitrary factor (as proposed in this study) until the threshold
is equal or lower than 0.10%; then, keep the threshold constant until a relative error of 1% is
reached for the prediction of the actual response.
Update of the response surface without the use of the threshold
This suggestion for further research goes in the direction to evaluate the one of the response
surfaces approaches briefly described in Section 3-4. They do not use the threshold as in Rit
of this study. Any of the approaches is thought to perform well for the examples analysed,
but maybe they will take a higher number of geomechanical model realisations.
In this sense, the simplest of the response surface approaches can be used, as proposed by
Rajashekhar and Ellingwood (1993), but with a slight change in order to reduce the number
of realisations regarding the updates of the response surface. The full procedure can consist
in producing a first estimate of the design point, but instead of making a new design of
experiments around it, as in Rajashekhar and Ellingwood (1993)) a new realisation is made
for the design point only and the relative error between the estimated and the actual response
is computed. If the relative error is higher than 1%, the response surface is updated by using
the result of the new realisation and then, a new design point is computed. The process is
repeated until the relative error for the design point is less than 1%. After this, the probability
of failure can be obtained directly from the last location of the design point or by performing
a Monte Carlo analysis.
Diversifying the alternatives for failure area subsidence
In the conclusions it is mentioned that the only method available to compute the probability of
failure for the indicator of failure area subsidence is Mod{FORM}. By modifying slightly the
approach of the workflows Rit, it is possible to make the response surfaces also an alternative
for failure indicator.
The modifications that could be research are:

Pablo A. Vásconez RESTRICTED Master of Science Thesis

7-3 Future research 125

• Sampling in the region of non-zero responses, instead of P10, P50 and P90. This could
be done by using the response surface for the indicator maximum to reach the region
of non-zero responses.

• Use multiple response surfaces, each one assigned to one point of the output grid. The
response of each point is approximated and the failure area is derived from it.

The first modification attempts to use a single response surface, but to use the one for the
indicator of maximum instead of failure area. This is based on the fact that, if the limit value
is not surpassed for the maximum, then it is not surpassed on any value. In this context,
the tornado plot for indicator maximum can be obtained and, departing from it, a sample
that surpassed the limit value can be estimated. An experimental or a random design can be
carried out to obtain the necessary amount of samples (nearby the previously found sample)
in order to build up the response surface for failure area. Once started, the method should
be able to obtain good accuracies. It may be important to control that the threshold does
not provide samples in the zero-responses region.
The second modification tries to overcome the limitation that occurs when forming the re-
sponse surface for failure area: sampling in the regions of zero-responses, because the failure
indicator is based on the result of the geomechanical model directly. In the modification,
it is proposed that the failure indicator is computed based on many response surfaces, each
correspondent to the a point in the real space in a region of interest. For instance, if failure
area subsidence is computed, then it would be one response surface to compute the subsidence
for each point of the grid. In this manner, it is possible to compute values higher than 0 for
failure area when samples far from the mean-values point are used. This is not possible when
all the coefficients of a global response surface are zero.
Another reason that motivates the first modification, is that for the maximum and average
indicators, the use of response surface was capable to find good accuracies. By assigning to
each point of the grid a response surface, the prediction is likely to be between maximum
(prediction of one point) and average (mean prediction of all points). Note that if the re-
liability method chosen is FORM, the problem of no convergence will occur again, but this
could be solved by changing the start vector.

Master of Science Thesis RESTRICTED Pablo A. Vásconez

 Appendix A

Geomechanical Models

In the study it is mentioned the utilisation of so-called geomechanical models. A geomechanical

model is a representation of the the stress and strain relations that occur in the subsurface due
to a given phenomena (i.e. loading, predefined deformation, change in pore pressure, etc.).
It captures not all attributes of the phenomena, but rather only those that seem relevant.
This means that the geomechanical models has limitations and should be used for its specific
purpose.
Two geomechanical models are used in this study:

1. Geertsma

2. Quickblocks

Geertsma
It is an analytical model that computes strains and stresses in the whole continuum due to
a change in pressure in a disk-shape reservoir, located at a certain depth from the surface.
The distribution of the strains and stresses is assumed to be axisymmetric and the behavior
of the continuum, linear elastic. It also assumes a homogeneous subsurface, i.e. the material
properties are equal in every point of the continuum, no horizontal or vertical variability
(layering) exists. The existence of the reservoir is defined by the change in pressure rather
than a change in properties compared to the surroundings.
The name of this geomechanical model owes to J. Geertsma, who in 1973 proposed this
analytical solution (Geertsma, 1973). His worked is based in the so-called nucleus-of-strain
concept in the half-space introduced by Mindlin and Cheng (1950) that originally allowed to
compute the subsidence of a small but finite volume V under the influence of a pore-pressure
reduction ∆p. Assuming that the properties in the reservoir and the surroundings are equal,
the subsidence for a disk-shaped reservoir can be found by integrating the nucleus solution
over the entire reservoir volume.

Pablo A. Vásconez RESTRICTED Master of Science Thesis

 127

All the equations may be found in the mentioned paper, but here the one at the center of the
reservoir is shown:
!
D/R
Lsub (x) = 2 ∗ cm (1 − ν)∆p ∗ h 1 − p (A-1)
1 + (D/R)2

where cm is the compaction coefficient, ν is the Poisson’s ratio, ∆p is the change in pore
pressure in the whole reservoir, D is the depth of the reservoir and R is the radius of the
reservoir as observed in Figure A-1.

Surface

R
Δp h

Reservoir
ν
cm

Figure A-1: Sketch of the Geomechanical model Geertsma

A typical example profile of the subsidence, starting from the center of the reservoir is shown
in Figure A-2. Note that the subsidence in the center is maximum and diminishes with the
radial distance.
Geertsma did not only find a solution for the subsidence, but also for the stress in all directions
and in any point of the reservoir. This allows to estimate if the failure mechanisms in Table 4-
1 do occur. Nevertheless, be aware that this solution considers linear elastic theory only, and
thus their results are not realistic for modelling post-failure behaviour.

Master of Science Thesis RESTRICTED Pablo A. Vásconez

128 Geomechanical Models

Distance from the center of reservoir (m)

0 2000 4000 6000 8000 10000 12000 14000 16000
0.00

0.05
Subsidence

0.10

0.15

0.20

0.25

Figure A-2: Subsidence profile with distance, starting from the center of the reservoir

Pablo A. Vásconez RESTRICTED Master of Science Thesis

 129

Quickblocks
Quickblocks is a semi-analytical model that computes strains and stresses due to a change in
pressure in a reservoir of any shape. Small cubic blocks are positioned one next to the other
to build up the desired shape. The application computes (semi-)analytical solutions for the
linear elastic geomechanical response of a block-shaped reservoir and subsequently adds the
single-block responses to obtain the field response.
For this geomechanical model, the solution is computed by superposing the nucleus-of-strain
concept of Mindlin and Cheng (1950), rather than integrating in a closed-form solution. This
allows to use any shape of the reservoir, by building it as the addition of blocks one next to
another.
In addition, this model considers a reflectivity method (Kuvshinov, 2007). This allows also
to consider layered subsurface, which is an important improvement, since it is well known
that the contrast between the reservoir and the over or the underburden affect significant the
results.
The two features incorporated by Quickblocks are depicted in Figure A-4, by showing three
layers with different properties and the composition of the reservoir by several blocks. Note
that it is possible to assign a different change in pressure for each block, allowing more realistic
inputs for the problems. A drawback, shown also in the sketch, is that the properties inside
and outside the reservoir, but within the same layer, are equal.
From a determinisitic point of view, Quickblocks is able to produce the same results as
Geertsma. Figure A-4 shows the comparison made for the same problem for the two geome-
chanical models. All the layers were set to have the same geomechanical properties in order
to be as close as possible.
Figure A-4 shows the subsidence with the radial direction for the same case as Figure A-4. In
Quickblocks, it is possible to define all the layers equal in order to reproduce the homogeneous
subsurface.
As Geertsma, the geomechanical model Quickblocks cannot be used for modelling post-failure
behaviour (fault slip, pore collapse, plastic shear and resulting stress redistribution) either.
Nevertheless, it can be applied for the following situations:

• Modelling of surface deformations for e.g. facilities and pipeline integrity, license-to-
operate, reservoir surveillance and management.

• Evaluation of geomechanical risks such as cap rock and well failure.

• Modelling of the stress state around faults to assess the risk of fault (re-)activation

• Modelling of seismic timeshifts for reservoir surveillance and management.

• Modelling of reservoir compaction for compaction drive.

Master of Science Thesis RESTRICTED Pablo A. Vásconez

130 Geomechanical Models

Δp1 Δp2
Upper:
R=2540m
E1, ν1

Δp3
Reservoir: Δp3
E2, ν2

Lower:
E3, ν3

Figure A-3: Sketch of the Geomechanical Model Quickblocks

Distance from the center of reservoir (m)

0 2000 4000 6000 8000 10000 12000 14000 16000
0.00

0.05
Subsidence

0.10 Quickblocks
Geertsma

0.15

0.20

0.25

Figure A-4: Subsidence profile with distance from the center of the reservoir, using the same
data but different geomechanical models: Geertsma and Quickblocks

Pablo A. Vásconez RESTRICTED Master of Science Thesis

 Appendix B

Additional Results

B-1 Example: Average SCU EWG02

Table B-1: Results of the evaluation of workflows Mod for Average SCU EWG02

Workflow Probability of failure, Accuracy, Number of

pf Ap f model realisations
Nmod
Mod{MonteCarlo} 0.00219 1.00 45638
Mod{FORM} 0.00209 0.95 30

Master of Science Thesis RESTRICTED Pablo A. Vásconez

132 Additional Results

(a)
2.50
FO10
Probability of Failure, pf [10−3]

FO90
2.00 FOD
SO
ED
1.50 exact

1.00

0.50

0.00
0.10 0.20 0.30 0.40 0.50 0.60 0.70
Initial Threshold Average SCU [-]

(b)
1.00
Accuracy Probability of Failure, Ap̂f

FO10
0.90 FO90
0.80 FOD
SO
0.70 ED
0.60
0.50
0.40
0.30
0.20
0.10
0.00
0.10 0.20 0.30 0.40 0.50 0.60 0.70
Initial Threshold Average SCU [-]

Figure B-1: Performance of workflow Rit{Monte Carlo} for Average SCU of example EWG02.
(a) Probability of Failure pf (b) Accuracy of the probability of failure Apˆf

Pablo A. Vásconez RESTRICTED Master of Science Thesis

B-1 Example: Average SCU EWG02 133

(a)
2.50
FO10
Probability of Failure, pf [10−3]

FO90
2.00 SO
ED
exact
1.50

1.00

0.50

0.00
0.10 0.20 0.30 0.40 0.50 0.60 0.70
Initial Threshold Average SCU [-]

(b)
1.00
Accuracy Probability of Failure, Ap̂f

FO10
0.90 FO90
0.80 SO
ED
0.70
0.60
0.50
0.40
0.30
0.20
0.10
0.00
0.10 0.20 0.30 0.40 0.50 0.60 0.70
Initial Threshold Average SCU [-]

Figure B-2: Performance of workflow Rit{FORM} for Average SCU of example EWG02. (a)
Probability of Failure pf (b) Accuracy of the probability of failure Apˆf

Master of Science Thesis RESTRICTED Pablo A. Vásconez

134 Additional Results

B-2 Example: Average Subsidence EWQ01

Table B-2: Results of the evaluation of workflows Mod for Average Subsience EWQ01

Workflow Probability of failure, Accuracy, Number of

pf Ap f model realisations
Nmod
Mod{MonteCarlo} 0.001901 1.00 20000
Mod{FORM} 0.00202 0.92 72

(a)
4.50
FO10
4.00
Probability of Failure, pf [10−3]

FO90
3.50 FOD
SO
3.00 ED
exact
2.50
2.00
1.50
1.00
0.50
0.00
0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40
Initial Threshold Average Subsidence [m]

(b)
1.00
Accuracy Probability of Failure, Ap̂f

FO10
0.90 FO90
0.80 FOD
SO
0.70 ED
0.60
0.50
0.40
0.30
0.20
0.10
0.00
0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40
Initial Threshold Average Subsidence [m]

Figure B-3: Performance of workflow Rit{Monte Carlo} for Average Subsidence of example
EWQ01. (a) Probability of Failure pf (b) Accuracy of the probability of failure Apˆf

Pablo A. Vásconez RESTRICTED Master of Science Thesis

B-2 Example: Average Subsidence EWQ01 135

(a)
4.50
FO10
4.00
Probability of Failure, pf [10−3]

FO90
3.50 SO
ED
3.00 exact
2.50
2.00
1.50
1.00
0.50
0.00
0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40
Initial Threshold Average Subsidence [m]

(b)
1.00
Accuracy Probability of Failure, Ap̂f

FO10
0.90 FO90
0.80 SO
ED
0.70
0.60
0.50
0.40
0.30
0.20
0.10
0.00
0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40
Initial Threshold Average Subsidence [m]

Figure B-4: Performance of workflow Rit{FORM} for Average Subsidence of example EWQ01.
(a) Probability of Failure pf (b) Accuracy of the probability of failure Apˆf

Master of Science Thesis RESTRICTED Pablo A. Vásconez

136 Additional Results

B-3 Example: Average Subsidence EWQ02

Table B-3: Results of the evaluation of workflows Mod for Average Subsience EWQ02

Workflow Probability of failure, Accuracy, Number of

pf Ap f model realisations
Nmod
Mod{MonteCarlo} 0.00196 1.00 20000
Mod{FORM} 0.00219 0.84 84

(a)
4.50
FO10
4.00
Probability of Failure, pf [10−3]

FO90
3.50 FOD
SO
3.00 ED
exact
2.50
2.00
1.50
1.00
0.50
0.00
0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80
Initial Threshold Avarage Subsidence [m]

(b)
1.00
Accuracy Probability of Failure, Ap̂f

FO10
0.90 FO90
0.80 FOD
SO
0.70 ED
0.60
0.50
0.40
0.30
0.20
0.10
0.00
0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80
Initial Threshold Average Subsidence [m]

Figure B-5: Performance of workflow Rit{Monte Carlo} for Average Subsidence of example
EWQ02. (a) Probability of Failure pf (b) Accuracy of the probability of failure Apˆf

Pablo A. Vásconez RESTRICTED Master of Science Thesis

B-3 Example: Average Subsidence EWQ02 137

(a)
4.50
FO10
4.00
Probability of Failure, pf [10−3]

FO90
3.50 SO
ED
3.00 exact
2.50
2.00
1.50
1.00
0.50
0.00
0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80
Initial Threshold Average Subsidence [m]

(b)
1.00
Accuracy Probability of Failure, Ap̂f

FO10
0.90 FO90
0.80 SO
ED
0.70
0.60
0.50
0.40
0.30
0.20
0.10
0.00
0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80
Initial Threshold Average Subsidence [m]

Figure B-6: Performance of workflow Rit{FORM} for Average Subsidence of example EWQ02.
(a) Probability of Failure pf (b) Accuracy of the probability of failure Apˆf

Master of Science Thesis RESTRICTED Pablo A. Vásconez

 References

Ayyub, B. and McCuen, R. (2003). Probability, Statistics, and Reliability for Engineers and
Scientists, Second Edition. Chapman & Hall.

Baecher, G. B. (1982). Statistical methods in site characterization. Engineering Foundation,

Santa Barbara, California.

Brinkman, F. D. (2012). Hydra ring scientific documentation. Draft 1206006-004-ZWS-0001,

Deltares.

Bucher, C. and Bourgund, U. (1990). A fast and efficient response surface approach for
structural reliability problems. Structural Safety, 7(1):57 – 66.

Caflisch, R. E. (1998). Monte carlo and quasi-monte carlo methods. Acta Numerica, 7:1–49.

Christian, J. T. (2004). Geotechnical engineering reliability: How well do we know what we

are doing? Journal of Geotechnical and Geoenvironmental Engineering, 130(10):985–1003.

Das, P. and Zheng, Y. (2000). Cumulative formation of response surface and its use in
reliability analysis. Probabilistic Engineering Mechanics, 15(4):309 – 315.

Deak, I. (1980). Three digit accurate multiple normal probabilities. Numerische Mathematik,
35:369–380.

Der Kiureghian, A., Lin, H.-Z., and Hwang, S.-J. (1987). Second-order reliability approxima-
tions. Journal of Engineering Mechanics, 113(8):1208–1225.

Ditlevsen, O. and Madsen, H. O. (1996). Structural reliability methods, volume 178. Citeseer.

Ditlevsen, O., Melchers, R., and Gluver, H. (1990). General multi-dimensional probability
integration by directional simulation. Computers and Structures, 36(2):355 – 368.

Du, X. (2005). First and second order reliability methods. Lectures Notes in Probabilistic
Engineering Design, University of Missouri - Rolla.

EN 1997 - Eurocode 7 (2006). Eurocode 7: Geotechnical design.

Pablo A. Vásconez RESTRICTED Master of Science Thesis

REFERENCES 139

Engelund, S. and Rackwitz, R. (1993). A benchmark study on importance sampling techniques

in structural reliability. Structural Safety, 12(4):255 – 276.

Faravelli, L. (1989). Response-surface approach for reliability analysis. Journal of Engineering

Mechanics, 115(12):2763–2781.

Feller, W. (1968). An introduction to probability theory and its applications. Number v. 1

in Wiley series in probability and mathematical statistics. Probability and mathematical
statistics. Wiley.

Gayton, N., Bourinet, J., and Lemaire, M. (2003). Cq2rs: a new statistical approach to the
response surface method for reliability analysis. Structural Safety, 25(1):99 – 121.

Geertsma, J. (1973). Land subsidence above compacting oil and gas reservoirs. Journal of
Petroleum Technology, 25(6):734–744.

Grooteman, F. (2011). An adaptive directional importance sampling method for structural

reliability. Probabilistic Engineering Mechanics, 26(2):134 – 141.

Gupta, S. and Manohar, C. (2004). An improved response surface method for the determi-
nation of failure probability and importance measures. Structural Safety, 26(2):123–139.

Hammersley, J. (1960). Monte carlo methods for solving multivariable problems. Annals of
the New York Academy of Sciences, 86(3):844–874.

Harbitz, A. (1983). Efficient and accurate probability of failure calculation by use of the
importance sampling technique. Proc. of the 4th Int. Conf. on Applications of Statistics
and Probability in Soil and Structural Engineering.

Harbitz, A. (1986). An efficient sampling method for probability of failure calculation. Struc-
tural Safety, 3(2):109 – 115.

Hasofer, A., Lind, N., and of Waterloo. Solid Mechanics Division, U. (1974). An exact and
invariant first-order reliability format. Journal of Engineering Mechanics Division, ASCE.

Hohenbichler, M., Gollwitzer, S., Kruse, W., and Rackwitz, R. (1987). New light on first-and
second-order reliability methods. Structural Safety, 4(4):267–284.

Karamchandani, A. and Cornell, C. (1992). Sensitivity estimation within first and second
order reliability methods. Structural Safety, 11(2):95 – 107.

Kaymaz, I. and McMahon, C. A. (2005). A response surface method based on weighted

regression for structural reliability analysis. Probabilistic Engineering Mechanics, 20(1):11–
17.

Kim, S.-H. and Na, S.-W. (1997). Response surface method using vector projected sampling
points. Structural Safety, 19(1):3 – 19. <ce:title>Asian-Pacific Symposium on Structural
Reliability and Its Applications</ce:title>.

Kuvshinov, B. N. (2007). Reflectivity method for geomechanical equilibria. Geophysical

Journal International, 170(2):567–579.

Master of Science Thesis RESTRICTED Pablo A. Vásconez

140 REFERENCES

Lemaire, M. (2010). Structural reliability. Wiley-ISTE.

Liu, Y. W. and Moses, F. (1994). A sequential response surface method and its application
in the reliability analysis of aircraft structural systems. Structural Safety, 16(1âĂŞ2):39 –
46.

Melchers, R. (1990). Radial importance sampling for structural reliability. Journal of Engi-
neering Mechanics, 116(1):189–203.

Metropolis, N. and Ulam, S. (1949). The monte carlo method. Journal of the American
statistical association, 44(247):335–341.

Miller, R. G. (1974). The jackknife-a review. Biometrika, 61(1):1–15.

Mindlin, R. D. and Cheng, D. H. (1950). Thermoelastic stress in the semi-infinite solid.

Journal of Applied Physics, 21(9):931–933.

Montgomery, D. C. (2008). Design and analysis of experiments. Wiley.

Nie, J. and Ellingwood, B. R. (2000). Directional methods for structural reliability analysis.
Structural Safety, 22(3):233 – 249.

Phoon, K. (2008). Reliability-Based Design in Geotechnical Engineering: Computations and

Applications. Taylor & Francis.

Rajashekhar, M. R. and Ellingwood, B. R. (1993). A new look at the response surface

approach for reliability analysis. Structural safety, 12(3):205–220.

Rosenblatt, M. (1952). Remarks on a Multivariate Transformation. The Annals of Mathe-

matical Statistics, 23(3):470–472.

Shinozuka, M. (1983). Basic analysis of structural safety. Journal of Structural Engineering,

109(3):721–740.

Swiler, L. P. and West, N. J. (2010). Importance sampling: Promises and limitations. In

Proceedings of the 12th AIAA Non-Deterministic Approaches Conference, volume 91.

Waarts, P. H. (2000). Structural Reliability using Finite Element Analysis. PhD thesis, Delft
University of Technology.

Pablo A. Vásconez RESTRICTED Master of Science Thesis

 Glossary

List of Acronyms

CDF Cumulative Distribution Function

CORA Containment Reliability Assessment

COV Coefficient of Variation

DS Directional Sampling

ED Response Surface Experimental Design

FEM Finite Element Method

FO10 Response Surface First-Order 10

FO90 Response Surface First-Order 90

FOD Response Surface First-Order Double

FORM First-Order Reliability Method

GM Geomechanical Model

IS Importance Sampling

LSF Limit State Function

MC Monte Carlo

Mod Workflow using the Geomechanical Model only

P10 Percentile 10th of the density function

P50 Percentile 50th of the density function

P90 Percentile 90th of the density function

PDF Probability Density Function

Master of Science Thesis RESTRICTED Pablo A. Vásconez

142 Glossary

Res Workflow using a Response surface only

Rit Workflow using a Response surface with updates according to a threshold

RS Response Surface

SCU Shear Capacity Utilization

SLS Serviciability Limit State

SO Second-Order Proxy

SORM Second-Order Reliability Method

ULS Ultimate Limit State

Pablo A. Vásconez RESTRICTED Master of Science Thesis