RELIABILITY ANALYSIS OF RETAINING WALL

USING ARTIFICIAL NEURAL NETWORK (ANN)

AND ADAPTIVE NEURO FUZZY INFERENCE
SYSTEM (ANFIS)

Pratishtha Mishra1 and Pijush Samui2

Department of Civil Engineering, National Institute of Technology Patna, Patna- 800 005
E-mail: [email protected]; [email protected]

Abstract: Reliability of Geotechnical structures is the main concern of Geotech-

nical engineers as is clear from previous studies and evaluations. It also helps us
to determine probability of failure. First order second moment method (FOSM)
helps us to determine the reliability index of geo-structure. This study employs
Artificial neural network (ANN) and adaptive neuro fuzzy inference system
(ANFIS) for determination of reliability index of retaining wall based on sliding
criterion. ANN has played a vital role in the field of geotechnical engineering as
it has reduced cumbersome calculations and has increased the precision of result.
The strong non-linear relationship between the known random variables and un-
known output or result is mapped easily by using ANN. ANN also ascertains the
result by removing the uncertainties involved in the problem. ANFIS is an ANN
system which uses fuzzy logic in contemplating the data. It works on removing
the fuzziness of the values entered (random variables) and gives more realistic
values of the output as compared to other approaches. This study adopts ANN
and ANFIS as regression techniques. The performance of ANN and ANFIS has
been assessed based on different parameters such as coefficient of correlation,
root mean square error, mean absolute error, etc. A comparative study has been
presented between the FOSM, ANN based FOSM and ANFIS FOSM models.
Therefore, this study concludes that ANN and ANFIS is a better alternative to
solve for the reliability of the retaining wall.

Keywords: Retaining Wall, Reliability Index, ANN, ANFIS.

1. Introduction
Retaining wall, a geotechnical structure is of sheer importance for the stability of
slopes. From geotechnical learning and rapid advancements, it is known that slopes fail
due to different mechanisms. For instance, slopes suffer rotational failure, translational
failure, compound failure, wedge failure and other failures in the form of flows and
spreads. Many remedial measures are followed to avoid the failure of the slope and
construction of retaining wall is among one of the remedies. For construction of a re-
taining wall the soil parameters that influence the bearing capacity of the soil along
with the earth pressure are evaluated. Primitive parameters that define the failure are
cohesion intercept, angle of shearing resistance, unit weight and angle of wall friction.
Considering these parameters, the factor of safety is calculated. Also, to measure the
ability to meet requirements under a specified period of time, reliability analysis is per-
formed. For reliability analysis, First Order Second Moment Method (FOSM) is widely
2

used but this technique is quite time consuming [1-2]. This problem has been remedied
by the researchers by using certain other methods such as response surface method [3-
4], multiple tangent plane surface [5], multi-plane surfaces method etc. which are used
to solve the ambiguities of non-linear limit state surface. But these approaches are lim-
ited to nonlinear convex or concave surfaces only. This article performed Reliability
analysis of retaining wall by using Artificial Neural Network (ANN) [6-7] and Adap-
tive Neuro Fuzzy Inference System (ANFIS). ANN has made progress in many fields
like in medical, geotechnical, defense etc. Applications of ANN in geotechnical engi-
neering are prediction of pile capacity, settlement of foundation, soil properties and
behavior, characterization of site, determination of liquefaction potential, evaluation of
stability of slopes, prediction of settlement of underground structures such as tunnels
and estimation of maximum deflection of earth retaining structures [8]. ANFIS has also
covered many areas of geotechnical engineering for example applications employed in
triaxial testing, resonant column testing and liquefaction triggering. ANN and ANFIS
amalgamates the different probabilities of occurring of events and pops up with accu-
rate results considering all the possibilities as these are trained with set of data which
when tested brings in modified correct output. The predicted values of the output are
further used for reliability analysis. In this paper back propagation technique is em-
ployed in ANN and clustering technique in ANFIS. Also, the reliability index of the
results generated from both ANN and ANFIS is calculated and compared thereafter.

2. Artificial Neural Network (ANN) model

Artificial intelligence has left a major mark in the computing field and other fields as
well. ANN is a branch of artificial intelligence or precisely machine learning. Neural
network is a representation of the human neural system. Networks here are defined
using three components- transfer function, architecture of network and learning law.
These components depend upon the type of problem to be solved. Using this algorithm
machine can be trained to give appropriate result by changing the weights given to the
inputs and using certain formulations. One of the renowned neural network is the back
propagation network. Although there are several other algorithms as well but back
propagation (in ANN) [9-10] is most versatile and robust among all. Back propagation
is a concept of machine learning that works on reducing the cost function. After giving
the first result and comparing it using the cost function, mechanism propagates back-
wards changing the weight factors and thereby bringing the change in the result until
and unless it reduces the cost function resulting in accurate result. Neural networks
trains and tests data like a human mind does. Mainly, back propagation’s objective is
to change or reduce the error in a quick response of time. Also, it uses partial derivative
of cost function for all the weights individually. Cost function (equation 1) is nothing
but root mean squared error (RMSE).

∑|𝑦(𝑥) − 𝑎|2
𝐶= (1)
2𝑛
Here C is the cost function, x is the input from the training set, y(x) is the observed
output, n is the total number of input training set and a is the output from the model.
Cost function is minimized in order to get results up to the mark.
 3

Back propagation works in two phases. First phase is propagation in which setting
and initialization of weights take place. Input is worked upon to generate proper output.
Errors are calculated and then output is propagated back in the neural system to generate
errors in the output and hidden MLPs. Second phase is concerned with updating weights
of connections. In this phase calculation of the gradient of the weight is performed and
certain percentage of this gradient (based on the learning) is subtracted from the weight.
Each work of this technique is done in the hidden neuron layers where different com-
position of inputs with different weights is taken. A network can have many hidden
neurons in accordance with the need of the problem. The back propagation technique
is shown in the figure 1. It can be seen that a primary level neural network has one input
layer, at least one hidden layer (there can be many depending upon the complexity of
the input) and one output layer and all the connections are given particular weight. The
hidden layers are also called perceptron which behave like human neurons. These can
be contained in large number in a network to bifurcate large inputs into different pos-
sibilities. These Multi-Layer Perceptron (MLPs) are trained to give unbiased and
learned results are these are highly capable of data mapping. The weights of the con-
nections are altered accordingly depending upon the error and weight gradient.
This model is fed with four inputs required for calculating the factor of safety of the
retaining wall based on sliding criteria i.e. are cohesion intercept (c), angle of shearing
resistance (φ), angle of wall friction (δ) and unit weight (γ) and corresponding output
to train the data. We have total 80 data out of which 70% is taken for training model
and 30% is used for testing.

Hidden layer

Input layer Output layer

Fig. 1. ANN three-layer network

3. Adaptive Neuro-Fuzzy Interference System (ANFIS) model.

Role and perfection of ANN model is already explained in the previous section but a
shortcoming of the ANN model is the complexity of the connection weights of MLPs
which cannot be deciphered. Therefore, the rules defining the relation between input
and output variable are difficult to quantify. To overcome this drawback, neurofuzzy
models are used. These models are trained to provide data mappings. Also, it extracts
knowledge about the relationship between model input and corresponding output data.
ANFIS has removed the drawbacks of the other models in use and has provided us with
accurate results comparatively. Advancement in modeling techniques has led to soft
4

computing, artificial intelligence and fuzzy modeling system. ANFIS is a hybrid tech-
nique based on understanding of the researchers. Fuzzy logic works on ‘if then’ rules
to establish a qualitative relation between input and output variables. It is a heuristic
approach. Concept of clustering is used to resolve the problem. This approach is based
on forming the unsupervised group of input and output data based on their similarities
and dissimilarities. Neurofuzzy networks employ fuzzy conditional statement i.e. if-
then rule. For instance, If U then V where U and V are labels of unsupervised fuzzy
set. This rule makes us aware of contribution of set of inputs to the output. All the fuzzy
logic systems have two components: sets and rules. To determine fuzzy sets linguistic
terms are interpreted mathematically as membership functions and variables in the
model are fuzzified to be fractional or partial members of the membership functions in
the interval of (0,1). For each and every variable, fuzzy sets overlap and necessary range
of variation is covered, this process is called fuzzification. Now, as the output of the
fuzzified input is fuzzified too therefore defuzzification algorithm such as the mean of
maxima ad center of gravity is applied, to get real valued outputs.
Fuzzy interference system is also known as fuzzy rule based system, fuzzy associa-
tive memories, fuzzy models or fuzzy controllers. This system is made of five blocks:
 Rule base (consists of neurofuzzy if-then rules)
 Dataset (defines membership functions of fuzzy sets used in the neurofuzzy
rules)
 Decision making unit (performs interference operation on rules)
 Fuzzification interface (converts real valued inputs into degree of match with
linguistic values)
 Defuzzification interface (converts fuzzified output into real value output)
Rule base and database are collectively referred to as Knowledge base. Neuro fuzzy
system enhances the generalization capacity of the network i.e. when data inputted is
beyond the training data set, networks inbuilt learning program helps it to extrapolate
the result and revert back an appropriate result based on the learning. Learning process
is knowledge based but data driven. Being adaptive of the system explains the depend-
ency of nodes to the linked parameters. It supports the learning rule which minimizes
the errors by making changes in the linked parameters. Figure 2 shows an adaptive
network.

Fig. 2. An Adaptive Network

 5

This model was fed with the same variables as in ANN with 70-30% bifurcation of
data for training and testing respectively. Also, certain values are calculated which as-
certain the model behavior and its accuracy in producing result. These values compare
the results from both the models.

4. Model performance assessment and reliability index

Reliability index was calculated using First Order Second Moment Method which gave
us the probability of failure of the retaining wall which is showed using reliability index
‘β’. Deducing this value gives an overview of the structural reliability. And as it is a
statistical value based on number of variations, when used for different models, func-
tion of model and accuracy of its prediction becomes understandable. This particular
value along with other values predicts the failure probability of a model and the perfec-
tion of the model. As in this paper problem is modelled using ANN and ANFIS, values
measuring their extent of perfection are given as follows:

1. Nash-Sutcliffe efficiency (NS) [11] indicates the predictive power of the mod-
els. More the NS value closer to 1 more is predictive power.
∑𝑛𝑖=1(𝑑𝑖 − 𝑦𝑖 )2
𝑁𝑆 = 1 − (2)
∑𝑛𝑖=1(𝑑𝑖 − 𝑑𝑚𝑒𝑎𝑛 )2

2. Root Mean Square Error (RMSE) [12] value closer or equal to 0 indicates that
the error in prediction is less.
𝑛
1
𝑅𝑀𝑆𝐸 = √ ∑(𝑑𝑖 − 𝑦𝑖 )2 (3)
𝑁
𝑖=1

3. Variance Account Factor (VAF) [13-14] value equal to 100% shows model
performance gives good result.
𝑣𝑎𝑟( 𝑑𝑖 − 𝑦𝑖 )
𝑉𝐴𝐹 = (1 − ) × 100
𝑣𝑎𝑟( 𝑑𝑖 ) (4)

4. R2 (Coefficient of determination) and Adj. R2 (adjusted determination coeffi-

cient) [15] values should be closer to 1 and also closer to each other shows
that the model used most of the variability in soil parameters.
∑𝑛𝑖=1(𝑑𝑖 − 𝑑𝑚𝑒𝑎𝑛 )2 − ∑𝑛𝑖=1(𝑑𝑖 − 𝑦𝑖 )2
𝑅2 =
∑𝑛𝑖=1(𝑑𝑖 − 𝑑𝑚𝑒𝑎𝑛 )2 (5)

(𝑛 − 1)
𝐴𝑑𝑗𝑅2 = 1 − (1 − 𝑅2 )
(𝑛 − 𝑝 − 1) (6)

5. Performance Index (PI) [16] indicates the performance of the models.

𝑃𝐼 = 𝑎𝑑𝑗. 𝑅2 + 0.01𝑉𝐴𝐹 − 𝑅𝑀𝑆𝐸
(7)
6

6. Bias Factor is a factor whose value more than unity represents the overesti-
mated model, value of less than unity represents an underestimation model,
and a value of unity indicates a prediction which is unbiased [17].
𝑛
1 𝑦𝑖
Bias Factor= ∑
𝑁 𝑑𝑖 (8)
𝑖=1

7. Root mean square error to observation’s Standard deviation Ratio (RSR) [18]
have the benefit of error index statistics. More the value closer to 0 more the
is the prediction power.
𝑅𝑀𝑆𝐸
𝑅𝑆𝑅 =
1
√ ∑𝑛𝑖=1(𝑑𝑖 − 𝑑𝑚𝑒𝑎𝑛 )2 (91)
𝑁

8. Normalized Mean Bias Error (NMBE) calculates the model’s ability to predict
a value which is away from the mean value. NMBE equal to 0 indicates perfect
model [19].
1 𝑛
∑ (𝑦 − 𝑑𝑖 )
𝑁 𝑖=1 𝑖
𝑁𝑀𝐵𝐸(%) = 1 𝑛 × 100
∑𝑖=1 𝑑𝑖 (102)
𝑁

9. Mean Absolute Percentage Error (MAPE) [20] value closer to 0 shows high
prediction accuracy.
𝑛
1 𝑑𝑖 − 𝑦𝑖
𝑀𝐴𝑃𝐸 = ∑ | |
𝑁 𝑑𝑖 (11)
𝑖=1

10. Relative Percentage Difference (RPD) [21] is given in eq. 13

Table 1 RPD values for evaluating models.

RPD Model type
<1 Very poor models
1.0 - 1.4 Poor models
1.4 - 1.8 Fair models
1.8 - 2.0 Good models
2.0 - 2.5 Very good models
> 2.5 Excellent models

𝑆𝐷
𝑅𝑃𝐷 = (12)
𝑅𝑀𝑆𝐸

11. Willmott’s Index of agreement (WI) shows the degree of model prediction
error. WI range is from 0 to 1 and WI = 1 shows perfect model [22-23].
 7

∑𝑁
𝑖=1(𝑑𝑖 − 𝑦𝑖 )
2
𝑊𝐼 = 1 − [ 2]
∑𝑁 (13)
𝑖=1(|𝑦𝑖 − 𝑑𝑚𝑒𝑎𝑛 | + |𝑑𝑖 − 𝑑𝑚𝑒𝑎𝑛 |)

12. Mean Bias Error (MBE) and Mean Absolute Error (MAE) values closer to 0
shows lesser error in prediction [24].
𝑛
1
𝑀𝐵𝐸 = ∑(𝑦𝑖 − 𝑑𝑖 )
𝑁
𝑖=1 (14)

𝑛
1
𝑀𝐴𝐸 = ∑|(𝑦𝑖 − 𝑑𝑖 )|
𝑁 (15)
𝑖=1
13. The Range of Legate and McCabe’s Index (LMI) is (-∞, 1) [25-26]. Values
closer to 1 represents a perfect model. The lesser the value, the more is diver-
gence between observed and predicted values.
∑𝑁
𝑖=1 |𝑑𝑖 − 𝑦𝑖 |
𝐿𝑀𝐼 = 1 − [ 𝑁 ]
∑𝑖=1 |𝑑𝑖 − 𝑑𝑚𝑒𝑎𝑛 | (16)

14. Expanded uncertainty (U95) indicates the short-term performance of the

model. Smaller the value high the performance of model [27].
𝑈95 = 1.96(𝑆𝐷 2 + 𝑅𝑀𝑆𝐸 2 )1/2
(17)

15. t-statistic smaller value indicates the superior performance of model [28].
(𝑁 − 1)𝑀𝐵𝐸 2
𝑡-stat = √ (18)
𝑅𝑀𝑆𝐸 2 − 𝑀𝐵𝐸 2

16. Global Performance Indicator (GPI) use all the parameters to analyze model
in a single parameter. Higher the value of GPI higher is the accuracy of model.
GPI = MBE × RMSE × 𝑈95 × 𝑡𝑠𝑡𝑎𝑡 × (1 − 𝑅2 )
(19)

17. Reliability Index (β) is a parameter to check the reliability of the model. It is
calculated using FOSM. Value of reliability index between 3 and 4 indicates
good performance of model [29].
µ−1
𝛽= (20)
𝜎

Here di and yi are the observed and predicted ith value respectively, dmean is the aver-
age of observed value, SD is the standard deviation, is the standard deviation and
is the mean of the dataset generated. n is the number of training or testing samples and
p is the model input quantity.
8

5. Result and discussion

Table 2. Performance assessment using various parameters of both the models ANN
and ANFIS

S. ASSESMENT ANN ANN ANFIS ANFIS

N VALUES (TRAINING (TESTING) (TRAINING) (TESTING
O. ) )

1. WMAPE 0.0077 0.0201 8.83E-05 0.0224

2. NS 0.9992 0.989 1 0.9832

3. RMSE 0.0301 0.0722 0.0004 0.0894
4. VAF 99.942 99.285 100 98.615
5. R2 0.9992 0.989 1 0.9832
6. AdjR2 0.9992 0.9867 1 0.9797
7. PI 1.9685 1.9074 1.9996 1.8764
8. RMSD 0.0301 0.0722 0.0004 0.0894
9. BIAS
0.9947 0.9831 1 0.9819
FACTOR
10 RSR
0.028 0.1047 0.0004 0.1296
.
11 NMBE
-0.5659 -1.739 0.0009 -1.53
.
12 MAPE
0.021 0.0492 0.0002 0.0551
.
13 RPD
0.8573 0.8073 0.9978 0.9078
.
14 WI
0.9998 0.9972 1 0.9959
.
15 MAE
0.021 0.0492 0.0002 0.0551
.
16 MBE
-0.0155 -0.043 2.53E-05 -0.037
.
17 LMI
0.9764 0.9096 0.9997 0.8988
.
18 U95
0.0776 0.1819 0.001 0.2366
.
19 t-stat
4.4526 3.5057 0.4982 2.216
.
20 GPI -2.15E-
-1.26E-07 6.16E-19 -2.95E-05
. 05
21 β
1.6134 2.0884 1.6174 1.9751
.
 9

All the models are analyzed on the basis of various parameters (table 1) VAF,
RMSE, R2, Adj. R2, MAE, PI, RSR, NS, the bias factor, LMI, NMBE, RPD, MAPE,
U95, t-statistic, GPI and β.
From the observation table 2 it is visible that ANFIS is a better model than ANN but
with less deviation of assessment values from the desired values for instance bias factor
is almost 1 in the ANFIS model and this predicts that model is neither overestimated
nor underestimated. Also, RSR is almost zero in ANFIS which is as desired out of the
model. RPD value is less than 1 in ANN which makes ANFIS comparatively better
than ANN in performance. Also, MAE and MBE when calculated gives result in the
favor of ANFIS but ANN shares the same range i.e. its value is near to zero. There are
other variables for which values are not distinct for both the models but ANFIS has an
upper hand therefore it is suggested to use ANFIS above ANN. But it is observed that
training data model is more appropriate as compared to that of testing in case of ANFIS,
as there is ample amount of input in training model (70 percent) therefore we can con-
clude that adaptive networks work good with fuzzy logic in wide range of data. Also,
if value of reliability index is compared with the reliability index value of observed
dataset (1.6174 for 70% observed data and 2.103 for 30% observed data) it is observed
that both models data almost coincide the observed dataset. Overall assessment con-
cludes that ANFIS is better to work with if wide range of data is to be worked upon but
ANN cannot be discarded as a model because it works equally well for both training
and testing models.

ANFIS MODEL
1.2
PREDICTED VALUE OF FOS

0.8

0.6

0.4

0.2

0
ACTUAL VALUE OF FOS

Fig. 3. Actual values vs. Predicted values of FOS using ANN model
10

Figure 3 and 4 shows the plot of predicted values against actual or observed values
of both training and testing data using ANN and ANFIS model. It is observed that data
points coincide each other and there is not much difference in the result. Therefore, it
can be concluded that these models prediction power is high. But when they are exam-
ined with other parameters, overall assessment shows that ANFIS model is better than
the ANN model.

1.2
ANFIS MODEL
PREDICTED VALUE OF FOS

0.8

0.6 TRAINING

0.4 TESTING

0.2

0
ACTUAL VALUE OF FOS

Fig. 4. Actual values vs Predicted values of FOS using ANFIS model

5.1 AOC-REC Curve

Regression Error Characteristics curve (REC) [30] is a probability curve and a metric
system to check performance of regression model. Area Over Curve (AOC) is the meas-
ure of distinction of predicted data of the model from the actual data.
From the AOC value it is clear that due to less value of AOC of ANFIS training
model, it proves to be a better model as compared to the other models. Also, it is ob-
served that AOC value of ANN testing model is less than ANFIS testing model, it is
already mentioned in sections above that fuzziness works good with wide range of data.

5.2 Taylor diagram

Taylor diagram [31] is the graphical representation of how closely the pattern (or pat-
terns) matches observation which is quantified in terms of correlation, root mean
square error and amplitude of their variations (standard deviations). This diagram
evaluates the aspects of different complex models and performs a comparative analysis
of these models with the reference data (self-observed data).
 11

Fig. 5. REC curve plot for ANN and ANFIS (training data set)

Table 3. Area Over REC plot value for ANN and ANFIS models.

MODELS AOC
ANN (Training) 0.0200
ANFIS (Training) 2.2721e-04
ANN (Testing) 0.0449
ANFIS (Testing) 0.0498

Fig 6. Taylor Diagram for ANN and ANFIS

12

Figure 6 shows that both models lie near to the observed value and deviation is quite
less. Therefore, both these models have a good performance and overall experimenta-
tion shows that there is a good agreement between the predicted values generated from
the models and actual values calculated.
Figure 7 is the plot of Fp/Fm against cumulative probability for both ANN and
ANFIS model respectively. From figure 7 result can be extracted that for
ANFIS(training) at P50, value of Fp/Fm is near to 1 i.e., for training set, value of Fp/Fm
is 1.0000 and for testing set, value is 0.9916. whereas for ANN model at P50 Fp/Fm is
0.9950 and 0.9876 for training and testing data sets respectively. Therefore, ANFIS
observation is comparable to ANN but to due to slight change and less deviation from
the value 1 ANFIS acts as a better model. But when P 90 is checked for, it is seen that
all the values of all the models are almost one. Therefore, both these models stand a
good chance but ANFIS has already proven to be a better model. Figure 8 is the plot of
probability density function against Fp/Fm for both ANN and ANFIS training and test-
ing dataset. This plot is lognormal distribution for Fp/Fm and from this plot it is ob-
served that ANFIS functions better in training period than ANN as probability of FOS
within 20% accuracy level is concerned under lognormal distribution and it is clear
from figure 8 that most of the points lie in the region that is under 20% accuracy level
also it is visible that for ANN there is deviation of graph from 1. Therefore, ANFIS is
better than ANN. But for testing dataset both ANN and ANFIS show same accuracy
levels.

Fig. 7. Cumulative probability plot for Fp/Fm for ANN and ANFIS models
 13

45
PROBABILITY DENSITY FUNCTION

40
35
30
25
20
15
10
5
0
0.88 0.9 0.92 0.94 0.96 0.98 1 1.02 1.04
Fp/Fm
ANN(training) ANFIS(training) ANN(testing) ANFIS(testing)

Fig. 8. Lognormal distribution for Fp/Fm for ANN and ANFIS for training and
testing data

6. Conclusion
In this paper, reliability index of the retaining wall with c, φ, γ and δ as input parameters
along with other performance assessment values were calculated for ANN and ANFIS
models. Bothe these models were compared with different parameters and both of them
showed equal efficiency. Also, their results almost converged to the same value. But
ANFIS with wide range of data outperformed as fuzziness works good with ample data.
70 % of total data was used for training and 30% for testing therefore in training dataset
modeling ANFIS performed better but for testing dataset both ANN and ANFIS per-
formed equally well converging to same result or just say FOS value 1. Therefore, both
these models can be used as soft computing technique for computing the factor of safety
as testing model does and then calculating the reliability of the retaining wall. With
collective observations ANFIS can be concluded as the better model amongst both.

7. References

1. Wu, Tien H., and Leland M. Kraft.: Safety analysis of slopes. Journal of Soil Me-
chanics & Foundations Div., 609-632 (1970).
2. C.A. Cornell.: First-order uncertainty analysis of soils deformation and stability. in:
Proceedings of the 1st International Conference on Application of Probability and Sta-
tistics in Soil and Structural Engineering (ICAPI), pp. 129–144, Hong Kong, (1971).
14

3. F.S. Wong.: Slope reliability and response surface method. Journal of Geotechnical
Engineering Div. ASCE, 111(1), 32–53, (1985).
4. Faravelli, Lucia.: Response-surface approach for reliability analysis. Journal of En-
gineering Mechanics 115(12), 2763-2781 (1989).
5. Guan, X. L., and R. E. Melchers.: Multitangent-plane surface method for reliability
calculation. Journal of Engineering Mechanics 123(10), 996-1002 (1997).
6. Gomes, Herbert Martins, and Armando Miguel Awruch.: Comparison of response
surface and neural network with other methods for structural reliability analysis. Struc-
tural safety 26(1), 49-67, (2004).
7. Deng, Jian, et al.: Structural reliability analysis for implicit performance functions
using artificial neural network. Structural Safety 27(1), 25-48, (2005).
8. Shahin, Mohamed A., Mark B. Jaksa, and Holger R. Maier.: Artificial neural network
applications in geotechnical engineering. Australian geomechanics 36(1), 49-62(
2001).
9. Saravanan Kand S. Sasithra.: Review on classification based on artificial neural net-
works. International Journal of Ambient Systems and Applications (IJASA) 2(4), De-
cember 2014
10. Kandel, E.R., Schwartz, J.H. and Jessell, T.M. eds.: Principles of neural science vol.
4, 1227-1246, McGraw-hill, New York (2000)
11. Jain, Sharad K., and K. P. Sudheer.: Fitting of hydrologic models: a close look at
the Nash–Sutcliffe index. Journal of hydrologic engineering 13(10), 981-986 (2008).
12. Kisi, Ozgur, Jalal Shiri, and Mustafa Tombul..: Modeling rainfall-runoff process
using soft computing techniques. Computers & Geosciences 51, 108-117 (2013).
13. Grima, M. Alvarez, and R. Babuška.: Fuzzy model for the prediction of unconfined
compressive strength of rock samples. International Journal of Rock Mechanics and
Mining Sciences 36(3), 339-349 (1999).
14. Gokceoglu, Candan.: A fuzzy triangular chart to predict the uniaxial compressive
strength of the Ankara agglomerates from their petrographic composition. Engineering
Geology 66(1-2), 39-51 (2002).
15. Babu, GL Sivakumar, and Amit Srivastava.: Reliability analysis of allowable
pressure on shallow foundation using response surface method. Computers and
Geotechnics 34(3), 187-194 (2007).
16. Kung, Gordon T., et al.: Simplified model for wall deflection and ground-surface
settlement caused by braced excavation in clays. Journal of Geotechnical and Geoen-
vironmental Engineering 133(6), 731-747 (2007).
17. Prasomphan, Sathit, and Shigeru Mase.: Generating prediction map for geostatisti-
cal data based on an adaptive neural network using only nearest neighbors. Interna-
tional Journal of Machine Learning and Computing 3(1), 98 (2013).
18. Moriasi, Daniel N., et al.: Model evaluation guidelines for systematic quantification
of accuracy in watershed simulations. Transactions of the ASABE 50(3), 885-900
(2007).
19. Srinivasulu, Sanaga, and Ashu Jain.: A comparative analysis of training methods
for artificial neural network rainfall–runoff models. Applied Soft Computing 6(3), 295-
306 (2006).
20. Armstrong, J. Scott, and Fred Collopy.: Error measures for generalizing about fore-
casting methods: Empirical comparisons. International journal of forecasting 8(1), 69-
80 (1992).
 15

21. Rossel, RA Viscarra, R. N. McGlynn, and A. B. McBratney.: Determining the com-

position of mineral-organic mixes using UV–vis–NIR diffuse reflectance spectroscopy.
Geoderma 137(1-2), 70-82 (2006).
22. Willmott, Cort J.: On the validation of models. Physical geography 2(2), 184-194
(1981).
23. Willmott, Cort J.: Some comments on the evaluation of model performance. Bulle-
tin of the American Meteorological Society 63(11), 1309-1313 (1982).
24. Raventos-Duran, T., et al.: Structure-activity relationships to estimate the effective
Henry's law constants of organics of atmospheric interest. Atmospheric Chemistry and
Physics 10(16), 7643-7654 (2010).
25. Legates, David R., and Robert E. Davis.: The continuing search for an anthropo-
genic climate change signal: Limitations of correlation‐based approaches. Geophysical
Research Letters 24(18), 2319-2322 (1997).
26. Legates, David R., and Gregory J. McCabe.: Evaluating the use of “goodness‐of‐
fit” measures in hydrologic and hydroclimatic model validation. Water resources re-
search 35(1), 233-241 (1999).
27. Gueymard, Christian A.: A review of validation methodologies and statistical per-
formance indicators for modeled solar radiation data: Towards a better bankability of
solar projects. Renewable and Sustainable Energy Reviews 39, 1024-1034 (2014).
28. Stone, R. J.: Improved statistical procedure for the evaluation of solar radiation
estimation models. Solar energy 51(4), 289-291 (1993).
29. USACE Risk-based analysis in geotechnical engineering for support of planning
studies, engineering and design. Dept Army, USACE Washington, DC ll, (1999).
30. Fawcett, Tom.: An introduction to ROC analysis. Pattern recognition letters 27(8),
861-874 (2006).
31. Taylor, Karl E.: Summarizing multiple aspects of model performance in a single
diagram. Journal of Geophysical Research: Atmospheres 106(D7), 7183-7192 (2001).