Efstratiadis 2010 One Decade of Multi Objective Calib

Hydrological Sciences Journal – Journal des Sciences
Hydrologiques
ISSN: 0262-6667 (Print) 2150-3435 (Online) Journal homepage: https://www.tandfonline.com/loi/thsj20
One decade of multi-objective calibration

approaches in hydrological modelling: a review
Andreas Efstratiadis & Demetris Koutsoyiannis
To cite this article: Andreas Efstratiadis & Demetris Koutsoyiannis (2010) One decade of multi-
objective calibration approaches in hydrological modelling: a review, Hydrological Sciences
Journal – Journal des Sciences Hydrologiques, 55:1, 58-78, DOI: 10.1080/02626660903526292
To link to this article: https://doi.org/10.1080/02626660903526292
Published online: 11 Mar 2010.
Submit your article to this journal
Article views: 7292
View related articles
Citing articles: 64 View citing articles
Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=thsj20
58 Hydrological Sciences Journal – Journal des Sciences Hydrologiques, 55(1) 2010
THSJ
One decade of multi-objective calibration approaches in hydrological

modelling: a review
Andreas Efstratiadis & Demetris Koutsoyiannis

Multi-objective calibration approaches in hydrological modelling
Department of Water Resources and Environmental Engineering, School of Civil Engineering, National Technical University of Athens,
Heroon Polytechneiou 5, GR-15780 Zographou, Greece
[email protected]
Received 30 July 2008; accepted 31 August 2009; open for discussion until 1 August 2010
Citation Efstratiadis, A. & Koutsoyiannis, D. (2010) One decade of multi-objective calibration approaches in hydrological modelling:
a review. Hydrol. Sci. J. 55(1), 58–78.
Abstract One decade after the first publications on multi-objective calibration of hydrological models, we
summarize the experience gained so far by underlining the key perspectives offered by such approaches to improve
parameter identification. After reviewing the fundamentals of vector optimization theory and the algorithmic issues,
we link the multi-criteria calibration approach with the concepts of uncertainty and equifinality. Specifically, the
multi-criteria framework enables recognition and handling of errors and uncertainties, and detection of prominent
behavioural solutions with acceptable trade-offs. Particularly in models of complex parameterization, a multi-
objective approach becomes essential for improving the identifiability of parameters and augmenting the
information contained in calibration by means of both multi-response measurements and empirical metrics (“soft”
data), which account for the hydrological expertise. Based on the literature review, we also provide alternative
techniques for dealing with conflicting and non-commeasurable criteria, and hybrid strategies to utilize the
information gained towards identifying promising compromise solutions that ensure consistent and reliable
calibrations.
Key words multi-objective evolutionary algorithms; multiple responses; uncertainty; equifinality; hybrid calibration; soft
data
Une décennie d’approches de calage multi-objectifs en modélisation hydrologique: une revue

Résumé Une décennie après les premières publications sur le calage multi-objectifs des modèles hydrologiques,
nous résumons l’expérience acquise jusqu’ici en soulignant les perspectives clefs offertes par de telles approches
pour améliorer l’identification des paramètres. Après la revue des éléments fondamentaux de la théorie de
l’optimisation de vecteurs et des problèmes algorithmiques, nous relions l’approche de calage multi-critères avec
les concepts d’incertitude et d’équifinalité. Spécifiquement, le cadre multi-critères permet de reconnaître et de
gérer des erreurs et des incertitudes, et d’identifier les principales solutions comportementales selon des compromis
acceptables. En particulier pour des modèles ayant un paramétrage complexe, une approche multi-objectifs devient
essentielle pour améliorer l’identification des paramètres et augmenter l’information contenue dans le calage au
moyen de mesures à réponses multiples et de métriques empiriques (données “molles”), qui tiennent compte de
l’expertise hydrologique. Sur la base d’une revue de la littérature, nous fournissons également des techniques
alternatives pour gérer les critères contradictoires et incommensurables, et des stratégies hybrides pour utiliser
l’information obtenue durant l’identification de compromis prometteurs qui assurent des calages cohérents et
fiables.
Mots clefs algorithmes évolutifs multi-objectifs; réponses multiples; incertitude; équifinalité; calage hybride; données molles
1 INTRODUCTION the formulation of a scalar performance criterion

(objective function) that measures the differences
Even today, a common practice of parameter estima-
between the two sets (i.e. simulated and observed
tion in hydrological modelling is built on the hypoth-
values), the determination of lower and upper bounds
esis that a unique set of parameter values exists that
for the model parameter values (control variables),
ensures a “global optimum” fitting of the computed
and the selection of a robust searching procedure
model responses to the observed ones. This involves
(algorithm) to optimize the parameters with respect
ISSN 0262-6667 print/ISSN 2150-3435 online
© 2010 IAHS Press
doi:10.1080/02626660903526292
http://www.informaworld.com
Multi-objective calibration approaches in hydrological modelling 59
to the aforementioned criterion. This automatic cali- alternative parameter sets that are optimal, on the
bration practice was significantly favoured by the basis of the Pareto-dominance concept explained
great improvement of computer capabilities (in terms below (Section 2.1). Madsen & Khu (2002) report
of both memory and processing speed), as well as by that early attempts are found in the work of Harlin
the development of advanced nonlinear optimization (1991), who formulated an iterative procedure that
methods, most of which were implemented within focuses on different process descriptions and associ-
evolutionary schemes. Such methods have been ated performance measures. However, the use of
proved effective and efficient against the various automatic routines employing Pareto-based calibra-
peculiarities (e.g. multiple peaks at all scales, discon- tion was only established in the last decade, after the
tinuous first derivatives, extended flat areas, long and pioneering work by Yapo et al. (1998), while multi-
curved multi-dimensional ridges, etc.) of the highly objective optimization approaches appeared in water
non-convex response surfaces, which derive from the resources technology a few years earlier (Ritzel et al.,
typical fitting measures used within hydrological 1994; Cieniawski et al., 1995; Halhal et al., 1997).
calibration. These issues are thoroughly analysed in Here we review the recent history of multi-
the classic work of Duan et al. (1992; see also Beven, objective hydrological calibration and its usefulness
2001, pp. 219–222). towards establishing more faithful and consistent
Despite the progress of the “algorithmic” com- models. The following section presents the mathe-
ponent of the parameter estimation procedure, it was matical background of multi-objective optimization
soon recognized that the above approach has many and the relevant computer tools. Next, we introduce
drawbacks, since it may result in a black-box mathe- the concepts of uncertainty and equifinality as well
matical game that fails to ensure satisfactory predic- as their relationship with the parameter estimation
tive capacity and realistic parameter values. Thus, procedure. In the following section we investigate
many researchers demonstrated the necessity for five key issues of multiple objective model fitting,
establishing a more powerful paradigm that takes taking into account the experience obtained from
into account the inherent multi-objective nature of characteristic examples from literature. The possible
the calibration problem and the major role of model drawbacks as well as the future perspectives of
errors and uncertainties (Gupta et al., 1998). This multi-objective calibration are discussed in the
issue became more imperative due to the expansion closing section.
of complex modelling schemes (semi- or fully-
distributed) to represent multiple fluxes and reflect
the spatial heterogeneities of the hydrological mech- 2 MULTI-OBJECTIVE SEARCH:
anisms and their related attributes across a river MATHEMATICAL BACKGROUND
basin. Several studies (e.g. Mroczkowski et al., 1997; AND COMPUTER TOOLS
Refsgaard, 1997; Gupta et al., 1998; Kuczera & 2.1 Fundamental notions
Mroczkowski, 1998; Franks et al., 1999) revealed the
utility of conditioning hydrological models on multi- A multi-objective search problem involves the simulta-
ple responses (or various aspects of each single neous optimization (for convenience, minimization) of
response), in order to reduce uncertainties and pro- m numerical measures that represent the components
vide more faithful predictions. Moreover, the (criteria) of a vector objective function f(x) = [f1(x),
hypothesis of parameter set uniqueness, where the f2(x), … fm(x)], with respect to a vector of control varia-
global calibration paradigm is founded, has been bles x ∈ X, where X ⊆ Rn is the feasible control space;
intensively disputed in favour of the so-called “equi- assuming unconstrained optimization, except for the
finality” concept (Beven & Binley, 1992; Beven, control variable bounds (which is the typical configura-
1993), where multiple model and parameter configu- tion in hydrological calibration problems), the feasible
rations are considered as acceptable simulators of the space becomes a hyper-rectangle in Rn.
real-world system. When the criteria are conflicting, there is no fea-
Accordingly, during the past years, much atten- sible point that optimizes all of them simultaneously.
tion has been given to employing vector (instead of In that case, we look for acceptable trade-offs rather
scalar) search techniques to optimize the model than a unique solution, according to the fundamental
parameters. This allows for incorporating multiple concept of Edgeworth-Pareto optimality (commonly
criteria within calibration to provide a number of referred to as Pareto optimality), introduced within
60 A. Efstratiadis & D. Koutsoyiannis
welfare economics theory at the end of 19th century. problems are its subjectivity (e.g. in choosing
In particular, we define a vector of control variables weights) and the fact that it hides the competitions
x* to be Pareto optimal if there does not exist among the conflicting criteria. Additionally, a step-
another feasible vector x such that fi(x) ≤ fi(x*) for all by-step approximation of representative trade-offs is
i = 1, …, m and fi(x) < fi(x*) for at least one i. The computationally inefficient or even, in the case of
above definition implies that x* is Pareto optimal if non-convex Pareto fronts, infeasible. Finally, when
there is no feasible vector that would improve some incommensurate criteria are involved, the use of
criterion without causing a simultaneous deteriora- aggregation schemes without appropriate scaling
tion of at least one other criterion. results in extremely rough response surfaces.
The concept of Pareto optimality leads to a set
of feasible vectors, called the Pareto set and sym-
2.3 Multi-objective evolutionary algorithms
bolized as X* ⊂ X; all Pareto optimal vectors x* ∈ X*
(MOEAs)
are called non-inferior or non-dominated. The
image of the non-dominated set in the objective Evolutionary algorithms (EAs) are well-established
space is called the Pareto front, denoted as F*. In tools for handling nonlinear optimization problems
the absence of further information, all non- of any complexity. Their key feature is the parallel
dominated solutions are assumed equivalent or, search of the feasible space, through a set (popula-
according to the formal mathematical terminology, tion) of randomly generated points that evolves on
indifferent. However, within real-world decision- the basis of stochastic transition schemes, e.g. the
making, it is usually required to determine a single genetic operators. Their multi-objective versions aim
solution from the Pareto set; the latter is called the to spread the population along the Pareto front
best-compromise solution and is either selected by instead of converging around a single optimum. For
“intuition” or systematically, i.e. on the basis of this purpose, some essential adaptations are imple-
external criteria or by maximizing a utility function, mented with the original selection mechanisms of
which allows the comparison of all alternative solu- EAs, by assigning dummy fitness values to the indi-
tions, even the indifferent ones, on the basis of a viduals, to guide the search mechanism towards well-
scalar measure (Cohon, 1978, pp. 164–173). distributed non-dominated solutions.
Early multi-objective evolutionary attempts
appeared in the mid-1980s. The first is the Vector
2.2 Classical approaches through aggregating
Evaluated Genetic Algorithm (VEGA) by Shaffer
schemes
(1984), where the population is divided into sub-sets,
Optimization problems involving multiple and con- each one evolving according to a different criterion;
flicting objectives have been traditionally handled by thus, for a problem with m objectives, m sub-popula-
combining the objectives into a scalar function and, tions, each of size N/m, are generated, assuming a
next, solving the equivalent single-optimization population of N points. These sub-populations are
problem to identify the best-compromise solution. then shuffled together to get a new population, on
The combination schemes, usually referred to as which the genetic operators are employed. However,
aggregating functions, are the oldest mathematical clear Pareto approaches (commonly referred as first-
programming approaches, since they originate from generation techniques), using the dominance con-
the Kuhn-Tucker conditions for non-dominated cept, were developed in the mid-1990s. The most
solutions (Cohon, 1978, pp. 77–82). The character- representative were the Multi-Objective Genetic
istics of the optimal solution are expressed using Algorithm (MOGA; Fonseca & Fleming, 1993), the
multipliers (e.g. the weighting method), target- Nondominated Sorting Genetic Algorithm (NSGA;
values (e.g. goal-programming, goal-attainment and Srinivas & Deb, 1994) and the Niched-Pareto
e-constraint methods) or priorities (e.g. lexico- Genetic Algorithm (NPGA; Horn et al., 1994). Their
graphic ordering). By changing the arguments of common strategy involves the assignment of dummy
the aggregating function (e.g. the weighting coeffi- fitness functions on the basis of Pareto ranking or
cients), one can obtain alternative solutions from slight variations of it (Goldberg, 1989, pp. 99–101),
the Pareto set. and fitness sharing, which enables diversity to be
The above approach to multi-objective optimiza- maintained and avoids convergence to single solu-
tion has some serious disadvantages. The major tions (Coello Coello, 2005).
More recent advances on MOEAs, known as which implements a generalized definition of domi-
second-generation approaches, introduce the notion nance to effectively handle problems with more than
of elitism that denotes the use of an archive or exter- two criteria, and also imposes feasibility bounds on the
nal population to retain the non-dominated individu- objective space. This allows rejection of non-dominated
als found so far that eliminate the risk to be lost due solutions that lie on the outer ends of the Pareto front,
to random effects. In addition, they aim to provide thus focusing only on trade-offs with practical interest.
more efficient ranking and clustering schemes used
within the fitness evaluation procedure. Some of the
most popular algorithms, according to the state-of- 3 UNCERTAINTY, EQUIFINALITY AND
the-art review of Coello Coello (2005), are the MULTI-OBJECTIVE CALIBRATION OF
Strength Pareto Evolutionary Algorithm (SPEA; HYDROLOGICAL MODELS
Zitzler & Thiele, 1999) and its successor SPEA II 3.1 The concepts of uncertainty and equifinality
(Zitzler et al., 2001), the Pareto Archive Evolution
in hydrological modelling
Strategy (PAES; Knowles & Corne, 2000), the Non-
dominated Sorting Genetic Algorithm II (NSGA II; Uncertainty is a structural and inevitable characteris-
Deb et al., 2002), the Pareto Envelope-based Selec- tic of all hydrological processes, arising from the
tion Algorithm (PESA; Corne et al., 2001) and the intrinsic complexity of the related natural systems. In
Micro Genetic Algorithm (Coello Coello & Pulido, water resources engineering, the management of
2001). An extended and systematically updated uncertainty is of major interest, and necessary to
repository containing MOEA references and tools is account for the risk within planning (e.g. uncertainty
available at www.lania.mx/∼ccoello/EMOO/. in the design variables) and decision-making (e.g.
The contribution of hydrologists in the develop- uncertainty in the forecasts; Montanari, 2007). Yet,
ment of MOEAs is not negligible. Significant the wide use of deterministic tools for hydrological
progress was made at the University of Arizona, ini- predictions introduces additional burden to uncer-
tially with the Multi-objective Complex Evolution tainty handling. Uncertainty originates from the
(MOCOM) algorithm (Yapo et al., 1998) and the inherent complexity of natural mechanisms, as well
Multi-objective Shuffled Complex Evolution as from errors and inappropriate assumptions within
Metropolis algorithm (MOSCEM; Vrugt et al., the entire modelling procedure. These errors or
2003a). The former is a first-generation multi-objec- assumptions, forming the so-called “epistemic”
tive optimizer that employs Pareto ranking within a uncertainty, span from the field observations to the
simplex-based pattern in the objective space. The conceptualization of processes and the parameter
MOSCEM algorithm is an extended version of the estimation strategy. Specifically, epistemic uncer-
SCEM-UA method for uncertainty assessment tainty is related to the following factors: (a) measure-
(Vrugt et al., 2003b), and merges the strength of ment errors; (b) use of over-parameterized model
complex shuffling with the probabilistic covariance- structures, whose complexity is inconsistent with the
based search strategy of the Metropolis algorithm available information about the system behaviour;
and the fitness assignment procedure employed (c) inappropriate representation of the temporal and
within the SPEA algorithm (Zitzler & Thiele, 1999). spatial variability of model inputs, which are
Reed et al. (2003) proposed an enhanced version of obtained either from processed data (e.g. discharge
the NSGA-II method, called e-NSGA-II, where they records based on stage information) or point observa-
employ e-dominance archiving, adaptive population tions (e.g. precipitation, temperature); (d) poor identi-
sizing and automatic termination to minimize the fication of initial and boundary conditions; (e) non-
need for extensive parameter calibration. Notably, informativeness of calibration data with regard to the
the concept of e-dominance allows users to specify entire system regime; (f) use of statistically inconsist-
the precision with which they want to quantify each ent fitting criteria (e.g. error metrics not accounting for
objective to optimize. The procedure was also built heteroscedasticity); (g) weaknesses of nonlinear opti-
within a parallelization framework, which radically mization algorithms on rough and high-dimensional
improves the efficiency and reliability of the multi- response surfaces; and (h) inconsistent assumption of
objective search (Tang et al., 2007). Another example parameters constant in time whilst the environment is
is the Multi-objective Evolutionary Annealing-Simplex changing, e.g. due to urbanization, deforestation,
method (MEAS; Efstratiadis & Koutsoyiannis, 2008), stream lining and other human interventions (Beven
& Binley, 1992; Wagener & Gupta, 2005; Rosbjerg against uncertainties (Seibert & McDonnell, 2002;
& Madsen, 2005; Engeland et al., 2005; Efstratiadis Wagener & Gupta, 2005; Beven, 2006). The limita-
et al., 2008; Beven et al., 2008). Evidently, models tions of the unique parameter set concept have been
are, by nature, imperfect representations of the real emphasized by Beven & Binley (1992) and Beven
world and thus model uncertainty, even though it (1993), who introduced the term “equifinality” to
may be decreased in some of the above components, illustrate the existence of multiple “behavioural”
will be always present. parameter sets, which are all acceptable albeit not
The classical paradigm of model fitting on obser- equivalent, on the basis of different conceptualiza-
vations through automatic optimization based on a tions, data and fitting criteria. It is clearly admitted that
single performance criterion conceals all above issues, equifinality arises from uncertainty (Freer et al.,
since the entire procedure degenerates to a “computa- 1996), thus making it impossible to identify a “global”
tional trick” of recycling errors and uncertainties optimal simulator that definitely better reproduces the
(Fig. 1). Yet, non-expert users often adopt such a entire hydrological regime of a river basin. Even when
black-box approach, which may result in: (a) ostensi- assuming a specific structure and a single performance
ble best-fitted parameter values that are inconsistent measure (a scalar calibration function) it remains diffi-
with their physical interpretation; (b) poor predictive cult to locate a unique solution whose measure differs
model capacity against an independent control period significantly from other feasible ones across the search
(validation); and (c) unreasonable regimes of model space. Such poor parameter identifiability may result
responses that are not controlled by measurements in considerable uncertainty in the model outputs and,
(e.g. evapotranspiration, underground losses) as well also, preclude relating of the optimized parameter val-
as internal model variables (e.g. soil and groundwater ues with the observable characteristics of the basin
storage) (Refsgaard, 1997; Wagener et al., 2001; (Vrugt et al., 2003b).
Rozos et al., 2004; Efstratiadis et al., 2008). All the Current advances in hydrological research pro-
above are contrary to the targets of the traditional vide a variety of computational techniques to deal
manual calibration, which requires a comprehensive with these drawbacks and quantify the model predic-
understanding of the model, the real system and the tive uncertainty, by seeking for promising trajectories
data, to ensure reliable results (Boyle et al., 2000). of its outputs on the basis of different parameter sets.
The context examined so far reveals a typical So far, the most common uncertainty assessment pro-
conflict in hydrological modelling, where the principle cedure is the Generalized Likelihood Uncertainty
of consistency (i.e. building models that are consistent Estimation (GLUE), proposed by Beven & Binley
with the behaviour of the real system) has been gener- (1992) and applied in a wide range of hydrological and
ally accepted as a working paradigm instead of the environmental models. Founded on a quasi-Bayesian
principle of optimality, since the latter is too weak framework of uncertainty, it employs Monte Carlo
Raw input data Processed

(point observations)
?
input data
Initial &
? boundary
? conditions
Real input data
(distributed)
Hydrological Optimized Optimization
?
model parameters procedure
?
Real response ?
data (continuous)
Computed Fitting
responses function
Raw response Processed

? ?
data (discrete) response data
Fig. 1 An automatic calibration procedure – a black-box game of recycling errors and uncertainties.
simulation, assuming a known prior distribution of of the observed responses. Hence, the almost negligible
the parameter values, in order to identify behavioural dissemination of similar approaches in problems of the
parameter sets according to either a single or multi- every day engineering practice and the reluctance to
ple, appropriately combined, likelihood measures. provide uncertainty estimation results to decision-mak-
Next, the empirical cumulative likelihood weighted ers and stakeholders is not surprising. Besides, the sci-
distribution of simulations is used to estimate quan- entific community remains sceptical, if not divided,
tiles for model predictions at any time step (Beven, about the concepts of uncertainty and equifinality and
2001, pp. 234–240). the proper use of Bayesian inference methods in hydro-
While the GLUE method estimates the global logical modelling, as implied from several recent dis-
uncertainty of predictions, without reference to the cussions (Beven, 2006; Pappenberger & Beven, 2006;
individual effects of the input, parameter and model Hamilton, 2007; Hall et al., 2007; Todini & Montovan,
structure components, other approaches attempt to 2007; Montanari, 2007; Andréassian et al., 2007;
handle them individually. These include multi-normal Todini, 2007; Sivakumar, 2008; Beven et al., 2008).
approximations (Kuczera & Mroczkowski, 1998),
simple uniform random sampling (Uhlenbrook et al.,
3.2 The multi-objective calibration paradigm
1999), Markov Chain Monte Carlo methods (Kuczera
& Parent, 1998; Thiemann et al., 2001; Vrugt et al., Despite the criticism of the equifinality concept,
2003b; Engeland et al., 2005), meta-Gaussian tech- hydrologists agree now that is impossible to formulate
niques (Montanari & Brath, 2004), sequential data a unique modelling structure and assign a unique
assimilation (Vrugt et al., 2005), multi-model averag- parameter set to it, thus identifying the globally optimal
ing methods (Ajami et al., 2007) and coupled schemes simulator of all processes of a river basin using a
(Blasone at al., 2008). For instance, the Shuffled Com- unique objective function. In fact, more than three dec-
plex Evolution Metropolis (SCEM-UA) algorithm by ades of research have demonstrated that it is impossible
Vrugt et al. (2003b) is a combined uncertainty assess- to assign an appropriate formal error structure for the
ment and parameter optimization procedure, based on model residuals and, on the basis of the latter, detect a
a modified version of the SCE method for global opti- particular statistical measure that is better suited for
mization. It is Bayesian in nature and operates by fitting model outputs to observations (e.g. Diskin &
merging the strengths of the Metropolis algorithm, Simon, 1977; Sorooshian et al., 1983; Yapo et al.,
controlled random search, competitive evolution and 1996). This is because the non-systematic interaction
complex shuffling, to continuously update the prior of uncertainties and errors within all modelling aspects
distribution and evolve the sampler to the posterior tar- precludes defining a statistically-proper fitting function
get distribution (Feyen et al., 2008). Moreover, the and, consequently, making a statistically-correct choice
simultaneous optimization and data assimilation for the model parameters (Gupta et al., 1998).
(SODA) method by Vrugt et al. (2005) aims for a joint In reality, any parameter estimation procedure
assessment of the uncertainty of model parameters and through data-fitting is inherently multi-objective. Let
observations (Montanari, 2007). e(q) = {e1(q), e2(q), …, eM(q)} represent the model
Regardless of their background, most of the above residuals, i.e. the departures of the observed
procedures do not enable incorporating the user’s responses from the computed ones, where q is the
experience in parameter estimation, which is the key vector of parameters. We can evidently define cali-
advantage of manual calibration. They are generally bration as the simultaneous minimization of the
too complicated for non-experts, whilst some of them absolute departures |ei(q)| with respect to q, i.e.:
(especially when employing random sampling) are
computationally inefficient, thus being impractical for minimize e( q )
models with complex parameterization. Additionally, (1)
they imply considerable subjectivity with respect to the = { e1 ( q ) , e2 ( q ) , . . ., eM ( q ) }, q ÎΘ
selection of prior probability distributions, likelihood
functions and cut-off thresholds (Stedinger et al., where Θ is the feasible parameter space, expressing
2008). Inappropriate configurations may result in over- the prior uncertainty of parameters. Given that hydro-
estimation of uncertainty, thus providing prediction logical models are, as discussed before, imperfect
ranges that are comparable to those computed through simulators of complex natural systems, the above vec-
statistical uncertainty measures (e.g. confidence limits) tor optimization problem is ill-posed. This prevents
the possibility of finding a utopian solution, namely a preserve the information contained in the observa-
specific parameter set that simultaneously minimizes tions, and m is the reduced dimension, with m << M.
all residuals. However, on the basis of the Pareto opti- The above problem is handled using either an aggre-
mality notion, we can locate a subset of the feasible gating or a multi-objective evolutionary approach to
parameter space Θ* ⊂ Θ, which contains the non- identify a single solution or a Pareto optimal set,
dominated vectors of parameters, while the rest of the respectively. While the first strategy is typically
space is captured by the dominated vectors, corre- employed in practice, the second one is definitely
sponding to non-acceptable trade-offs of the residuals. more integrated, since it allows for investigating pos-
The above formulation entails the separate mini- sible conflicts between the components of the vector
mization of all model residuals, whose number is objective function (equation 2).
impractically large; for instance, given a single From a mathematical point-of-view, all parameter
observable response to fit, the problem dimension is sets that are non-dominated with respect to criteria gi
equal to the calibration horizon. This makes the inter- correspond to equivalently optimal (in the Pareto
pretation of their trade-offs impossible, since the sense) solutions of equation (2). This reveals that equi-
Pareto front becomes too extended, if not tending to finality (mainly as treated within the GLUE frame-
cover the entire M-dimensional objective space work) and dominance are closely related (but not
(Coello Coello, 2005). Moreover, the magnitudes of identical), since both seek feasible model configura-
the individual residuals ei(q) are directly related tions that are then distinguished in two categories cor-
through the model structure, thus equation (1) is not responding to acceptable or not acceptable
properly defined in multi-objective terms (Gupta representations of the physical system. But while the
et al., 1998). So, instead of minimizing residuals GLUE method utilizes subjective criteria to differenti-
themselves, we can correctly state a multi-objective ate the behavioural simulators from the non-behav-
configuration of the calibration problem, assuming a ioural ones, the multi-objective paradigm is founded on
limited number of fitting criteria that account for rep- a stricter notion, i.e. the principle of dominance, for
resentative aspects of the model performance with evaluating alternative solutions. Moreover, in GLUE,
regard to the behaviour of the hydrological system. the behavioural solutions are not equivalent since they
Therefore, the problem is reduced to: are classified according to the likelihood function. As
shown in Fig. 2, a non-dominated solution obtained
maximize g ⎡⎣e (q )⎤⎦ through multi-objective analysis is not necessarily
(2) behavioural and vice versa. On the other hand, formal
= {g1 [e( q )], g2 [e( q )], …, gm [e( q )]}, q ÎΘ Bayesian inference techniques do not differentiate
behavioural from non-behavioural models—they only
where gi[e(q)] are scalar performance measures that give a tiny likelihood to poor simulators. Further dis-
ideally should be approximately uncorrelated and cussion on the comparison of the above approaches is
provided in Section 4.3.
Feasible objective space

f2
f2
Non-acceptable
trade-offs
e2 Sub-set of e2
behavioural
solutions
Sub-set of f1
Pareto front (non- e1 f1 promising e1
dominated solutions) trade-offs
Fig. 2 Graphical examples illustrating Pareto-optimal and behavioural solutions in the objective space, for two hypotheti-
cal problems of simultaneous minimization of two criteria [f1, f2] with smooth (left diagram) and steep (right diagram)
trade-offs. Vector e = [e1, e2] indicates limits of acceptability, i.e. cut-off thresholds for distinguishing behavioural and
non-behavioural solutions.
4 CRITICAL ISSUES IN MULTI-OBJECTIVE the model structure with as few parameters as pos-
CALIBRATION sible and accepts that simpler parameterizations are
preferred from more complex ones, provided that
Multi-objective calibration has received great atten-
both ensure similarly good fitting. Specifically, in
tion in the last decade, as indicated in Table 1, where
hydrological modelling, several investigations about
we quote representative case studies from the litera-
the practical use of this concept (e.g. Beven, 1989;
ture. For each one, we provide synoptic information
Jakeman & Hornberger, 1993; Ye et al., 1997;
about the application area, the modelling framework,
Uhlenbrook et al., 1999; Perrin et al., 2001) con-
the number of parameters and criteria to optimize,
cluded that parsimony is the guise for well-posed
and the calibration strategy. We distinguish between
models. Specifically, in the case of lumped concep-
pure Pareto-based approaches, where a set of non-
tual schemes, up to five or six parameters can be
dominated solutions is detected using a MOEA, and
identified from time series of external system varia-
aggregating ones, where a unique compromise
bles (e.g. rainfall, streamflow) through single-objective
parameter set is identified on the basis of multiple
calibration approaches (Wagener et al., 2001; see
criteria embedded in a scalar performance function.
also earlier discussions by Dawdy & O’Donnell,
We note that, while most of early studies focused on
1965, and Kirkby, 1975). Attempts to use additional
lumped rainfall–runoff models, there is a growing
parameters, in the absence of supplementary data to
number of recent studies on semi-distributed and dis-
support them, usually fail to notably improve the
tributed schemes, usually involving a small portion
model fitting and result in poorly identified parame-
of the total model parameters (Madsen, 2003; Ajami
ters (Gupta & Sorooshian, 1983; Hornberger et al.,
et al., 2004; Muleta & Nicklow, 2005; Vrugt et al.,
1985; Kuczera & Mroczkowski, 1998). In this man-
2005; Kunstmann et al., 2006). The spatial scale of
ner, model complexity, defined as the formulation of
applications varies from experimental basins of a few
non-parsimonious (over-parameterized) structures,
hectares (Seibert & McDonnell, 2002; Meixner et al.,
becomes a key origin of equifinality, thus increasing
2002; Tang et al., 2006) to very large basins of
uncertainty within the parameter estimation proce-
thousands of square kilometres (Schoups et al.,
dure. Additionally, the use of such structures reveals
2005a; Cheng et al., 2005a; Engeland et al., 2006;
a critical problem known as over-fitting, which is
Feyen et al., 2008). Most applications use two or
recognized by the surprisingly poor validation of a
three objectives, and only a few explore more crite-
model with significantly good fitting in calibration.
ria, ranging from statistical fitting functions to empir-
Yet, the preservation of parsimony is question-
ical and fuzzy metrics (Schoups et al., 2005a;
able in modern modelling tools with distributed or
Parajka et al., 2007; Efstratiadis et al., 2008; Moussa
semi-distributed structures and, thus, with a large
& Chahinian, 2009). Finally, only few items of the
number of parameters for representing the spatial
wide spectrum of second-generation multi-objective
heterogeneities of both basin characteristics and forc-
evolutionary tools have been tested in hydrological
ing data. Similar difficulties arise when hydrological
calibration applications (NSGA-II, SPEA-II,
models are coupled with water management schemes
e-NSGA). We have found only two studies which
to provide forecasts of inflows and abstractions at
compare their performance characteristics (Tang
multiple sites (Efstratiadis et al., 2008). Distributed
et al., 2006, 2007).
schemes are founded on small-scale physics, which,
Taking into account the rich experience of this
in theory, would allow for obtaining all parameter
last decade, we next discuss five key issues of multi-
values from field data, thus avoiding calibration
objective calibration, also attempting to propose some
effort. However, the idea that the natural heterogene-
guidelines for appropriate use of such approaches to
ity could be modelled without calibration based on
ensure faithful and reliable models.
field measurements of physically meaningful proper-
ties in a detailed spatial scale is fundamentally
flawed and unrealistic. For this reason, some model-
4.1 Preservation of the principle of parsimony in
lers employ an intermediate strategy, aiming to opti-
complex models
mize a small portion of parameters, while the rest of
The principle of parsimony is a key notion in model- them are approximated on the basis of known proper-
ling, where model parameters are estimated by fitting ties of the basin (e.g. Refsgaard, 1997; Muleta &
computed outputs to observed data. It aims to represent Nicklow, 2005). In contrast, semi-distributed models
Table 1 Characteristic applications of multiobjective calibration of hydrological models (pure Pareto approaches are
annotated with *).
Reference Study basin(s) Model Problem formulation Calibration method(s)
(parameters and objectives)
*Yapo et al. (1998) Leaf River, USA (1950 km2) SAC-SMA 13 parameters, 2 objectives MOCOM-UA
(RMSE, HMLE)
Seibert (2000) Lilla Tivsjön (12.8 km2) and HBV 10 parameters, 2 objectives Modified genetic
Tärnsjö (14.0 km2), (fuzzy measures algorithm
Sweden combining EF for runoff
and CD for groundwater
levels)
Madsen (2000) Tryggevaelde, Denmark MIKE 11/NAM 9 parameters, 4 objectives Weighted SCE-UA
(130 km2) (overall volume error,
RMSE, RMSE of peak
and low flows)
Yu & Tang (2000) Gao-Oing Creek, Taiwan HBV 9 parameters, 3 objectives SCE-UA
(3257 km2) (RMSE, MPE, fuzzy
MPE-based function for
11 flow stages)
*Liong et al. (2001) UBT, Singapore (6.11 km2) HydroWorks 8 parameters, 2 objectives VEGA, MOGA,
(overall volume error, NSGA, ACGA
peak discharge error) with NN
*Madsen & Khu (2002) Tryggevaelde, Denmark MIKE 11/NAM 9 parameters, 2 objectives Weighted SCE-UA,
(130 km2) (RMSE of high and low PROSCE
flows)
*Beldring (2002) Sæternbekken, Norway Physically-based 11 parameters, 3 objectives MOCOM-UA
(6.32 km2) rainfall–runoff (EF of runoff and two
model groundwater level series)
Seibert & McDonnell (2002) Maimai M8, New Zealand 3-box lumped 16 parameters, 3 fuzzy Modified genetic
(3.8 ha) conceptual functions (one based on algorithm
model runoff and two
groundwater level series,
and two rules based on
“soft” data)
Cheng et al. (2002) Shuangpai, China Xinanjiang 16 parameters, 3 objectives Multiple objective
(10 594 km2) (peak value, peak time, GA
total runoff volume)
*Meixner et al. (2002) Emerald Lake, Sequoia Alpine 15 parameters, 4 objectives MOCOM-UA,
National Park, USA Hydrochemical (sub-sets of 21 chemical combined with
(120 ha) Model (AHM) and hydrological criteria) sensitivity analysis
*Madsen (2003) Karup, Denmark (440 km2) MIKE-SHE 12 parameters, 2 objectives Weighted SCE-UA
(RMSE of runoff, avg.
RMSE of 17 groundwater
level series)
*Vrugt et al. (2003a) Leaf River, USA (1950 km2) SAC-SMA 13 parameters, 2 objectives MOCOM-UA,
(RMSE for driven and MOSCEM-UA
non-driven parts of
hydrograph)
Ajami et al. (2004) Illinois River, USA Multiple structures 13 parameters, 2 objectives Multi-step calibration
(1645 km2) of SAC-SMA (RMSE and Log-RMSE with SCE-UA
for fitting on high and
low flows)
*Schoups et al. (2005b) San Joaquin Valley, USA MOD-HMS 10 parameters, 3 objectives SCEM-UA,
(1400 km2) (RMSE of water table, MOSCEM-UA
annual pumping and
subsurface drainage)
Muleta & Nicklow (2005) Big Creek, USA (133 km2) SWAT 16 parameters, 2 objectives Sensitivity analysis,
(RMSE of runoff and GA, GLUE
sediment yield)
(Continued)
Table 1 (Continued).
*Khu & Madsen (2005) Tryggevaelde, Denmark MIKE 11/NAM 9 parameters, 4 objectives NSGA-II with Pareto
(130 km2) (overall volume error, preference ordering
RMSE, RMSE of peak
and low flows)
*Schoups et al. (2005a) Yaqui Valley, Mexico Integrated surface 10 parameters, 4 objectives MOSCEM-UA
(6800 km2) water- (RMSE of water table,
groundwater aquifer head, drainage
model volume and canal seepage
volume)
Cheng et al. (2005a) Shuangpai Reservoir, China Xinanjiang 16 parameters, 3 objectives Serial and parallel
(10 594 km2) (peak value, peak time, GAs
total runoff volume)
*Engeland et al. (2006) Saone, France (11 700 km2) Ecomag 10 parameters, 7 objectives MOCOM-UA
for calibration, 15
independent objectives
for validation (EF of 22
runoff series)
*Tang et al. (2006) Leaf River, USA (1950 km2) SAC-SMA 13 parameters, 2 objectives NSGA-II, SPEA-II,
(RMSE & RMSE with MOSCEM-UA
Box-Cox transformation)
*Tang et al. (2006) Shale Hills, USA (19.8 ha) Integrated surface- 13 parameters, 2 objectives NSGA-II, SPEA-II,
subsurface model (RMSE, RMSE of peak MOSCEM-UA
and low flows)
Kunstmann et al. (2006) Ammer River, Germany WaSiM (distributed 37 parameters, 8 objectives PEST (two-step
(710 km2); Alpine model) (EF at 8 discharge approach)
catchment gauges, using
transformed flows)
Rouhani et al. (2007) Grote Nete, Belgium SWAT 10 parameters, 5 objectives Manual calibration
(383 km2) (bias, RMSE for total and
slow flow, quick flow
maxima and slow flow
minima)
Moussa et al. (2007) Gardon d’Anduze, France ModSpa 5 parameters, 7 objectives Single and multi-site
(543 km2) (EF, bias and CC of 7 manual calibration
runoff series)
*De Vos & Rientjes (2007) Geer River, Belgium HBV 10 parameters, 3 objectives NSGA-II
(494 km2) (RMSE, log-RMSE and
MSDE)
*Bekele & Nicklow (2007) Big Creek, USA (133 km2) SWAT (two 16 parameters, 2 objectives NSGA-II
calibration per scenario (RMSE and
scenarios) log-RMSE of runoff;
RMSE and log-RMSE of
sediment yield; RMSE of
runoff and sediment
yield)
*Tang et al. (2007) Leaf River, USA (1950 km2) SAC-SMA 13 parameters, 2 objectives Serial and parallel
(RMSE and RMSE with implementations of
Box-Cox transformation) e-NSGA-II
*Confesor & Whittaker Calapooia, USA (963 km2) SWAT 139 parameters, 2 objectives NSGA-II (cluster of
(2007) (RMSE and MAE of 24 parallel
daily flows) computers)
*Parajka et al. (2007) 320 Austrian catchments Modified HBV 14 parameters, 2 objectives Weighted SCE-UA,
(with semi- (weighted function of EF MOSCEM-UA
distributed and bias of runoff, time
structure) ratio with poor snow
cover simulation)
(Continued)
Table 1 (Continued).
Kim et al. (2007) North River, Virginia HSPF 11 parameters, 6 objectives Manual and automatic
(973 km2) (MSE of daily flows, calibration, using
50% of lowest flows the PEST software
exceed., 10% of highest
flows exceed., storm
peaks, seasonal volume,
storm volume)
*Fenicia et al. (2007a) Hesperange, Luxemburg FLEX (simple & 8 to 11 parameters, 3 MOSCEM-UA, SCA
(288 km2) complex objectives (RMSE, log- (stepped
structure) RMSE and CC of runoff) calibration)
*Fenicia et al. (2007b) Hesperange, Luxemburg HBV-96 9 parameters, 2 objectives MOSCEM-UA
(288 km2) accounting for the low
and high portions of the
hydrographs
Efstratiadis et al. (2008) Boeoticos Kephissos, Greece HYDROGEIOS 99 parameters, 40 objectives Hybrid, using the
(1956 km2) (surface water, (EF and bias of 7 runoff evolutionary
groundwater and series, penalties to control annealing-simplex
water flow interruption events method
management and unrealistic trends of
model) groundwater level series)
*Khu et al. (2008) Karup, Denmark (440 km2) MIKE-SHE 11 parameters, 5 to 9 Preference ordering
objectives (RMSE of genetic algorithm
runoff, avg. RMSE and (POGA)
st.dev. of residuals for
representative
piezometric series, after
grouping of multisite
data)
Feyen et al. (2008) Morava, Austria, Czech Rep. LISFLOOD (three 9 parameters (lumped or SCEM-UA
and Slovak Rep. calibration semi-distributed over 7
(∼10 000 km2) scenarios) sub-basins), 3 objectives
per hydrograph (bias, EF
and CD) with
transformed flows
*Moussa & Chahinian Gardon d’ Anduze, France Lumped, two- 7 parameters, 1 to 3 Multi-step
(2009) (543 km2) reservoir-layer objectives resulting from aggregation
model, event- 6 fitting criteria (global method for 29
based and relative bias, global flood events
and relative RMSE,
global and relative peak
flow error)
RMSE: root mean square error; log-RMSE: RMSE of logarithmically transformed data; HMLE: heteroscedastic maximum likelihood
error; EF: Nash-Sutcliffe efficiency; CD: coefficient of determination; CC: coefficient of correlation; MPE: mean absolute percentage
error; MAE: mean absolute error; MSDE: mean square derivative error.
do not hide the fact that they are conceptual in nature. in calibration, by introducing additional outputs for
Yet, they involve calibration of many more free vari- model fitting or by improving the knowledge already
ables, if compared to analogous schemes with available, e.g. using different data periods to identify
lumped or semi-lumped parameterization (Ajami different parameters (Wagener et al., 2001). As a
et al., 2004). first approach, and extending the empirical rule
In the case of complex models with many expressed for lumped models, we should retain a
parameters, multi-objective calibration provides a ratio of about 1:5 to 1:6 between the number of crite-
favourable framework for preserving parsimony ria and the number of parameters to optimize, to pro-
and thus reducing uncertainty. This presupposes vide a parsimonious representation of the multi-
the increase of independent information contained objective calibration problem. Typically, significant
effort is required to formulate uncorrelated criteria fitting its parameters to the corresponding data. The
that really add new information, based on the avail- advantages of “conditioning” the model parameters on
able measures, as further analysed in the following multiple responses are extensively discussed by Gupta
sub-section. et al. (1998). In addition, Kuczera & Mroczkowski
A deeper inspection of the above framework (1998) use the term joint calibration to describe a suit-
reveals the need for fundamental changes to the clas- able framework for compromising between model com-
sical rainfall–runoff modelling strategy, assumed so plexity and the principle of parsimony. In the absence of
far as a staged procedure where conceptualization major structural errors, this approach enhances the cali-
(i.e. the representation of system dynamics through bration procedure with additional information about the
parametric equations) precedes calibration (Beven, physical system, thereby leading to a better identifica-
2001, p. 4). This approach has little flexibility, since tion of the model parameters (Boyle et al., 2000).
the model structure and, subsequently, the number of Following the terminology of Madsen (2003),
parameters, is a priori specified. Yet, for poorly the multi-objective fitting function may be formu-
measured hydrosystems, it is impossible to have lated on the basis of the following three types of
sufficient information to formulate the number of cri- information:
teria that is necessary to justify the detail of the
– multi-variable data: different observable fluxes
adopted delineation. An efficient way to avoid this is
that are reproduced by conjunctive simulation
to disconnect the schematization, involving the spa-
schemes, including flows, piezometric levels,
tial detail of process description (which is imposed
sediment load, geochemical tracers, distributed
by the specific scope of study), from parameteriza-
soil moisture, etc.;
tion, which assigns the model free variables to the
– multi-site data: historical records obtained from a
characteristics of the physical system (Efstratiadis
number of gauges within the river basin, which
et al., 2008). However, in most known distributed
measure the same variable and are reproduced by
tools schematization dictates parameterization, since
semi- or fully-distributed schemes;
parameters refer to contiguous spatial elements,
– multi-response models: independent criteria
usually grid cells, whose number is typically huge.
accounting for various aspects of a single process
Not only does this contrast the principle of parsi-
(typically discharge), which is reproduced even
mony but also makes optimization inefficient, due to
by lumped conceptual schemes.
the curse of dimensionality and the large time effort
of simulation. In groundwater modelling, the prob- In particular, the last type of information origi-
lem is typically addressed through regularization nates from the same historical sample, which is uti-
techniques, i.e. by using spatial zonation patterns lized from different points of view. This approach
through the aquifer or by constraining parameters to aims to ensure a satisfactory agreement of the spe-
preferred values or relationships. While such cific components making up the observed discharge
approaches are widely used to obtain a unique solu- series, and not an average good match across all flow
tion to the inverse problem, an oversimplified param- ranges (Yapo et al., 1998; Madsen, 2000; Moussa &
eterization dramatically reduces the model accuracy Chahinian, 2009). It is in full accordance with a man-
at local scales (Moore & Doherty, 2006; see also dis- ual calibration strategy, where the expert hydrologist
cussion by Hunt et al., 2007). follows a trial-and-error approach to reproduce all
features of a hydrograph, regarding both flow quan-
tity and timing. Moreover, focusing on different
4.2 Model fitting on multiple responses
aspects ensures more realistic and robust parameter
Fully- and semi-distributed models estimate the values, given that different parameters activate dif-
basin fluxes at multiple sites (grid and sub-basin ferent hydrological mechanisms, which are finally
scale, respectively) while conjunctive simulation reflected on the shape of the hydrograph (Rouhani
schemes, i.e. surface–groundwater models, hydro- et al., 2007).
chemical models and sediment transport models, pro- A multi-objective fitting strategy should not be
vide estimations for multiple processes. When restricted to systematic measurements for all varia-
systematic measurements exist for those variables, the bles involved in calibration. Even sparse observa-
role of multi-objective calibration becomes evident, in tions, or rough estimations about the average
order to maximize the model predictive capacity by quantities or their long-term fluctuation, are useful to
enhance the information contained in calibration and by Gallart et al. (2007), reports that the uncertainty
reduce uncertainties. This issue becomes critical of the predicted discharges has been significantly
when the number of the observed variables is insuffi- restricted.
cient to support the number of parameters. In that The above reveals a common misconception
case, the hydrologist should take advantage of his with regard to multi-objective calibration, which is
experience to “invent” empirical criteria so as to be that as more information about the system becomes
compatible with the principle of parsimony in param- available, the uncertainty of predictions is definitely
eterization. Seibert & McDonnell (2002) introduced reduced. Kuczera & Mroczkowski (1998) highlight
the term “soft data” to characterize the qualitative this danger, indicating that the improvement of the
rather than the quantitative knowledge about the parameter identifiability mainly depends on how the
behaviour of a basin, in contradistinction to “hard model structure interacts with each response, and less
data”, namely measurements derived from well- on the amount of data itself. In addition, a consistent
recorded variables. This approach represents a new formulation of the multi-objective calibration prob-
dimension to calibration that favours the dialogue lem is far form being a straightforward task. For
between experimentalists and modellers, ensures instance, the criteria are not expected to be uncorre-
reasonableness and consistency of internal model lated (since the basin fluxes are mutually correlated
structures and simulations, and also helps to specify with precipitation and evapotranspiration) and are
realistic parameter ranges. Moreover, it helps in also related with commensurability and uncertainty
providing reliable simulations for model responses issues. A proper evaluation of the information
and internal variables that are not controlled by meas- content of additional observations, as well as the
urements, e.g. evapotranspiration, moisture storage, development of a generalized approach that may
groundwater storage, underground losses, etc. allow us to benefit from different types of informa-
While hard data are typically represented by tion (including multi-site observations and soft data),
statistical fitting functions (e.g. RMSE, efficiency), remains an open issue in hydrological research
the incorporation of soft data within calibration is (Beven, 2006; Montanari, 2007; Khu et al., 2008).
implemented through empirical or fuzzy metrics,
which are introduced as independent components of
4.3 Recognition of model errors
the multi-objective function (e.g. Yu & Yang, 2000;
and uncertainties
Seibert & McDonnell, 2002; Cheng et al., 2002;
Rozos et al., 2004; Parajka et al., 2007; Efstratiadis The limitations of a model can be empirically
et al., 2008). This certainly increases the effort of addressed within a multi-objective calibration frame-
calibration and provides less attractive results with work, by investigating the trade-offs between the dif-
regard to an approach that is merely based on hard ferent objectives of the Pareto optimal solutions
data. Nevertheless, this is the cost paid to obtain a (Gupta et al., 1998). Although, from a statistical
better overall model performance and ensure consist- point-of-view, it is difficult to isolate the different
ency within all of its aspects (Seibert & McDonnell, categories of errors from parameter uncertainty (Ros-
2002). bjerg & Madsen, 2005), an irregular shape of the
The effects on model predictive capacity of con- Pareto front is a usual evidence of ill-posed models.
ditioning its responses on multiple objectives have For instance, significant trade-offs in fitting two or
been also examined within uncertainty assessment more objectives may indicate that the model is
approaches, employing the GLUE technique (Lamb wrongly parameterized (Schoups et al., 2005a,b). In
et al., 1998; Blazkova et al., 2002; Freer et al., 2004; addition, an asymmetrically extended spread of the
Mo & Beven, 2004; Blazkova & Beven, 2004; Zhang Pareto solutions along one particular axis indicates
et al., 2006; Choi & Beven, 2007; Gallart et al., considerably high uncertainty in reproducing the
2007). In some of the above studies, this involved the processes that are controlled by the corresponding
evaluation of the performance of TOPMODEL against criterion. Similarly, the generation of very steep
discharge, water table and saturated area observations, fronts, almost resembling right angles (Fig. 2, right)
through appropriate likelihood measures. All con- denotes the sensitivity of parameters to the corre-
cluded that the use of internal catchment information sponding criteria, since a small perturbation of the
definitely helped to narrow the posterior distributions parameter values, in the direction of improving one cri-
for the related parameters. Yet, only the last paper, terion, leads to significant deterioration of the others
(Efstratiadis & Koutsoyiannis, 2008). Valuable a hyperrectangle in the m-dimensional objective

information about the possible model errors is also space (containing both dominated and non-dominated
provided by deriving the ranges of non-dominated sets), whereas Pareto optimal solutions are searched
parameter sets, as well as the ranges of the simulated across hypersurfaces of dimension m – 1, i.e. in a much
responses (“envelopes”) against the criteria. For restricted area. Their cross-section determines a sub-set
instance, when these envelopes fail to enclose all that encloses solutions that are simultaneously non-
observed values of a hydrograph, an expert hydrolo- dominated and behavioural, while in the case of very
gist can easily recognize whether this failure is due to steep Pareto fronts one should further restrict its
an inappropriate model structure (e.g. by examining limits to seek for promising trade-offs (Fig. 2).
which specific parts of the hydrograph systematically Finally, when new objectives are included, while in a
remain out of the Pareto-optimal range) or inaccurate Bayesian inference approach the parameter uncer-
data. In contrast, a single-objective calibration would tainty possibly decreases (or remains unchanged), the
not allow recognition of whether the departures of Pareto set definitely extends, thus resulting in
the modelled outputs from the observations are due increased uncertainty. This is a known characteristic
to structural (or data) errors or a statistically incon- of multi-objective theory, where criteria are consid-
sistent fitting function. ered as degrees of freedom and not as constraints.
In some cases, the increased information pro- Indeed, on the basis of Pareto optimality, if one
vided after employing a multi-objective framework solution outperforms another one against even a sin-
may even lead to rejection of an inappropriate model, gle criterion, then the two alternatives are indifferent.
which would appear as proper against a single crite- Therefore, by adding criteria, the existing non-domi-
rion. An interesting example is given by Choi & nated set not only remains non-dominated, but
Beven (2007), who attempted to fit TOPMODEL in spreads across the new dimensions. For this reason,
an experimental catchment in Korea, taking advant- and given that even state-of-the-art multi-objective
age of both annual and seasonal (30 day) calibration optimization algorithms incur serious performance
data. While the model showed good performance (by deterioration in high-dimensional objective spaces, it
means of efficiency) at the annual level, no model is not practical to employ Pareto-based optimization
implementations were found that were behavioural on the basis of more than three to four criteria. Other-
over all multi-period clusters and all performance wise, it is necessary to implement some form of
measures (mainly in dry periods). The authors aggregation of objectives, e.g. through clustering
claimed that the model rejection strategy of their techniques (Khu et al., 2008), or even review the
GLUE approach served to focus attention on possible concept of dominance as the only evaluation prin-
model deficiencies, thus making it necessary to add ciple, by employing some kind of filtering among
more parameters for the description of the time- indifferent solutions (Efstratiadis & Koutsoyiannis,
varying recession and evapotranspiration processes. 2008).
Since the Pareto set can be used to generate
envelopes that contain all acceptable (according to
4.4 Handling non-commeasurable
the dominance concept) model outputs, multi-objective
fitting criteria
calibration has some links with Bayesian inference
methods for uncertainty assessment. Yet, there are Several studies seek a single parameter set that
also key differences, as Engeland et al. (2006) ensures satisfactory performance against all conflict-
explain, especially with respect to the GLUE ing criteria, namely an intermediate solution from the
method. First, Bayesian methods evaluate the uncer- Pareto front. However, approximating this front
tainty around a single performance measure, namely through a MOEA and then manually picking up a
the likelihood function, while a multi-objective con- suitable solution on the basis of external-empirical
text requires at least two criteria to make sense. The criteria is time-consuming, not well-understood and
GLUE framework also allows combining multiple thus far away from the usual practice. On the other
objectives, provided that they can be expressed in hand, the traditional manipulation through an equi-
terms of likelihoods—yet, the evaluation of these valent single-objective optimization approach (e.g.
objectives is not based on the principle of domi- weighting method) involves many more difficulties
nance but on arbitrary acceptability thresholds. than when optimizing a particular criterion. Some of
Thus, the behavioural solutions are searched inside the practical drawbacks of the so-called aggregating
approaches have been already discussed in Section thus formulating non-convex response surfaces of
2.2. Specifically, within a calibration problem invol- irregular geometry. In that case, even the most
ving many criteria, it is necessary to broadly specify sophisticated global optimization methods are pos-
the desirable characteristics of the best-compromise sible to trap, thus failing to locate a suitable com-
solution, through suitable configuration of the scalar promise that ensures satisfactory performance
objective function. But in some cases, it is even hard against all criteria. This negates all the benefits dis-
to recognize whether two criteria are conflicting or cussed so far, regarding multi-criteria calibration. In
not, since their behaviours differentiate across the this respect, hybrid strategies taking advantage of the
feasible parameter space. Further problems arise strengths of both manual and automatic calibration,
when the criteria are non-commeasurable, which can be most suitable approaches for such problems
requires proper scaling to avoid over-emphasis of (Boyle et al., 2000). These allow guiding “by hand”
specific components of the objective function, in the search towards acceptable compromises, since an
contrast to others (Madsen, 2000). Obviously, an expert hydrologist easily recognizes the conflicts of
incautious formulation of the problem may result in criteria. In contrast, a black-box algorithmic proce-
asymmetrically good fitting for some criteria in con- dure, which evolves on the basis of an aggregating
trast to the rest of them (solutions lying in the scalar function, has no insight on the trade-offs of
extremes of the Pareto front), unless limits of accept- criteria and thus may converge to solutions with
ability are imposed, as shown in Fig. 2. It is interest- unsatisfactory performance. Characteristic studies
ing to notice that, in some cases, it is desirable to focus involving hybrid manipulations of the multi-criteria
on specific criteria in order to obtain more accurate pre- problem (Ajami et al., 2004; Kunstmann et al., 2006;
dictions at local rather than global scales. For instance, Rouhani et al., 2007; Moussa et al., 2007; Efstratiadis
Pappenberger et al. (2007) used a vulnerability- et al., 2008; Moussa & Chahinian, 2009) are
weighted approach to ensure better calibration of a included in Table 1.
flood inundation model to locations that are of particu-
lar interest to flood planners and risk assessors.
4.5 Identifying a best-compromise
Scaling problems occur when dealing with vari-
parameter set
ables measured in different units (e.g. runoff and
groundwater level), when combining dimensional While multi-objective calibration provides new per-
measures with non-dimensional ones, and when spectives to the parameter estimation problem, the
combining statistical and empirical or fuzzy mea- detection of a unique parameter set, to be utilized for
sures. The different criteria require assigning proper hydrological planning, management and forecasting,
transformations, most typically weighting coeffi- remains a common practice. This is confirmed by the
cients. The latter may be either empirically deter- recent calibration studies (Table 1), where many of
mined (Cheng et al., 2005; Rouhani et al., 2007; them attempt to identify the most “prominent” solu-
Parajka et al., 2007), or specified analytically at the tion against the conflicting criteria, usually following
beginning of the evolution procedure, according to a semi-automatic strategy, where the hydrological
the properties of the initial population (Madsen, experience plays a key role. In contrast to the black-
2000, 2003; Moussa & Chahinian, 2009), or manu- box approaches of the 1990s, the current trend
ally re-evaluated during optimization, taking into favours the incorporation of the user’s judgment in
account the progress achieved so far and the conflicts order to retrieve a good compromise among the
to compromise (Rozos et al., 2004; Kim et al., 2007; multiple non-dominated solutions. This major issue
Efstratiadis et al., 2008). Fuzzy multi-objective func- was comprehensively addressed by Boyle et al.
tions are also used that ensure flexibility and allow (2000), who proposed a hybrid calibration procedure
for combining criteria that are not directly analogous comprising two steps. In the first step, an automatic
(Yu & Tang, 2000; Seibert & McDonnell, 2002; search of the feasible parameter space is imple-
Cheng et al., 2002, 2006). All of the above mented, to define a representative sample of Pareto
approaches are in accordance with the hybrid optimal parameter steps, on the basis of user-selected
calibration paradigm for selecting a single criteria that measure different aspects of the close-
“balanced” solution. ness of the model outputs and observations. In the
In general, the aggregation of criteria leads to sig- second step, the solutions having unacceptable trade-
nificantly high complexity of the objective function, offs are rejected, and additional criteria (both objective
and subjective) are introduced to narrow the search against different criteria and different periods are
space, also accounting for the overall statistical char- strong evidence of the robustness of the best-com-
acteristics of the model responses (e.g. long-term promise solution (Efstratiadis & Koutsoyiannis,
biases and overall residual variance). 2009; cf. Choi & Beven, 2007).
Madsen et al. (2002) investigated three strate- Although manual strategies take full advantage
gies that utilize multiple objectives and allow user of the hydrological experience, they are very time-
intervention on different levels and different stages consuming and too difficult to computerize. Thus,
in the calibration process, specifically: some recent approaches have focused on developing
effective and “friendly” filtering tools and embed-
– a generic search routine, where the user specifies
ding them within multi-objective search. For
the priorities to be given to certain objectives that
instance, Schoups et al. (2005a,b) used various pro-
are aggregated into one measure which is then
cedures for identifying the best-compromise solu-
optimized automatically;
tion, including the minimization of the Euclidean
– a method using different automatic search tech-
distance in the normalized objective function space.
niques (cluster analysis, simulated annealing and
They claimed that the optimal choice depends on the
multi-criteria optimization) in combination with
individual interests as defined by the user, thus
different calibration objectives, which requires
emphasizing the decision-making process rather than
user intervention at different stages in the calibra-
the hydrological problem. Khu & Madsen (2005)
tion process;
proposed an automatic routine, based on multi-objec-
– a knowledge-based expert system, reflecting the
tive genetic algorithms and Pareto preference order-
course of a trial-and-error effort of experienced
ing, which enables one to sift through the numerous
hydrologists, where user intervention is required
Pareto optimal solutions and retain a short-list of pre-
for subjective evaluation of different calibration
ferred ones for further investigation; this list contains
criteria.
non-dominated solutions that remain non-dominated
These different methods focused on different aspects of in different subspace combinations of the objective
the examined model responses, but none of them functions space. Finally, Fenicia et al. (2007a) com-
proved superior with respect to all criteria considered. bined vector optimization with a stepped calibration
Several recent studies focus on the exploitation strategy to explore the deficiencies of the model
of the valuable information provided by vector opti- structure and determine a solution that is consistent
mization approaches and the development of guide- with the data available.
lines for selecting the best-suited parameter set
among multiple non-dominated ones. Rozos et al.
5 SYNOPSIS AND DISCUSSION
(2004) used several empirical criteria for evaluating
Pareto optimal solutions, including the overall model The progress in integrated representation of hydro-
performance against all the measured responses as logical processes through detailed modelling tools
well as the likelihood of the unmeasured ones, the has highlighted the weaknesses of automatic, single-
consistency of the optimized parameters against their objective calibration approaches. At the same time,
broad physical interpretation, and the model predictive as models become more complex, multi-objective
capacity, i.e. the performance of each non-dominated strategies for parameter estimation have exhibited
solution in validation. With regard to the last issue, it several strong points; they: (a) ensure parsimony,
was not surprising that the majority of the solutions namely consistency between the number of criteria
obtained within calibration were clearly rejected, since against parameters to optimize, thus improving their
their performance was significantly deteriorated when identifiability; (b) fit the distributed responses of
moved to another time period (i.e. validation). This models on multiple measurements (“hard” data), also
reveals a serious drawback of multi-objective cali- enhancing the information contained in calibration
bration, which seems to be rather inefficient at on the basis of “soft” data, derived through expert
providing solutions that remain non-dominated (or knowledge; (c) recognize the uncertainties and struc-
approximately non-dominated) across different con- tural errors related to the model configuration and the
trol periods, since the Pareto set obtained on the basis parameter estimation procedure; (d) effectively handle
of a specific data set is obviously non-unique. On the criteria of different scales or criteria having contradic-
other hand, the existence of satisfactory trade-offs tory performance; and (e) utilize the experience
obtained after investigating the trade-offs of criteria A multi-objective approach does not necessarily
for identifying a best-compromise solution, which guarantee the detection of calibrations that are
should be consistent with the existing knowledge acceptable from a hydrological perspective. In fact,
(i.e. experience and data). Such strategies are advan- because of the past emphasis on finding the “best”
tageous even for calibrating simple models with a model (in either a global- or Pareto-optimal sense,
few parameters, because by taking into account vari- both based on fitting metrics requiring systematic
ous objectives (both quantitative and qualitative), measurements), there has been little consideration of
they ensure consistency against multiple aspects of whether this optimal model is actually a consistent
the system under study. simulator according to an expert hydrologist (Choi &
In this last decade, significant progress was Beven, 2007). Thus, the attention is now given to soft
made with regard to different components of the data, usually expressed through empirical criteria
multi-objective calibration problem, including: (a) that also reflect the expert knowledge on the system
the algorithmic manipulation; (b) the formulation of under study. This allows for controlling different
objectives; (c) the interpretation of non-dominated modelling aspects from a macroscopic point-of-view,
solutions and the guidance to a best-compromise e.g. to ensure realistic fluctuations of internal model
choice, and (d) the link with uncertainty assessment variables (Efstratiadis et al., 2008). It also offers a
approaches. Still, there are many open issues that means to partially handle the huge uncertainty result-
have been recognized after the experience gained by ing from the complexity of model parameterizations
employing the multi-objective framework in a wide in contrast to data scarcity, which is a global engin-
spectrum of applications. eering problem that is getting increasingly severe.
Specifically, recent advances in computer sci- Yet, we emphatically note that soft data are auxiliary
ence provide a number of robust multi-objective information and cannot substitute measurements;
optimization tools, typically employed as adaptations moreover, a “bulimic” use of empirical criteria that
of genetic algorithms. Yet their dissemination in real- are not supported by some kind of documentation
world hydrological applications is relatively poor may lead to over-constraining the feasible parameter
and thus there is much research to be done on com- space and thus underestimating uncertainty. Actual
parative tests in challenging calibration problems. research should provide more guidance on the effect-
The definition of appropriate procedures for evaluat- ive combination of statistical and expert-based evalu-
ing MOEAs remains a challenging task in optimiza- ation procedures.
tion science (Zitzler et al., 2003; Coello Coello, The assessment of the richness of information
2005). The calibration problems of hydrological derived by Pareto-based calibration approaches also
models certainly present difficulties not usually offers additional research perspectives. For instance,
faced in other technological areas. First, the compu- the interpretation of the irregularities of the trade-off
tational time needed for a single simulation run in curves has been little investigated. There are also
complex models, makes it impossible to approach the many practical issues that remain open, such as the
Pareto front with reasonable effort. Second, there is development of a hybrid calibration framework
too little experience on multi-dimensional objective supporting interactive computerized facilities, for fil-
spaces, while a calibration problem may involve a tering through numerous Pareto-optimal solutions to
large number of fitting criteria, either statistical or detect the most promising ones. This last option may
empirical. In reality, not all of them are by nature be related to the non-uniqueness property of the
conflicting and the trade-offs appearing are mainly Pareto set—a critical point to which no attention has
due to ill-posed structures and deficient data. been given so far. For instance, a cross-validation on
However, as more objectives are included in the different data subsets may help to significantly
calibration, the set of Pareto optimal solutions tends reduce the number of solutions ensuring acceptable
to be impractically extended; thus, it is necessary to trade-offs through different control periods (Efstratiadis
provide guidelines for determining a limited number & Koutsoyiannis, 2009). Yet, since this is often not
of criteria that are best suited for Pareto analysis feasible, it is essential to provide a framework to
(Meixner at al., 2002). For example, Khu et al. effectively combine (and explain) the results
(2008) proposed a framework for classifying multi- obtained from multiple calibration periods, in order
site measurements into groups according to temporal to improve the model predictions (Beven et al.,
dynamics. 2008).
Many argue that the real challenge in hydrology Bekele, E. G., & Nicklow, J. W. (2007) Multi-objective automatic
calibration of SWAT using NSGA-II. J. Hydrol. 341(3-4),
is the development of a generalized uncertainty 165–176.
assessment framework that will allow hydrological Beldring, S. (2002) Multi-criteria validation of a precipitation–
models to profit from different types of information runoff model. J. Hydrol. 257, 189–211.
(e.g. Hamilton, 2007; Montanari, 2007). Indeed, Beven, K. J. (1989) Changing ideas in hydrology—the case of
physically-based models. J. Hydrol. 105, 157–172.
state-of-the-art research is actually focused on the inte- Beven, K. J. & Binley, A. M. (1992) The future of distributed mod-
grated handling of parameter estimation and uncertainty els: model calibration and uncertainty prediction. Hydrol.
assessment, using multiple objectives within Bayesian Processes 6(3), 279–298.
Beven, K. J. (1993) Prophecy, reality and uncertainty in distributed
inference techniques (Vrugt et al., 2003b, 2005). Until hydrological modeling. Adv. Water Resour. 16, 41–51.
now, the experience has been restricted to elementary Beven, K. J. (2001) Rainfall–Runoff Modelling: The Primer. Wiley,
models and it is difficult to predict their success in more Chichester, UK.
Beven, K. J. (2006) A manifesto for the equifinality thesis.
demanding applications as well as their dissemination in J. Hydrol. 320(1-2), 18–36.
the everyday engineering practice. Yet, the major prob- Beven, K. J., Smith, P. J. & Freer, J. (2008) So just why would a
lem is not only technical but also philosophical; a gener- modeller choose to be incoherent? J. Hydrol. 354, 15–32.
ally agreed definition of uncertainty is missing, as is a Blasone, R. S., Vrugt, J. A., Madsen, H., Rosbjerg, D., Robinson, B.
A. & Zyvoloski, G. A. (2008) Generalized likelihood uncer-
generally-accepted assessment of whether the existing tainty estimation (GLUE) using adaptive Markov chain Monte
approaches over- or underestimate the uncertainty of Carlo sampling. Adv. Water Resour. 31, 630–648.
predictions (Beven, 2006; Andréassian et al., 2007; Hall Blazkova, S. & Beven, K J. (2004) Flood frequency estimation by
continuous simulation of subcatchment rainfalls and
et al., 2007; Todini & Montovan, 2007; Beven et al., discharges with the aim of improving dam safety assessment in
2008). In this obscure environment, it is difficult to pre- a large basin in the Czech Republic. J. Hydrol. 292, 153–172.
dict the success of a unified approach to model calibra- Blazkova, S., Beven, K. J. & Kulasova, A. (2002) On constraining
TOPMODEL hydrograph simulations using partial saturated
tion and uncertainty assessment following the multi- area information. Hydrol. Processes 16(2), 441–458.
criteria paradigm, which requires subjective decisions Boyle, D. P., Gupta, H. V. & Sorooshian, S. (2000) Toward
and is based on qualitative considerations (i.e. soft data). improved calibration of hydrologic models: combining the
strengths of manual and automatic methods. Water Resour.
Res. 36(12), 3663–3674.
Acknowledgements The preliminary research for Cheng, C.-T., Ou, C. P. & Chau, K. W. (2002) Combining a fuzzy opti-
this paper was done within the PhD dissertation of mal model with a genetic algorithm to solve multi-objective rain-
the first author entitled “Non-linear methods in multi- fall–runoff model calibration. J. Hydrol. 268, 72–86.
Cheng, C.-T., Wu, X. Y. & Chau, K. W. (2005) Multiple criteria
objective water resource optimization problems, with rainfall–runoff model calibration using a parallel genetic
emphasis on the calibration of hydrological models” algorithm in a cluster of computers. Hydrol. Sci. J. 50(6),
(http://www.itia.ntua.gr/en/docinfo/838/) and sup- 1069–1087.
Cheng, C.-T., Zhao, M.-Y., Chau, K. W. & Wu, X.-Y. (2006) Using
ported by the scholarship project “Heracleitos”. The genetic algorithm and TOPSIS for Xinanjiang model calibra-
project was co-funded by the European Social Fund tion with a single procedure. J. Hydrol. 316(1-4), 129–140.
(75%) and National Resources (25%). We are grate- Choi, H. T. & Beven, K. (2007) Multi-period and multi-criteria
ful to Hoshin Gupta and Keith Beven for their useful model conditioning to reduce prediction uncertainty in an
application of TOPMODEL within the GLUE framework. J.
and constructive comments, critiques and sugges- Hydrol. 332(3-4), 316–336.
tions, which helped us to substantially improve the Cieniawski, S. E., Eheart, J. W. & Ranjithan, S. (1995) Using genetic
paper. algorithms to solve a multiobjective groundwater monitoring
problem. Water Resour. Res. 31(2), 399–409.
Coello Coello, C. A. (2005) Recent trends in evolutionary multiobjec-
tive optimization. In: Evolutionary Multiobjective Optimization:
REFERENCES Theoretical Advances and Applications (ed. by A. Abraham, L.
Ajami, N. K., Duan, Q. & Sorooshian, S. (2007) An integrated Jain & R. Goldberg), 7–32. Springer-Verlag, Berlin, Germany.
hydrologic Bayesian multimodel combination framework: Coello Coello, C. A. & Pulido, G. T. (2001) A micro-genetic algo-
confronting input, parameter, and model structural uncertainty rithm for multiobjective optimization. In: Proc. First Int.
in hydrologic prediction. Water Resour. Res. 43, W01403, Conf. on Evolutionary Multi-Criterion Optimization (ed. by
doi:10.1029/2005WR004745. E. Zitlzer, K. Deb, L. Thiele, C. A. Coello Coello & D.
Ajami, N. K., Gupta, H., Wagener, T. & Sorooshian, S. (2004) Cal- Corne), 126–140. Lecture Notes in Computer Science no
ibration of a semi-distributed hydrologic model for stream- 1993, Springer-Verlag, Berlin, Germany.
flow estimation along a river system. J. Hydrol. 298(1-4), Cohon, J. I. (1978) Multiobjective Programming and Planning. Aca-
112–135. demic Press, New York, USA.
Andréassian, V., Lerat, J., Loumagne, C., Mathevet, T., Michel, Confesor, R. B. & Whittaker, G. W. (2007) Automatic calibration of
C., Oudin, L. & Perrin, C. (2007) What is really undermin- hydrologic models with multi-objective evolutionary algo-
ing hydrologic science today? Hydrol. Processes 21(20), rithm and Pareto optimization. J. Am. Water Res. Assoc. 43(4),
2819–2822. 981–989.
Corne, D. W., Jerram, N. R., Knowles, J. D. & Oates, M. J. (2001) table information using fuzzy rule based performance
PESA-II: Region-based selection in evolutionary multiobjec- measures. J. Hydrol. 291, 254–277.
tive optimization. In: Proc. Genetic and Evolutionary Compu- Gallart, F., Latron, J., Llorens, P. & Beven, K. (2007) Using
tation Conf. (ed. by L. Spector, E. Goodman, A. Wu, W. B. internal catchment information to reduce the uncertainty of
Langdon, H.-M. Voigt, M. Gen, S. Sen, M. Dorigo, S. discharge and baseflow prediction. Adv. Water Resour.
Pezeshk, M. H. Garzon & E. Burke), 283–290, Morgan 30(4), 808–823.
Kaufmann, San Francisco, California, USA. Goldberg, D. E. (1989) Genetic Algorithms in Search, Optimization
Dawdy, D. R. & O’Donnell, T. (1965) Mathematical models of and Machine Learning. Addison-Wesley, Reading, Massachu-
catchment behaviour. J. Hydraul. Div. ASCE 91(HY4), setts, USA.
123–127. Gupta, V. K. & Sorooshian, S. (1983) Uniqueness and observability
De Vos, N. J. & Rientjes, T. H. M. (2007) Multi-objective perform- of conceptual rainfall–runoff model parameters: the percola-
ance comparison of an artificial neural network and a concep- tion process examined. Water Resour. Res. 19(1), 269–276.
tual rainfall–runoff model. Hydrol. Sci. J. 52(3), 397–413. Gupta, H. V., Sorooshian, S. & Yapo, P. O. (1998) Toward improved
Deb, K., Pratap, A., Agarwal, S. & Meyarivan, T. (2002) A fast and calibration of hydrologic models: multiple and non-commensura-
elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. ble measures of information. Water Resour. Res. 34(4), 751–763.
Evol. Comput. 6(2), 182–197. Halhal, D., Walters, G. A., Ouazar, D. & Savic, D. A. (1997) Water
Diskin, M. H. & Simon, E. (1997) A procedure for selection of network rehabilitation with structured messy genetic algo-
objective functions for hydrologic simulation models. J. rithm. J. Water Resour. Plan. Manag. 123(3), 137–146.
Hydrol. 34(1-2), 129–149. Hall, J., O’Connell, E. & Ewen, J. (2007) On not undermining the
Duan, Q., Sorooshian, S. & Gupta, V. (1992) Effective and efficient science: Discussion of invited commentary by Keith Beven
global optimization for conceptual rainfall–runoff models. (2006) in Hydrol. Processes 20, 3141–3146. Hydrol. Proc-
Water Resour. Res. 28(4), 1015–1031. esses 21(7), 985–988.
Efstratiadis, A. & Koutsoyiannis, D. (2008) Fitting hydrological mod- Hamilton, S. (2007) Just say NO to equifinality. Hydrol. Processes
els on multiple responses using the multiobjective evolutionary 21(14), 1979–1980.
annealing–simplex approach. In: Practical Hydroinformatics: Harlin, J. (1991) Development of a process oriented calibration
Computational Intelligence and Technological Developments in scheme for the HBV hydrological model. Nordic Hydrol. 22,
Water Applications (ed. by R. J. Abrahart, L. M. See & D. P. 15–26.
Solomatine), 259–273. Springer Water Science and Technology Hornberger, G. M., Beven, K. J., Cosby, B. J. & Sappington, D. E.
Library, vol. 68, Springer-Verlag, Berlin, Germany. (1985), Shenandoah Watershed Study: Calibration of a
Efstratiadis, A. & Koutsoyiannis, D. (2009) On the practical use of topography-based, variable contributing area hydrological
multiobjective optimisation in hydrological model calibration. model to a small forested catchment. Water Resour. Res.
EGU General Assembly 2009, Geophys. Res. Abstr., vol. 11 21(12), 1841–1850.
(http://www.itia.ntua.gr/en/docinfo/901/). Horn, J., Nafpliotis, N. & Goldberg, D. E. (1994) A niched Pareto
Efstratiadis, A., Nalbantis, I., Koukouvinos, A., Rozos, E. & genetic algorithm for multiobjective optimization. In: Proc.
Koutsoyiannis, D. (2008) HYDROGEIOS: A semi-distributed First IEEE Conference on Evolutionary Computation (IEEE
GIS-based hydrological model for modified river basins. World Congress on Computational Intelligence), vol. 1, 82–87.
Hydrol. Earth System Sci. 12, 989–1006. Hunt, R. J., Doherty, J. & Tonkin, M. J. (2007) Are models too
Engeland, K., Xu, C. Y. & Gottschalk, L. (2005) Assessing uncer- simple? Arguments for increased parameterization. Ground
tainties in a conceptual water balance model using Bayesian Water 45(3), 254–262.
methodology. Hydrol. Sci. J. 50(1), 45–63. Jakeman A. J. & Hornberger, G. M. (1993) How much complexity is
Engeland, K., Braud, I., Gottschalk, L. & Leblois, E. (2006) Multi- warranted in a rainfall–runoff model? Water Resour. Res. 29,
objective regional modelling. J. Hydrol. 327(3-4), 339–351. 2637–2649.
Fenicia, F., Savenije, H. H. G., Matgen, P. & Pfister, L. (2007a) A Khu, S. T. & Madsen, H. (2005) Multiobjective calibration with
comparison of alternative multiobjective calibration strategies Pareto preference ordering: an application to rainfall–runoff
for hydrological modeling. Water Resour. Res. 43, W03434, model calibration. Water Resour. Res. 41, W03004,
doi:10.1029/2006WR005098. doi:10.1029/2004WR003041.
Fenicia, F., Solomatine, D. P., Savenije, H. H. G. & Matgen, P. Khu, S. T., Madsen, H. & Di Pierro, F. (2008) Incorporating multiple
(2007b) Soft combination of local models in a multi-objective observations for distributed hydrologic model calibration: An
framework. Hydrol. Earth System Sci. 11, 1797–1809. approach using a multi-objective evolutionary algorithm and
Feyen, L., Kalas, M. & Vrugt, J. A. (2008) Semi-distributed parame- clustering. Adv. Water Resour. 31(10), 1387–1398.
ter optimization and uncertainty assessment for large-scale Kim, S. M., Benham, B. L., Brannan, K. M., Zeckoski, R. W. &
streamflow simulation using global optimization. Hydrol. Sci. Doherty, J. (2007) Comparison of hydrologic calibration of
J. 53(2), 293–308. HSPF using automatic and manual methods. Water Resour.
Fonseca, C. M. & Fleming, P. J. (1993) Genetic algorithms for mul- Res. 43, W01402, doi:10.1029/2006WR004883.
tiobjective optimization: formulation, discussion and generali- Kirkby, M. (1975) Hydrograph modelling strategies. In: Processes in
zation. In: Proc. Fifth Int. Conf. on Genetic Algorithms. Physical and Human Geography (ed. by R. Peel, M. Chisholm
Morgan Kaufmann, San Mateo, California, USA. & P. Haggett), 69–90. Heinemann, London, UK.
Franks, S. W., Beven, K. J. & Gash, J. H. C. (1999) Multi-objective Knowles, J. D. & Corne, D. W. (2000) Approximating the nondomi-
conditioning of a simple SVAT model. Hydrol. Earth System nated front using the Pareto archived evolution strategy. Evol.
Sci. 3(4), 477–489. Comput. 8(2), 149–172.
Freer, J., Beven, K. J. & Ambroise, B. (1996) Bayesian estimation of Kuczera, G. & Mroczkowski, M. (1998) Assessment of hydrologic
uncertainty in runoff prediction and the value of data: an parameter uncertainty and the worth of multiresponse data.
application of the GLUE approach. Water Resour. Res. 32(7), Water Resour. Res. 34(6), 1481–1489.
2161–2173. Kuczera, G. & Parent, E. (1998) Monte Carlo assessment of parame-
Freer, J., McMillan, H., McDonnell, J. J. & Beven, K J. (2004) Con- ter uncertainty in conceptual catchment models: the Metropolis
straining Dynamic TOPMODEL responses for imprecise water algorithm. J. Hydrol. 211, 69–85.
Kunstmann, H., Krause, J. & Mayr, S. (2006) Inverse distributed Reed, P., Minsker, B. S. & Goldberg, D. E. (2003) Simplifying mul-
hydrological modelling of Alpine catchments. Hydrol. Earth tiobjective optimization: an automated design methodology for
System Sci. 10, 395–412. the nondominated sorted genetic algorithm-II. Water Resour.
Lamb, R., Beven, K. J. & Myrabø, S. (1998) Use of spatially distrib- Res. 39(7), 1196, doi:10.1029/2002WR001483.
uted water table observations to constrain uncertainty in a rain- Refsgaard, J. C. (1997) Parameterisation, calibration and validation
fall–runoff model. Adv. Water Resour. 22(4), 305–317. of distributed hydrological models. J. Hydrol. 198, 69–97.
Liong, S.-Y, Khu, S.-T. & Chan, W.-T. (2001) Derivation of Pareto Ritzel, B. J., Eheart, J. W. & Ranjithan, S. (1994) Using genetic
front with genetic algorithm and neural network. J. Hydrol. algorithm to solve a multiobjective groundwater pollution con-
Engng. 6(1), 52–60. tainment problem. Water Resour. Res. 30(5), 1589–1603.
Madsen, H. (2000) Automatic calibration of a conceptual rain- Rosbjerg, D. & Madsen, H. (2005) Concepts of hydrologic mode-
fall–runoff model using multiple objectives. J. Hydrol. 235, ling. In: Encyclopedia of Hydrological Sciences (ed. by M. G.
276–288. Anderson), Chap. 10. John Wiley & Sons, Chichester, UK.
Madsen, H. (2003) Parameter estimation in distributed hydrological Rouhani, H., Willems, P., Wyseure, G. & Feyen, J. (2007) Parameter
catchment modelling using automatic calibration with multiple estimation in semi-distributed hydrological catchment model-
objectives. Adv. Water Resour. 26, 205–216. ling using a multi-criteria objective function. Hydrol. Proc-
Madsen, H. & Khu, S.-T. (2002) Parameter estimation in hydrologi- esses 21(22), 2998–3008.
cal modelling using multi-objective optimization. In: Proc. Rozos, E., Efstratiadis, A., Nalbantis, I. & Koutsoyiannis, D. (2004) Cal-
Fifth Int. Conf. on Hydroinformatics (Cardiff, UK, July 2002), ibration of a semi-distributed model for conjunctive simulation of
vol. 2, 1160–1165. IAHR, IWA, IAHS. surface and groundwater flows. Hydrol. Sci. J. 49(5), 819–842.
Madsen, H., Wilson, G. & Ammentorp, H.C. (2002) Comparison of Schaffer, J. (1984) Some experiments in machine learning using vec-
different automated strategies for calibration of rainfall–runoff tor evaluated genetic algorithms. PhD Thesis, Vanderbilt Uni-
models. J. Hydrol. 261, 48–59. versity, Nashville, USA.
Meixner, T., Bastidas, L. A., Gupta H. V. & Bales, R. C. (2002) Schoups, G., Addams, C. L. & Gorelick, S. M. (2005a) Multi-objec-
Multicriteria parameter estimation for models of stream chem- tive calibration of a surface water–groundwater flow model in
ical composition. Water Resour. Res. 38(3), 1027, an irrigated agricultural region: Yaqui Valley, Sonora, Mexico.
doi:10.1029/2000WR000112. Hydrol. Earth System Sci. 9, 549–568.
Mo, X. & Beven, K J. (2004) Multi-objective parameter condition- Schoups, G., Hopmans, J. W., Young, C. A., Vrugt, J. A. &
ing of a three-source wheat canopy model. Agric. For. Met. Wallender, W. W. (2005b) Multi-criteria optimization of a
122, 39–63. regional spatially-distributed subsurface water flow model. J.
Montanari, A. (2007) What do we mean by uncertainty? The need Hydrol. 311, 20–48.
for a consistent wording about uncertainty assessment in Seibert, J. (2000) Multi-criteria calibration of a conceptual runoff
hydrology. Hydrol. Processes 21(6), 841–845. model using a genetic algorithm. Hydrol. Earth System Sci.
Montanari, A. & Brath, A. (2004) A stochastic approach for assess- 4(2), 215–224.
ing the uncertainty of rainfall–runoff simulations. Water Seibert, J. & McDonnell, J. J. (2002) On the dialog between experi-
Resour. Res. 40, W01106, doi:10.1029/2003WR00254. mentalist and modeler in catchment hydrology: use of soft data
Moore, C. & Doherty, J. (2006) The cost of uniqueness in groundwa- for multicriteria model calibration. Water Resour. Res. 38(11),
ter model calibration. Adv. Water Resour. 29, 605–623. 1241, doi:10.1029/2001WR000978.
Moussa, R. & Chahinian, N. (2009) Comparison of different Sivakumar, B. (2008) Undermining the science or undermining
multi-objective calibration criteria using a conceptual rain- Nature? Hydrol. Processes 22, 893–897.
fall–runoff model of flood events. Hydrol. Earth System Sorooshian, S., Gupta, V. K. & Fulton, J. L. (1983) Evaluation of
Sci. 13, 519–535. maximum likelihood parameter estimation techniques for con-
Moussa, R., Chahinian, N. & Bocquillon, C. (2007) Distributed ceptual rainfall–runoff models: influence of calibration data
hydrological modelling of a Mediterranean mountainous variability and length on model credibility. Water Resour. Res.
catchment—Model construction and multi-site validation. J. 19(1), 251–259.
Hydrol. 337(1-2), 35–51. Srinivas, N. & Deb, K. (1994) Multiobjective optimization using
Mroczkowski, M., Raper, G. P. & Kuczera, G. (1997) The quest for nondominated sorting in genetic algorithms. Evol. Comput.
more powerful validation of conceptual catchment models. 2(3), 221–248.
Water Resour. Res. 33(10), 2325–2335. Stedinger, J. R., Vogel, R. M., Lee, S. U. & Batchelder, R. (2008)
Muleta, M. K. & Nicklow, J. W. (2005) Sensitivity and uncertainty Appraisal of the generalized likelihood uncertainty estimation
analysis coupled with automatic calibration for a distributed (GLUE) method. Water Resour. Res. 44, W00B06,
watershed model. J. Hydrol. 306, 127–145. doi:10.1029/2008WR006822.
Pappenberger, F. & Beven, K. J. (2006) Ignorance is bliss: or seven Tang, Y., Reed, P. & Wagener, T. (2006) How effective and efficient
reasons not to use uncertainty analysis. Water Resour. Res. 42, are multiobjective evolutionary algorithms at hydrologic
W05302, doi:10.1029/2005WR004820. model calibration? Hydrol. Earth System Sci. 10(2), 289–307.
Pappenberger, F., Beven, K. J., Frodsham, K., Romanowicz, R. & Tang, Y., Reed, P. & Kollat, J. (2007) Parallelization strategies
Matgen, P. (2007) Grasping the unavoidable subjectivity in for rapid and robust evolutionary multiobjective optimiza-
calibration of flood inundation models: a vulnerability tion in water resources applications. Adv. Water Resour.
weighted approach. J. Hydrol. 333, 275–287. 30(3), 335–353.
Parajka, J., Merz, R. & Blöschl, G. (2007) Uncertainty and multiple Thiemann, M., Trosser, M., Gupta, H. & Sorooshian, S. (2001)
objective calibration in regional water balance modelling: case Bayesian recursive parameter estimation for hydrologic mod-
study in 320 Austrian catchments. Hydrol. Processes 21(4), els. Water Resour. Res. 37(10), 2521–2536.
435–446. Todini, E. (2007) Hydrological catchment modelling: past, present
Perrin, C., Michel, C. & Andréassian, V. (2001) Does a large number and future. Hydrol. Earth System Sci. 11(1), 468–482.
of parameters enhance model performance? Comparative Todini, E. & Montovan, P. (2007) Comment on: On undermining the
assessment of common catchment model structures on 429 science? by Keith Beven (2006) in Hydrol. Processes 20,
catchments. J. Hydrol. 242(3-4), 275–301. 3141–3146. Hydrol. Processes 21(12), 1633–1638.
Uhlenbrook, S., Seibert, J., Leibundgut, C. & Rodhe, A. (1999) Pre- Yapo, P. O., Gupta, H. V. & Sorooshian, S. (1996) Automatic cali-
diction uncertainty of conceptual rainfall–runoff models bration of conceptual rainfall–runoff models: sensitivity to cal-
caused by problems in identifying model parameters and struc- ibration data. J. Hydrol. 181, 23–48.
ture. Hydrol. Sci. J. 44(5), 779–797. Yapo, P. O., Gupta, H. V. & Sorooshian, S. (1998) Multi-objective glo-
Vrugt, J. A., Gupta, H. V., Bastidas, L. A., Bouten, W. & bal optimization for hydrologic models. J. Hydrol. 204, 83–97.
Sorooshian, S. (2003a) Effective and efficient algorithm for Ye, W., Bates, B. C., Vinley, N. R. Sivapalan, M. & Jackeman, A. J.
multiobjective optimization of hydrologic models. Water (1997) Performance of conceptual rainfall–runoff models in
Resour. Res. 39(8), 1214, doi:10.1029/2002WR001746. low-yielding ephemeral catchments. Water Resour. Res. 33(1),
Vrugt, J. A., Gupta, H. V., Bouten, W. & Sorooshian, S. (2003b) A 153–166.
Shuffled Complex Evolution Metropolis algorithm for optimi- Yu, P.-S. & Yang, T.-C. (2000) Fuzzy multi-objective function for
zation and uncertainty assessment of hydrologic model param- rainfall–runoff model calibration. J. Hydrol. 238, 1–14.
eters. Water Resour. Res. 39(8), doi:10.1029/2002WR001642. Zitzler, E. & Thiele, L. (1999) Multiobjective evolutionary algo-
Vrugt, J. A., Diks, C. G. H., Gupta, H. V., Bouten, W. & Verstraten, rithms: a comparative case study and the strength Pareto
J. M. (2005) Improved treatment of uncertainty in hydrologic approach. IEEE Trans. Evol. Comput. 3(4), 257–271.
modeling: combining the strengths of global optimization and Zitzler, E., Laumanns, M. & Thiele, L. (2001) SPEA 2: Improving
data assimilation. Water Resour. Res. 41, W01017, the strength Pareto evolutionary algorithm. TIK-Report 103,
doi:10.1029/2004WR003059. Swiss Fed. Inst. Technol., Zurich, Switzerland.
Wagener, T. & Gupta, H. V. (2005) Model identification for hydro- Zitzler, E., Thiele, L., Laumanns, M., Fonseca C. M. & da Fonseca, V. G.
logical forecasting under uncertainty. Stoch. Environ. Res. Risk (2003) Performance assessment of multiobjective optimizers: an
Assess. 19, 378–387. analysis and review. IEEE Trans. Evol. Comput. 7(2), 117–132.
Wagener, T., Boyle, D. P., Lees, M. J., Wheater, H. S., Gupta, H. V. Zhang, D., Beven, K. J. & Mermoud, A. (2006) A comparison of
& Sorooshian, S. (2001) A framework for development and nonlinear least square and GLUE for model calibration and
application of hydrological models. Hydrol. Earth System Sci. uncertainty estimation for pesticide transport in soils. Adv.
5(1), 13–26. Water Resour. 29, 1924–1933.

Efstratiadis 2010 One Decade of Multi Objective Calib

Uploaded by

Copyright:

Available Formats

Efstratiadis 2010 One Decade of Multi Objective Calib

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Efstratiadis 2010 One Decade of Multi Objective Calib

Uploaded by

Copyright:

Available Formats

Hydrological Sciences Journal – Journal des Sciences

ISSN: 0262-6667 (Print) 2150-3435 (Online) Journal homepage: https://www.tandfonline.com/loi/thsj20

One decade of multi-objective calibration

Andreas Efstratiadis & Demetris Koutsoyiannis

To link to this article: https://doi.org/10.1080/02626660903526292

Published online: 11 Mar 2010.

Submit your article to this journal

Article views: 7292

View related articles

Citing articles: 64 View citing articles

Full Terms & Conditions of access and use can be found at

One decade of multi-objective calibration approaches in hydrological

Andreas Efstratiadis & Demetris Koutsoyiannis

Une décennie d’approches de calage multi-objectifs en modélisation hydrologique: une revue

1 INTRODUCTION the formulation of a scalar performance criterion

Raw input data Processed

Raw response Processed

Feasible objective space

(Efstratiadis & Koutsoyiannis, 2008). Valuable a hyperrectangle in the m-dimensional objective

You might also like