General Review of Rainfall-Runoff Modeling:

Model Calibration, Data Assimilation,

and Uncertainty Analysis

Hamid Moradkhani∗ and Soroosh Sorooshian

Keywords Data assimilation · Model calibration · Hydrologic uncertainties

1 Introduction

All Rainfall-Runoff (R-R) models and, in the broader sense, hydrologic models are
simplified characterizations of the real world system. A wide range of R-R mod-
els are currently used by researchers and practitioners, however the applications
of these models are highly dependent on the purposes for which the modeling is
made. Many R-R models are used merely for research purposes in order to enhance
the knowledge and understanding about the hydrological processes that govern a
real world system. Other types of models are developed and employed as tools for
simulation and prediction aiming ultimately to allow decision makers to take the
most effective decision for planning and operation while considering the interac-
tions of physical, ecological, economic, and social aspects of a real world system.
Examples of some of the implications of latter type of R-R models are: real-time
flood forecasting and warning, estimating flood frequencies, flood routing and inun-
dation prediction, impact assessment of climate and land use change and integrated
watershed management.
The development of R-R models could be recognized based on the importance
of available data which provides the learning data set for calibrating the nonlinear
behavior of these models. These data are used as a priori knowledge in the model
with the logic that gives the flexibility to the model to extrapolate the R-R process
for some future time. This line of thinking, known as batch model calibration (using
batch of data for calibration), has been challenged by another philosophy that avail-
ability of observation continuously gives the opportunity to the model components
(state variables and even parameters) to be updated (corrected) sequentially. This is
thought to give more flexibility for taking advantage of the temporal organization

H. Moradkhani
Department of Civil and Environmental Engineering, Portland State University, Portland, OR, USA
∗
Work was completed while at the University of California, Irvine

S. Sorooshian et al. (eds.), Hydrological Modelling and the Water Cycle, 1–24 1

C Springer Science+Business Media B.V. 2009
2 H. Moradkhani, S. Sorooshian

and structure of information content for better compliance of the model output with
observed system response. The real time adjustment (fine-tuning) of R-R models for
flood forecasting is categorized under the latter philosophy. In fact the need for on-
line adjustments of R-R flood forecasting models was emphasized by WMO (1992).

2 Various Modeling Approaches: Lumped, Distributed and More

Owing to the complex nature of rainfall-runoff processes determined by a number
of highly interconnected water, energy and vegetation processes at various spatial
scales, hydrologists rely on their own understanding of the system gained through
interaction with it, observation and experiments. This process is known as percep-
tual modeling (Beven, 2001). Perceptualization of a hydrologic system leads the
modelers to a variety of ways to classify rainfall-runoff models from deterministic
to stochastic models, from physically-based (white-box) to black-box or empirical
and to conceptual models, and the most distinctive, from lumped models to dis-
tributed models (Clarke, 1973; Beven, 1985; Wheater et al., 1993; Refsgaard, 1996;
Beven, 2001). In lumped models, the entire river basin is taken as one unit where
spatial variability is disregarded. In such a modeling approach one tries to relate the
forcing data, mainly precipitation inputs, to system outputs (streamflow) without
considering the spatial processes, patterns and organization of the characteristics
governing the processes. On the other hand, a distributed model is one which ac-
counts for spatial variations of variables and parameters, thereby explicit character-
ization of the processes and patterns is made (Beven, 1985; Refsgaard, 1996; Smith
et al., 2004). The history of lumped R-R models is traced back to the last century
with the rational method which directly relates the precipitation input to the peak
discharge through the runoff coefficient parameter. Due to some of the difficulties
that this method causes (Beven, 2001), numerous complex lumped R-R models
have been developed and documented (Fleming, 1975; Singh, 1995; Singh and
Frevert, 2002a,b). The Crawford and Linsley’s Stanford Watershed Model (1962)
was one of the notably successful efforts in introducing a complex R-R model ac-
counting for the dynamics of hydrologic processes governing in a watershed. The
attempt by Crawford and Linsley (1962, 1966) was the first in benefiting from the
digital computers for a quantitative description of the hydrologic processes. Other
examples of conceptual R-R models are the Xinanjiang Model (Zhao et al., 1980),
and the widely used operational model in the US National Weather Service (NWS)
for flood forecasting, Sacramento Soil Moisture Accounting Model (SAC-SMA),
developed by Burnash et al. (Burnash et al., 1973; Burnash, 1995).
The lumped hydrologic models impose many assumptions, especially in large wa-
tersheds, as variables and parameters are representative average values (lumped) for
a river basin with semi-empirical equations describing the physics (Refsgaard, 1996).
These models are generally designed to simulate the streamflow just at the watershed
outlet. However, one may want to estimate the flow at some interior locations in a river
basin for engineering design, for real time operational flood forecasting and also for
studying the effects of land use or climate change. In general a rainfall-runoff model
requires representing the interaction of surface and subsurface processes. Freeze and
General Review of Rainfall-Runoff Modeling 3

Harlen (1969) were first to outline the underlying physics to describe such processes.
The benefits of distributed models as outlined by Beven (1985) and summarized by
Smith et al. (2004) are the possibility of considering spatially variable inputs and
outputs, assessment of pollutants and sediment transport, and also analyzing the hy-
drological response at ungauged basins. The availability of high spatial resolution
data such as DEM, precipitation, vegetation, soil and other atmospheric variables has
led to a surge in developing many sophisticated distributed hydrologic models. As
pointed out by Refsgaard (1996), in a distributed physically-based model the flows of
water and energy fluxes are computed from the prevailing partial differential equations
(e.g., Saint Venant equations for overland and channel flow, Richard’s equation for
unsaturated flow and Boussinesq’s equation for groundwater flow). Several variants
of distributed watershed models have been developed and implemented including
SHE (Abbott et al., 1986a,b), TOPMODEL (Beven and Kirby, 1976, 1979), IHDM
(Beven et al., 1987), THALES (Grayson et al., 1992), MIKE SHE (Refsgaard and
Storm, 1995), KINEROS (Smith et al., 1995), HBV (Bergstrom, 1995); IHDM (Calver
and Wood, 1995) to name a few.
The National Weather Service Hydrology Laboratory (NWS-HL) recently
launched a Distributed Model Intercomparison Project (DMIP) to encourage col-
laborative research into appropriate levels of model complexity, value of spatially
distributed data, and methods suitable for model development and calibration. This
effort revealed the readiness of operational organizations to move towards dis-
tributed hydrologic modeling and the fact that they see the distributed modeling
as a key pathway to infuse new science into their river and flash flood forecast oper-
ations and services (Smith et al., 2004; Koren et al., 2001). However, they are still
facing the question that under what circumstances and for what type of forecasting,
distributed modeling is profitable (Beven, 1985; Smith et al., 2004). Furthermore, if
there is a justification to employ these complex models, which model is the best to
improve the NWS forecasting capabilities?
The research community is facing some challenges in the application of the dis-
tributed models. There are still many outstanding questions regarding the parameter-
ization, calibration, and error correction of these complex models. The estimation
of the excessive parameters within distributed models is the main source of un-
certainty in these models. Most of the studies on distributed modeling can be
divided into two categories in regard to parameterization. Some studies propose
certain assumptions to simplify the parameterization (e.g., Homogeneity or fixing
some of the parameters, etc.) (Beven and Binley, 1992) and then to use calibra-
tion techniques to find optimal values for the rest of the parameters. The others
take into account the heterogeneity that exists in the watershed and use soil and
vegetation data to physically estimate the value of the parameters based on water-
shed characteristics within all the hydrologic units. In this case calibration is an
intense, time consuming, and inefficient procedure due to the number of parameters
involved.
These advantages motivated the National Oceanic and Atmospheric Adminis-
tration’s National Weather Service (NOAA/NWS) to initiate a distributed model
intercomparison project (DMIP) to infuse new science and technology into its river
forecasting capability Smith et al. (2004).
4 H. Moradkhani, S. Sorooshian

3 The Problem of Model Calibration (Parameter Estimation)

In general terms, hydrologic models are defined by state or prognostic variables
which define the dynamics of a system, and also parameters as quantities character-
izing the system. Parameters may be classified into physical and process parameters
(Sorooshian and Gupta, 1995). Physical parameters are those which can be mea-
sured directly independent of the observable river basin responses, such as, water-
shed area, impervious area in a watershed, local permeability obtained using core
samples, fraction of vegetated area, and areal percentage of water bodies. The pro-
cess parameters, on the other hand, are those which can not be measured directly and
needs to be inferred by indirect means (Gupta et al., 1998), such as, effective depth
of soil moisture storage, effective lateral interflow, rate of drainage for hypothetical
lumped storages, mean hydraulic conductivity, and surface runoff coefficient. Al-
though these parameters cannot be expected to have physical interpretations, they
are assumed to be related to inherent properties having physical relevance in hydro-
logic systems. In order for a model to closely and consistently simulate the observed
response (dynamic behavior) of a river basin over some historical period for which
forcing data (precipitation) and system output (e.g., streamflow) are available, the
model parameters need to be tuned or calibrated. The linkage between data, model
and parameters are shown schematically in Fig. 1. A variety of model calibration
techniques have been developed and implemented to ensure conformity between
the model simulations of system behavior and observations. A basic approach to
obtain the parameter values is the trial and error procedure, the so called manual cal-
ibration. The model knowledge and large number of model performance measures
defined by objective functions accompanied by visual inspection of the agreement
and differences between model predictions and the observation, and above all, the
human judgment are all taken to guide the adjustments to the best guess for model
parameters (Boyle et al., 2000; Duan, 2003). There are 3 levels associated with

DATA

MODEL
ETER
PARAM TION
A
ESTIM

Fig. 1 Linkage of data,

model and parameter
estimation (calibration)
General Review of Rainfall-Runoff Modeling 5

manual calibration (Boyle et al., 2000). In level zero, the watershed data are exam-
ined and a priori estimates of the likely values of parameter sets developed. This
provides the initial uncertainty of the estimates by defining the feasible parameter
ranges using the estimates from similar adjacent watersheds, look-up tables, etc. In
level one, the analysis of some segments of streamflow hydrograph that are most
relevant to specific parameter/s is made followed by the parameter adjustment. In
this step, the parameter interaction is generally disregarded. Finally, in level two,
the most difficult step, while examining the watershed hydrograph the parameter
interaction is taken into account and adjustment of the parameters is made accord-
ingly. The process of manual calibration is less affected by noises in calibration
data; however, the multitude of nonlinearly interacting parameters in hydrologic
models makes this procedure very labor-intensive requiring extensive training. This
expertise is not only difficult to gain, but also hard to transfer from one hydrologist to
another. These problems and difficulties justified the need for development of auto-
matic calibration techniques (Sorooshian and Dracup, 1980; Sorooshian et al., 1993;
Duan et al., 1992, 1993; Gupta et al., 1998; Hogue et al., 2000; Gupta et al., 2003a).

3.1 Mathematical Modeling of Hydrologic Systems

An R-R model can be cast in a mathematical framework irrespective of whether the

model is physical, empirical and/or conceptual. In fact we are interested in math-
ematical models that simulate or predict outputs from inputs. We denote the R-R
model by a nonlinear function f (.) signifying the derivative of the state vector x
with respect to time t. In addition, the system is characterized by k-member vector
of parameters θ , and forcing field (input) variables u as follows:

d xt
= f (xt , θ, u t ) + wt (1)
dt

Where, xt ∈  Nx is an N x -dimensional vector representing the system state

(for example catchment soil moisture content) at time t. The nonlinear operator
f :  Nx →  Nx expresses the system transition over a time instant in response to the
model input vector (forcing data, u, e.g., mean areal precipitation). In regards with
model parameters, it is further assumed that

θ ∈ ⌰ ⊆ k (2)

Where, k denotes the k-dimensional Euclidean space.

Due to error associated with the observed input to the system, the uncertainty
in parameter estimation and/or parameter identification and also model structural
error for accurate representation of physical data generating process, the aggregate
uncertainty may be defined through an additive error term wt which may consider
to be white noise random sequence with mean zero and covariance Q t .Considering
that the hydrologic data are available at discrete time, the continuous differential
6 H. Moradkhani, S. Sorooshian

form in eq. (1) is written as following the discrete stochastic dynamic state-space
form:

xt = f (xt−1 , θ, u t ) + wt (3)

Equation (3) is considered as an intermediate step to obtain the amount of internal

storages and fluxes in a hydrologic system; however, the eventual goal in R-R mod-
eling is to predict the output (runoff or streamflow) which are related to the model
states through the following equation:

ŷt = h(xt , θ ) (4)

where, ŷt ∈  N y is anN y -dimensional observation vector (observation simulation,

e.g., streamflow) as a function of model parameters and forecasted state variables
through the nonlinear operator h :  Nx →  N y .
Given the observed output values of y, the residual error term is written as:

vt = yt − ŷt (5)

Combining eqs. (4) and (5) yields the standard formulation of observation equation:

yt = h(xt , θ ) + vt (6)

3.2 Inverse Methods

The mathematical formulation of the R-R system described in Section 3.1 is the
typical Inverse Problem (IP) which could be explained as a problem where the input
and output observations and sometimes state variables are known, but not the model
parameters. Therefore, the problem of model calibration can be seen as the inverse
problem. This is in contrast to a Forward Problem (FP) where the relevant properties
of the model including initial and boundary conditions and also the parameters of the
system are known. A model then predicts the states and outputs straightforwardly.
Simply speaking, in FP, one wants to find effects (streamflow) from causes (param-
eters), while in IP, causes need to be found on the basis of effects. Unlike forward
learning which is a many-to-one mapping problem, the inversion learning problem
is one-to-many, therefore the mapping from output to input is generally non-unique.
In other words, the same effects (streamflow) may be caused by different causes
(parameters).
A problem is said to be well-posed if the three conditions for the solution are met
according to the definition given by Hadamard (1990): existence, uniqueness and
stability. According to this definition, the parameter estimation (model calibration)
is an ill-posed problem due to non-uniqueness of the solution for model parameters
(see Fig. 2). This is the typical case for inverse problems. The general formulation
General Review of Rainfall-Runoff Modeling 7

(a) Forward Problem (b) Inverse Problem

Well-posed Ill-posed
Causes (Parameters) Causes (Parameters)
Known Unknown

θ1 θ1
Effect (Streamflow) Effect (Streamflow)
Unknown Known
θ2 θ2
Q Q
θ3 θ3

θ4 θ4

Fig. 2 Schematic of forward problem vs. Inverse problem

of the forward and inverse problems can also be shown as follows (Castelli and
Entekhabi, 2002):
Consider a model f (D, θ ) = [ f 1 (D, θ ), f 2 (D, θ ), . . . , f L (D, θ )]T = 0 Consist-
ing of L elements which relates a data vector, D = [d1 , d2 , . . . d N ]T and a parameter
vector θ = [θ1 , θ2 , . . . θ M ]T . In R-R modeling the streamflow observation (Q) can
be considered as data in such formulation. Therefore, FP is concerned with estima-
tion (prediction) of the data D given the model f and the estimates of the model
parameters θ . In IP, the concern is to estimate the model parameters θ given model
f and observation o f D.
In a statistical sense, IP can be considered as the parameter estimation problem.
In such framework, the reliability and/or uncertainty of the estimation in addition to
the parameter estimates can be derived.

3.3 Model Calibration as an Optimization Problem

The nonlinear relationship between the parameters and outputs of R-R models
makes the linear regression solution methods impractical to use, thus reliance on
iterative or sequential (recursive) procedures to attain an acceptable solution is re-
quired. Iterative schemes work by repeatedly computing the model accuracy based
on the guessed model parameters and all available data. Iterative methods are nec-
essarily restricted to off-line applications, where a batch of data has been previously
collected for processing. In contrast, in sequential procedures, one uses each mea-
surement (real system output) as soon as it becomes available to update the model
parameters and model states which result in improved model outputs (Thiemann
et al., 2001; Moradkhani et al., 2005a,b) This attribute makes such approaches prac-
tical in either on-line or off-line applications. In this section, we focus our attention
to the batch automatic model calibration. Since the advent of digital computers,
automatic model calibration through optimization methods has been used exten-
sively to calibrate the conceptual R-R model parameters. In automatic calibration,
8 H. Moradkhani, S. Sorooshian

Measured Measured
Inputs Outputs

Real World
Yt
+
–

MODEL (θ) t
Computed
Outputs

Prior
Info θ Optimization
Procedure

Fig. 3 The concept of model calibration as an optimization problem

the problem is formulated as an optimization problem through objective function or

sometimes called loss or cost function (Fig. 3). This is a measure of the ability of
the model to replicate the observed system response. The success of any model cal-
ibration depends on the observed data, model structure, calibration conditions and
optimization procedure. Gupta et al. (2005) list the necessary conditions for a real-
istic hydrologic simulation by effectively estimating the model parameters called a
well-calibrated model. These conditions are: (1) consistency of model structure and
its behavior with current understanding of hydrologic processes, (2) consistency
between input-state-output behavior of the model (as formulated in eqs. 1–6) and
measurements of watershed behavior, (3) accuracy (unbiasedness) and precision
(small uncertainty) of model predictions.
Early attempts for R-R model calibration have been made in 1960s and 1970s
typically based on local search algorithms (Dawdy and O’Donnell 1965; Nash and
Sutcliffe, 1970; Ibbitt, 1970). The local search methods are carried out by initial-
ization of parameter sets and then iteratively minimizing the objective function to
direct the parameter search towards local improvement. The local search methods
are classified into direct methods (derivative-free) and gradient methods (derivative-
based).
The rotating method of Rosenbrock (Rosenbrock, 1960), Pattern search (Hooke
and Jeeves, 1961) and downhill simplex (Nelder and Mead, 1965) are examples of
direct search methods. Gradient methods appear to be more powerful than direct
methods as they use more information (including the first and second derivatives
of objective function with respect to parameters) to obtain the optimum objective
function (error response surface) value. The Newton and Quasi-Newton methods
are examples of derivative-based local search methods. The local search methods
for hydrologic model calibration have been reported by many others (Johnston and
Pilgrim, 1976; Pickup, 1977; Gupta and Sorooshian, 1985; Hendrickson et al., 1988;
among others) with the conclusion that local search methods cannot be used reliably
to estimate the global optimal solution.
General Review of Rainfall-Runoff Modeling 9

The philosophy behind any search algorithm in model calibration, random or

nonrandom, for obtaining the best parameter set in the feasible range lies some-
where between exploration and exploitation. By exploration, one means to improve
the objective function by randomly looking different regions of parameter space
regardless of what has already been learned from previous sampling. An example
of this type of parameter space search algorithm is uniform search. On the other
hand, in exploitation the decision on how to sample the parameter space depends on
the previous sampling. Examples of this type are the steepest descent and Newton-
Raphson methods. The direct search Monte Carlo algorithms, such as importance
sampling, falls somewhere between the exploration and exploitation methods. Better
exploration and exploitation of response surface function to globally optimize the
parameter sets was realized to be necessary owing to the existence of multiple local
optima, discontinuous derivatives and multiple regions of attractions in the parame-
ter space. Therefore, the attention geared towards global searching algorithms. This
method should be able to not only cope with the aforementioned problems, they
should be efficient and robust enough to overcome the problems that arise from the
high dimensionality of parameter space in operational hydrologic models such as the
SAC-SMA model and also the high degree of parameter interaction in such models.
Examples of the global search methods are the Adaptive Random search (Masri
et al., 1980; Brazil, 1988), Genetic algorithm (Holland, 1975; Goldberg, 1989;
Franchini, 1996), Simulated Annealing (Kirkpatrick et al., 1983).
A novel global search approach called Shuffle Complex Evolution (SCE-UA)
was introduced and implemented in a variety of hydrologic model calibration ap-
plications (Duan et al., 1992, 1993; Sorooshian et al., 1993). SCE-UA benefits
from the strength of different procedures and combines their strategies including
the Downhill Simplex, Controlled Random Search, and Competitive Evolution with
the proposed idea of Complex Shuffling.

4 Ensemble Inference vs. Optimization

Despite the effectiveness, consistency and efficiency of some of the global opti-
mization methods such as SCE-UA in reliably finding the global solution, numerous
studies have shown that many combinations of parameter sets (while even widely
distributed in parameter space) may result in similar objective function value, mean-
ing that several optimum solutions may exist for a problem. This reveals the prob-
lem of nonuniqueness or nonidentifiability of parameters, the so-called equifinality
(Beven, 1993) which is the cause of the existence of multiple feasible solutions
for the same problem. This view suggests that there may exist many representa-
tions of a watershed (many possible parameter sets) that may be equally capable
of simulating the observed system response and therefore rejects the concept of
optimum parameter set despite the strong desire in environmental science in ob-
taining the single optimal representation of reality. Especially when the objective
function in an optimization problem is highly irregular and nonlinear, multimodal
10 H. Moradkhani, S. Sorooshian

and nonsmooth with discontinuities, the global optimization procedures and matrix
inversion may fail to converge to the optimum solution due to numerical instabili-
ties caused by ill-conditioned matrices. To cope with these problems, Monte Carlo
(MC) methods as ensemble inference may be employed. MC procedures work by
direct sampling from the parameter space or associated probability distributions,
they are not dependent on the objective function to be smooth nor do they suf-
fer from numerical instabilities. MC methods based on the definition provided by
Hammersley and Handscomb (1964) are considered as a branch of experimental
mathematics that is concerned with experiments on random number. An up- to-date
definition provided by Sambridge and Mosegaard (2002) states that MC methods
are experiments making use of random numbers to solve either deterministic or
probabilistic (stochastic) problems. Although MC methods originated as methods
to solve the stochastic problems, later they were recognized as procedures to deter-
ministic problems such as multidimensional integration (Arulampalam et al., 2002;
Moradkhani 2004; Moradkhani et al., 2005b). The direct simulation of probability
distributions can fundamentally be related to Monte Carlo methods. The use of MC
methods in inverse problems as the information inference from measurement has
garnered the attention of earth system scientists over the past decade. They may
be used for inversion, parameter estimation, optimization, ensemble inference, and
uncertainty assessment.

5 Hydrologic Uncertainties

Hydrologic prediction is highly influenced by the uncertainties in the forcing data

(generally taken as deterministic), observed system response (due to errors in mea-
suring the physical quantities), imperfection of the model structure and the param-
eter values resulting from the model calibration which is profoundly affected by
uncertainty sources.

5.1 Uncertainty in Observation

The observation in R-R modeling is the measurement of the input and output fluxes
of hydrologic system and even the storage in the system (states). The key to potential
improvement of R-R modeling is associated with true characterization of precipita-
tion uncertainty. Precipitation uncertainty is generally regarded as the most influen-
tial cause of uncertainty in flood forecasting. One of the early methods (Bergestrom,
1976) still used in an operational setting is to improve the model response by manu-
ally modifying the observed input and rerunning the model until there is a reasonable
agreement between model output and observed runoff. Arguing about the validity
of this approach is not the thrust of this chapter; however this shows that the main
source of uncertainty in R-R modeling is the input (forcing data) uncertainty caus-
ing the significant effect on predictive uncertainty, not only due to the precision in
General Review of Rainfall-Runoff Modeling 11

observation but due to the spatial and temporal averaging of these quantities.
Therefore these fluxes need to be considered as stochastic quantities (Kavetski, 2003,
2006a,b; Clark and Slater, 2006; Hong et al., 2006; Moradkhani et al., 2006). As
shown by Clark and Slater (2006), uncertainty in model simulation is strongly influ-
enced by the reliability of the forcing variable and adequate characterization of their
associated uncertainties. The role of input uncertainty becomes even more important
for the ungauged regions or those regions where precipitation data are missing. In
such cases, one may rely on remotely sensed precipitation products extracted from
different satellite platforms. This will pose another challenge on how to estimate
the error associated with the remotely-sensed product. Recently Hong et al. (2006)
showed how the satellite precipitation error can be estimated through a power law
function as follows:
 b  b
1 ⌬t 1 ⌬t
σe = f ( , , P) = a · · (P)d (7)
L T L T

Where, σe is the error in precipitation (standard deviation between the satellite and
radar data) which is a function of spatial coverage A (here substituted by L as spa-
tial scale, the side length of A), temporal scale (T ), satellite sampling frequency
(Δt), and the space-time average of precipitation rate (P). a, b, c, and d are the
parameters of error model need to be calibrated. Also, Moradkhani et al. (2006)
demonstrated how this error model can be used in conjunction with other uncer-
tainty sources for the ensemble streamflow forecasting and how the interrelation
of uncertainties through Particle Filtering (Moradkhani et al., 2005b) will result in
combined streamflow uncertainty.
In addition to input data uncertainty, observed system response (streamflow) is
subject to error reflected in the rating curve inaccuracies at very high and very low
flows. This problem was addressed by Sorooshian and Dracup (1980), the problem
known as heteroscedasticity (variance changing) of error with respect to the magni-
tude of flow as opposed to homoscedasticity (constant variance) of error.

5.2 Uncertainty in Parameter and State Estimation

As discussed earlier, significant consideration has been given to the development

of automatic calibration methods aiming to successfully find a single best fitting
parameter value; however less effort has gone into assessment of parameter uncer-
tainty in hydrologic models. Therefore, despite the success of some of the global
optimization methods such as SCE-UA, reliance on a “best” answer remains unreal-
istic due to the existence of many combinations of parameter sets in the feasible
region. Poor identifiability of parameters may result in considerable uncertainty
in model output. This problem may be resolved by means of ensemble inference
through Monte Carlo procedures adopted by new generation of practitioners. Some
of the available methods to estimate the parameter uncertainty using the ensemble
inference are the Generalized Likelihood Uncertainty Estimation (GLUE) method
12 H. Moradkhani, S. Sorooshian

(Beven and Binley, 1992), the Bayesian Recursive Estimation (BaRE) algorithm
developed by Thiemann et al., (2001); the Metropolis method reported by Kuczera
and Parent (1998) and the Shuffled Complex Evolution Metropolis (SCEM-UA)
algorithm of vrugt et al. (2003). The GLUE and SCEM-UA are implemented in
a batch processing scheme where the collected period of data is used for cali-
bration and uncertainty assessment; however as Gupta et al. (2005) reported, the
GLUE can be applied recursively by choosing an appropriate likelihood function.
The GLUE method works based on different realization of parameter sets in order
to estimate the sensitivity of model prediction to various parameter sets. By do-
ing so, the parameter sets are categorized into behavioral and non-behavioral via
a likelihood measure. Those that are considered as non-behavioral are discarded
for prediction. The recent study by Mantovan and Todini (2006), however, reported
the reduced capacity of this method owing to its inconsistency with the Bayesian
inference process leading to large overestimation on uncertainty, both for the pa-
rameter estimation and hydrologic forecasting uncertainty assessment. The BaRE
algorithm employs a sequential approach to derive the probabilities associated with
parameter sets in an on-line fashion. The parameter probabilities result in probabilis-
tic output (streamflow) prediction. The problem with the original BaRE algorithm
(Beven and Young, 2003; Gupta et al., 2003b) was its tendency to collapse onto
a single point, a drawback modified by Misirli et al. (2003) through a resampling
procedure. The Metropolis method of Kuczera and Parent (1998) for the parame-
ter uncertainty estimation uses a random walk in the parameter space that adapts
to the true probability distribution of parameters. The underlying idea in the MH
method as a MCMC algorithm is generating samples of a probability distribution
over a high dimensional space where no explicit mathematical expression exists
for the probability distribution. The SCEM-UA is the extension of the SCE-UA
algorithm (Duan et al., 1992) with the difference that the Downhill Simplex method
which was used for population evolution in SCE-UA was replaced by the Metropo-
lis Hastings (MH) algorithm in SCEM-UA. The usage of MH keeps the SCE-UA
solution from collapsing into a single point (global optimum) in parameter space;
this is due to the stochastic nature of MH as a Markov Chain Monte Carlo (MCMC)
procedure.
Another procedure to modify parameters and estimate the uncertainty associated
with them is real time calibration via filtering techniques. Recursive estimation based
upon Kalman filter-type algorithms are generally used for estimating the dynamic
state of the system. However, as several authors have reported, the real time updat-
ing of state variables and parameter values is essential allowing the model to more
closely reproduce the observed system response due to the updating criteria car-
ried out in each observation time. (Todini, 1976, 1978; Kitanidis and Bras, 1980a,b;
Georgakakos, 1986a,b; Rajaram and Georgakakos, 1989; Georgakakos and Smith,
1990; Kivman, 2003; Moradkhani et al., 2005a,b). The real-time calibration using
the Extended Kalman Filter (EKF) for the operational SACramento Soil Moisture
Accounting Model (SAC-SMA) proposed by Georgakakos (1986a,b) was investi-
gated by WMO (1992) and concluded that the approach should not be recommended
because of its instability of parameter estimates (as a result of linearization of the
General Review of Rainfall-Runoff Modeling 13

EnKF (x) xt State at time t

Observation yt

EnKF (θ) θt Parameter at time t

Fig. 4 Dual State-parameter estimation using Ensemble Kalman Filter (EnKF)

system) and also computational requirements of implementing of such system. The

recent development by Moradkhani et al. (2005a) on dual state-parameter estima-
tion (see the schematic in Fig. 4) showed how the drawback of previous approaches
could be overcome while using the Ensemble Kalman Filter (EnKF). In such an ap-
proach, multiple possible model realization including parameters and state variables,
while incorporating the input uncertainty for generating the model replicates, are
used and the states and parameters are updated by the availability of observation.

6 Bayesian Inference and Sequential Data Assimilation

In the optimization methods, estimates of error are not readily available while
Bayesian inference provides a mechanism to combine the quantitative (hydrologic
data) and qualitative (prior information obtained by the experience of experts in
the field) data to yield the posteriori as more informative probability distribution of
variable of interest.
Bayesian formulation allows hydrologists to estimate the uncertainty about pre-
diction in a meaningful way and can be accomplished without resort to calibra-
tion which is sometimes problematic in certain applications. However, as shown
by Thiemann et al. (2001), vrugt et al. (2003) and Moradkhani et al. (2005a,b),
calibration as a paramount element in hydrologic prediction can be made within the
Bayesian paradigm.
In fact, in a Bayesian formulation, the solution to an inverse problem is given by
posterior probability distribution P(M|D) over the model space. P(M|D) encom-
passes all the available information on model which are taken from both data (D)
through the likelihood function P(D|M), and also data-independent prior informa-
tion expressed by prior probability P(M)density. The mathematical description of
Bayes law is given in below:

P(D|M)P(M)
P(M|D) = (8)
P(D)

Where the denominator, p(D) is the normalization factor, i.e., it ensures that the in-
tegration of p(M|D) results to 1. The likelihood function p(D|M) which measures
14 H. Moradkhani, S. Sorooshian

the likelihood of a given model Mthrough its misfit e(.),the error between observa-
tion and model simulation, is given in general form as follows:

p(D|M) ∝ ex p(−e(.)) (9)

With the assumption that the model residuals are mutually independent, normally
distributed, with constant variance (i.i.d.), the likelihood function can be computed
using (Box and Tiao, 1973):
   
1  e(.) 2
p(D|M) ∝ exp − (10)
2 ␴

In the absence of an explicit mathematical expression for P(D|M) andP(M), which

is common in high dimensional problems, Monte Carlo sampling is used to ex-
plore posterior P(M|D). The importance sampling, Metropolis-Hastings algorithm
and Gibbs Sampler are the most commonly used sampling techniques in practice.It
should be noted that the sampling should not be biased toward any particular region
of parameter space and thereby no possibility of entrapment in local minima.
The original Bayes law explained above eq. (8) is in the batch form where the
available historical data is taken for the uncertainty estimation through that con-
ditional probability. However, this form makes no attempt to include information
from new observations when becoming available. The flexibility required to use
the new information is provided by a sequential Bayesian scheme. Moradkhani
et al. (2005a,b) showed that the methods based on sequential Bayesian estimation
seem better able to benefit from the temporal organization and structure of informa-
tion achieving better conformity of the model output with observations.
If we consider the state variable xt as the variable of interest to be estimated
within the Bayesian framework, because of its stochastic nature, the pertinent infor-
mation about it at time t can be extracted from the observation Yt = [y1 , y2 , . . . yt ]
through the recursive Bayes law:

p(yt |xt ) p(xt |Yt−1 ) p(yt |xt ) p(xt |Yt−1 )

p(xt |Yt ) = p(xt |yt , Yt−1 ) = = (11)
p(yt |y1:t−1 ) p(yt |xt ) p(xt |Yt−1 )d xt

As seen in the schematic of recursive Bayes law (Fig. 5), the forecast density of
p(xt |y1:t –1 ) can be estimated via Chapman-Kolmogorov equation (Jazwinski, 1970)
considering the Markovian property of order one holds in eq. (3), therefore:

p(xt |Yt−1 ) = p(xt |xt−1 ) p(xt−1 |Yt−1 )d xt−1 (12)

The main complication in using the recursive Bayes law is that the multidimensional
integration of forecast density as shown in eq. (12) makes the closed form solution
of eq. (11) intractable. Therefore, reliance on ensemble methods through the Monte
Carlo sampling is required, which makes such problems solvable in practice. In the
General Review of Rainfall-Runoff Modeling 15

Forecast Density
at time t
p(xt | Yt−1)
Posterior Density Posterior Density
at time t-1 at time t
p(xt | Yt)
p(xt−1 | Yt−1)
p(yt | xt)
⊗

Likelihood at time t

Fig. 5 Sequential Bayesian scheme for evolution of the conditional probability density of the state
variables by assimilating observations from time t–1 to time t

next section we will explain how the conditional distribution of variable of interest
can be estimated using the sequential ensemble filtering.

6.1 Ensemble Filtering

The aim of filtering is to use the observations Yt over the time interval [0,t] to im-
prove the current state xt of a system. Two sequential estimation operations are
discerned in filtering applications: (1) the forecasting step which is the transition of
state variables from one observation time to the next as shown in eqs. (1) and (3) and
probabilistically represented through transition probability p(xt |xt –1 ) in eq. (12), (2)
the analysis (updating) step which involves updating of the forecasted (propagated)
states with the new observation. Ensemble procedures present a practical alternative
to an exact Bayesian solution by relying on discrete estimation of forecast (priori)
and analysis (posteriori) densities through a set of random variables and correspond-
ing weights:


N
p(xt |Yt−1 ) ≈ wti− δ(xt − xti− ) (13)
i=1


N
p(xt |Yt ) ≈ wti+ δ(xt − xti+ ) (14)
i=1

These are the empirical approximation of forecast and analysis densities by summa-
tion of N Dirac delta functions where x i and wi denote the ith sample and its weight
before and after updating shown by minus and plus signs respectively. The random
replicates and associated weights are generated through a variety of methods, two
of which are the ensemble Kalman filter (EnKF) and the Particle filter (PF). The
16 H. Moradkhani, S. Sorooshian

forecasting step in both EnKF and PF is the same where the evolution of the model
for each ensemble member is equally weighted.

xti− = f (xt−1
i
, θ, u it ) (15)
1
wti− = (16)
N

Thereby, the forecast density will be as follows:

1 
N
p(xt |Yt−1 ) ≈ δ(xt − xti− ) (17)
N i=1

It is noted that in this process, the random input replicates of u it are required to
generate the state trajectories in eq. (15). One way to generate the input repli-
cates is to consider the standard error obtained from eq. (7) and generate the ran-
dom variable using the Gaussian distribution as illustrated in Moradkhani et al.
(2006).
If the dynamical system, including states and measurement equations, are linear
and all sources of uncertainty are normally distributed, the celebrated Kalman filter
(Kalman, 1960) provides the optimal recursive solution to the state updating prob-
lem. If the system is nonlinear, as is the case for most of the hydrologic systems,
the linearization of the system might be considered. Developed from the early work
using state-space filtering, Georgakakos and Sperflage (1995) implemented an au-
tomatic procedure into the NWSRFS using the EKF. Certain shortcomings of the
procedure have been discovered including: reformulation of the original SAC-SMA
model to a state-space form; using first order approximation of Taylor series which
leads to unstable results when the nonlinearity in the model is strong; and heavy
computational demands owing to error covariance propagation. To overcome the
limitation of the EKF, the EnKF was introduced by Evensen (1994) which was
used for assimilating data in large nonlinear ocean and atmospheric model. The
EnKF is also based upon Monte Carlo or ensemble generations where the approx-
imation of the forecast state error covariance matrix is made by propagating an
ensemble of model states using the updated states from the previous time step.
The key point in the performance of the EnKF is to generate the ensemble of
observations at each update time by introducing noise drawn from a distribution
with zero mean and covariance equal to the observational error covariance matrix;
otherwise the updated ensemble will possess a very low covariance (Moradkhani
et al., 2005a).

xti+ = xti− + K t (yti − ŷti ) (18)

x y−  yy vv
−1
Kt = + (19)
t t t
General Review of Rainfall-Runoff Modeling 17
 yy
t is the forecast error covariance matrix of the prediction ŷt = h (.),
i i
where
x y− i−
t  is the forecast cross covariance of the state variables x t and prediction ŷti
and vv t is the observation error covariance of ν in eq. (6).
If it is assumed that the forecast and measurement are jointly normal, their den-
sities are sufficiently characterized by their mean and covariances, meaning that the
higher order moments can be ignored in the update step.
As pointed out and implemented by Pham (2001), Arulampalam et al. (2002)
and Moradkhani et al., (2005b), to improve the estimation accuracy and stability,
one may want to track the time evolution of the model by means of all moment
characteristics through a full probability density function. This is facilitated by using
particle filters. If we use the particle filtering for the updating step, the updated
ensemble members (particles) are kept the same as the forecast values and only the
weights are updated. Therefore,

xti+ = xti− (20)

and from eq. (11), the filtering posterior, wti+ is calculated as follows:

M
wti+ = M. p(yt |xti− )wti− = p(yt |xti− ) (21)
N

where, M is the normalizing constant in eq. (21). The important issue in using
the particle filters is the sampling concept through different methods such as the
Sequential Importance sampling (SIS), Sequential Importance Resampling or Sam-
pling Importance Resampling (SIR) as the most commonly used sampling proce-
dures. Using the proper sampling technique keeps the particles from dispersion due
to stochastic behavior of the system or degeneracy which is the collapsing of all
particles to a single point. For detailed information on particle filter and the sampling
techniques, see Moradkhani et al., (2005b).
In general filtering is used to recursively estimate the posterior distribution of
the model state; however, the successful use of sequential data assimilation relies
on unbiased model state prediction, which is largely dependent on accurate param-
eter estimation (Moradkhani et al., 2005a,b). Moradkhani et al., (2005b) extended
the application of the Bayesian recursive technique within the Monte Carlo frame-
work for adaptive inference of the joint posterior distribution of the parameters and
state variables within the sequential ensemble filtering. The use of this methodol-
ogy relaxes the need for restrictive assumptions regarding the variables’ probability
density function; i.e., it can handle the propagation of non-Gaussian distribution
through a nonlinear model properly. It provides a platform for improved estimation
of hydrologic model components and uncertainty assessment by complete represen-
tation of forecast and analysis probability distributions. In Fig. 6 the schematic of
joint Bayesian recursive estimation as the extension of Bayesian state estimation
approach is displayed.
Figure 7 shows an example of how such a system can produce a probabilis-
tic state-parameter estimate for ensemble streamflow forecasting. In Fig. 5, the
18 H. Moradkhani, S. Sorooshian

p(θk| θk−1) p(θk+1 | θk)

Parameter Evolution θk−1 θk θk+1

uk−1 uk uk+1
Input

p(xk|xk−1, θk−1) p(xk+1 | xk, θk)

xk−1 xk xk+1
State Evolution

Predictive p(yk|yk−1) p(yk+1 | yk)

yk−1 yk yk+1
Distribution
(Evidence)

Fig. 6 Schematic of combined Bayesian filtering for state-parameter estimation

evolutions of the uncertainties associated with one model parameter, one state
variable, and the streamflow output are displayed. Shaded areas in subplot (a) cor-
respond to 95, 90, 68 and 10 percentile confidence intervals. Also gray areas in
subplots (b) and (c) are associated with 95 percentile confidence interval with the
mean of ensembles shown by solid line, while the cross marks in (c) represent the
actual observation.

Spatial Variability
of Watershed [a]
Soil Moisture
Storage
(parameter)

[b]
Soil Moisture
Storage
(State Variable)

[c]

Streamflow
(Observation &
Simulation)

Time [Day]

Fig. 7 Uncertainty bound evolution of hydrologic model components resulting in the ensemble
(probabilistic) streamflow prediction
General Review of Rainfall-Runoff Modeling 19

7 Summary
In this chapter, the brief description of rainfall-runoff models was placed into per-
spective with some history and background on various modeling approaches with
the premise that the rainfall-runoff models are lumped or distributed conceptual-
ization of the real-world system. The main elements of conceptual models were
discussed and the mathematical configuration of a hydrologic system was illustrated
followed by the simple input-state-output characterization of a system. This struc-
ture was put into a mathematical context. The concept of model calibration (param-
eter estimation) as an inverse problem was reviewed and the historical enhancement
of calibration procedures were elaborated. While discussing the values of optimiza-
tion techniques as objective approaches to do the automatic calibration, the concept
of ensemble inference as an alternative to optimization of model performance was
illustrated. Especially when the objective function in an optimization problem is
highly irregular and nonlinear, multimodal and nonsmooth with discontinuities, the
global optimization procedures and matrix inversion may fail to converge to the
optimum solution due to numerical instabilities caused by ill-conditioned matrices.
To cope with these problems, Monte Carlo (MC) methods as ensemble inference
may be employed. The sources of uncertainties in hydrologic prediction were briefly
discussed and it was shown how the Bayesian inference and the sequential data
assimilation can be used for simultaneous calibration and uncertainty estimation.
The ensemble filtering, mainly ensemble Kalman filter and particle filter, within the
Bayesian paradigm were explained and a simple application was demonstrated.

Acknowledgments Financial support for the co-authors’ research and time to prepare this review
chapter was provided by a number of sources. We wish to acknowledge the support provided by
NOAA, NASA and NSF under various grants over the years and the University of California,
Irvine.
The second author also wishes to express appreciation to the University of L’ Aquila and
CETEMPS as well as Regione Abruzzo and the National Civil Protection Agency of Italy for
their support of the workshop.
Many thanks to the Drs. Erika Coppola and Kuolin Hsu who co-directed the workshop.
The support of the local organizing committee of Drs. Marco Verdecchia, Barbara Tomassetti
and Guido Visconti is much appreciated.

References
Abbott, M. B., J. C. Bathurst, J. A. Cunge, P. E. O’Connel, and J. Rasmussen (1986a): An introduc-
tion to European hydrological system – Systeme Hydrologique European, “SHE”, 1: History
and Philosophy of a physically-based Distributed modeling system. Journal of Hydrology, 87,
47–59.
Abbott, M. B., J. C. Bathurst, J. A. Cunge, P. E. O’Connel, and J. Rasmussen (1986b): An introduc-
tion to European hydrological system – Systeme Hydrologique European, “SHE”, 1: Structure
of a physically-based Distributed modeling system. Journal of Hydrology, 87, 61–77.
20 H. Moradkhani, S. Sorooshian

Arulampalam, M. S., S. Maskell, N. Gordon, and T. Clapp (2002): A tutorial on particle filters for
online nonlinear/non-Gaussian Bayesian tracking. IEEE Trans. Signal Process, 50(2), 174–188.
Bergstrom, S. (1995): The HBV model. In: Singh, V. P. (Ed.), Computer Models of Watershed
Hydrology, Water Resources Publications, Highlands Ranch, CO, pp. 443–476.
Beven, K. J. and Kirkby, M. J. (1976): Towards a simple physically based variable contributing
model of catchment hydrology. WorkingPaper 154, School of Geography, University of Leeds.
Beven, K. J. and Kirkby, M. J. (1979): A physically based, variable contributing area model of
basin hydrology, Hydrologic Sciences Bulletin, 24, 1, 3.
Beven, K. J., (1985): Chapter 13, Distributed Models. In: Anderson, M. G., and Burt, T. P. (Eds.),
Hydrologic Forecasting, Wiley, New York.
Beven, K. J., A. Calver, and A. M. Morris (1987): The institute of hydrology distributed model,
Institute of hydrology report 98, Wallingford, UK.
Beven, K.J. and A.M. Binley (1992): The future of distributed models: model calibration and un-
certainty in prediction. Hydrological Processes, 6, 279–298.
Beven, K. J. (1993): Prophecy, reality and uncertainty in distributed hydrological modeling. Ad-
vanced Water Resource, 16, 41–51.
Beven, K. J. (2001): Rainfall-runoff modeling – The primer, Wiley: Chichester, UK.
Beven, K. J. and P. C.Young (2003): Comment on “Bayesian recursive parameter estimation for hy-
drologic models” by Thiemann, M., M. Trosset, H. Gupta, and S. Sorooshian. Water Resource
Research, 39(5), COM 1–1, 1–4.
Box, G. E. P. and G. C. Tiao (1973): Bayesian Inference in Statistical Analyses. Addison-Wesley,
Boston, Mass.
Boyle, D. P., H. V. Gupta, and S. Sorooshian (2000): Toward improved calibration of hydrological
models: Combining the strengths of manual and automatic methods. Water Resource Research,
36(12), 3663–3674.
Brazil, L. E. (1988): Mutilevel calibration strategy for complex hydrologic simulation models,
Unpublished Ph.D. Thesis, Colorado State University, Fort Collins.
Burnash, R. J. C., R. L. Ferreal, and R. A. McGuire (1973): A generalized streamflow Simulation
System: Conceptual Modeling for Digital Computers, U.S. Department of Commerce, National
Weather Service and Department of Water Resources.
Burnash, R. J. C. (1995): The NWS river Forecast System-Catchment Modeling. In: Singh, V. P.
(Ed.), Computer Models of Watershed Hydrology, Water Resources – Publications, Highlands
Ranch, CO, pp. 311–366.
Calver, A. and W. L. Wood (1995): The institute of hydrology distributed model. Singh, V. P. (Ed.),
Computer Models of Watershed Hydrology, Water Resources – Publications, Highlands Ranch,
CO, pp. 595–626.
Castelli, F. and D. Entekhabi (2002): Models and Observations: Data Assimilation in Hydrologi-
cal and Geological Sciences, CNR-MIT Cooperation on Climate Change and Hydrogeological
Hazards in the Mediterranean Area.
Clark, M. P. and A. G. Slater (2006): Probabilistic quantitative precipitation estimation in complex
terrain. Journal of Hydrometeorology, 7(1), 3–22.
Clarke, R. T. (1973): A review of mathematical models used in hydrology, with some observations
on their calibration and use. Journal of Hydrology, 19, 1–20.
Crawford, N. H. and R. K. Linsley (1962): The synthesis of continuous streamflow hydrographs
on a digital computer, Technical report 12, Civil Engineering Department, Stanford University.
Crawford, N. H. and R. K. Linsley (1966): Digital simulation in hydrology: Stanford Watershed
Mode IV, Technical report 39, Civil Engineering Department, Stanford University.
Dawdy, D. R. and T. O’Donnell (1965): Mathematical models of catchment behavior. Journal of
Hydraulics Division, ASCE, 91 (HY4), 113–137.
Duan, Q., S. Sorooshian, and V. K. Gupta (1992): Effective and efficient global optimization for
conceptual rainfall-runoff models. Water Resources Research, 28(4), 1015–1031.
Duan, Q., S. Sorooshian, and V. K. Gupta (1993): Shuffled complex evolution approach for effec-
tive and efficient global optimization. Journal of Optimization Theory and Applications, 76(3),
501–521.
General Review of Rainfall-Runoff Modeling 21

Duan, Q. (2003): Global Optimization for Watershed Model Calibration, in Calibration of Wa-
tershed Models, Water Science Application Series (6), edited by Duan, Q. et al., AGU,
Washington, DC, 89–104.
Evensen, G. (1994): Sequential data assimilation with a nonlinear quasi-geostrophic model us-
ing Monte Carlo methods to forecast error statistics. Journal of Geophysical Research, 99,
10143–10162.
Fleming, G. (1975): Computer Simulation techniques in hydrology. Elsevier, New York.
Franchini, M. (1996): Use of a genetic algorithm combined with a local search method for the
automatic calibration of conceptual rainfall-runoff models. Hydrologic Science Journal, 41(1),
21–37.
Freeze, R. A. and R. L. Harlen (1969): Blueprint for a physically-based digitally simulated hydro-
logical response model. Journal of Hydrology, 9, 237–258.
Georgakakos, K. P. (1986a): A generalized stochastic hydrometeorological model for flood and
flash flood forecasting, 1. Formulation. Water Resources Research, 22(13), 2083–2095.
Georgakakos, K. P. (1986b): A generalized stochastic hydrometeorological model for flood and
flash flood forecasting, 1. Case studies. Water Resources Research, 22(13), 2096–2106.
Georgakakos, K. P. and G. F. Smith (1990): On improved Hydrologic Forecasting-Results from a
WMO real-time forecasting experiment. Journal of Hydrology, 114, 17–45.
Georgakakos, K. F. and J. A. Sperflage (1995): Hydrologic Forecast system-HFS: A user’s manual.
HRC Tech. Note 1, Hydrologic Research Center, San Diego, CA, 17pp.
Goldberg, D. E. (1989): Genetic Algorithms in search, optimization and machine learning,
Addison-Wesley publishing company, Reading, MA, p. 412.
Grayson, R. B., I. D. Moore, and T. A. McMahon (1992): Physically-based hydrologic model-
ing. 1. A terrain-based model for investigative purposes. Water Resources Research, 28(10),
2639–2658.
Gupta, V. K. and S. Sorooshian (1985): The relationship between data and precision of parameter
estimates of hydrologic models. Journal of Hydrology, 81, 57–77.
Gupta, V. K., S. Sorooshian, and P. O. Yapo (1998): Towards improved calibration of hydrological
models: multiple and noncomensurable measures of information. Water Resources Research,
34, 751–763.
Gupta, H. V., S. Sorooshian, T. Hogue, and D. P. Boyle (2003a): Advances in automatic calibration
of watershed models, in Calibration of Watershed Models, Water Science Application Series
(6), edited by Duan, Q. et al., AGU, Washington, D. C., pp. 113–124.
Gupta, H., M. Thiemann, M. Trosset, and S. Sorooshian (2003b): Reply to comment by K. Beven
and P. Young on “Bayesian recursive parameter estimation for hydrologic models”. Water Re-
source Research, 39(5), 1117, doi:10.1029/2002WR001405.
Gupta, H. V., K. J. Beven, and T. Wagener (2005): Model calibration and uncertainty estimation.
In: Anderson, M. G. and McDonnell, J. (Eds.), Encyclopedia of Hydrologic Sciences, Wiley,
Chichester, UK.
Hadamard, J. (1990): Psychology of Invention of Mathematical Field, Dover Publications, ISBN
0486201074.
Hammersley, J. M. and D. C. Handscomb (1964): Monte Carlo Methods. Chapman and Hall,
New York.
Hendrickson, J. D., S. Sorooshian, and L. Brazil (1988): Comparison of Newton type and direct
search algorithms for calibration of conceptual hydrologic models. Water Resources Research,
24(5), 691–700.
Hogue, T. S., S. Sorooshian, H. Gupta, A. Holz, and D. Braatz (2000): A multistep auto-
matic calibration scheme for river forecasting models. AMS Journal of Hydrometeorology, 1,
524–542.
Holland, J. H. (1975): Adaption in natural and artificial systems, University of Michigan press,
Ann Arbor, USA.
Hong, Y., K. Hsu, H. Moradkhani, and S. Sorooshian (2006): Uncertainty quantification of satellite
precipitation estimation and Monte Carlo assessment of the error propagation into hydrological
response. Water Resources Research, 42, W08421, doi: 10.1029/2005WR004398.
22 H. Moradkhani, S. Sorooshian

Hooke, R. and T. A. Jeeves (1961): Direct search solutions of numerical and statistical problems.
Journal of the Association for Computing Machinery, 8(2), 212–229.
Ibbitt, R. P. (1970): Systematic parameter fitting for conceptual models of catchment hydrology,
Ph.D. Dissertation, University of London, London, England.
Jazwinski, A. H. (1970): Stochastic Processes and Filtering Theory, Academic, 376 pp., San
Diego, Calif, CA.
Johnston, P. R. and D. H. Pilgrim (1976): Parameter optimization for watershed models. Water
Resources Research, 12(3), 477–486.
Kalman, R. (1960): New approach to linear filtering and prediction problems. Trans., AMSE
Journal of Basin Engineering, 82D, 35–45.
Kavetski, D., S.W. Franks, and G. Kuczera (2003):, Confronting input uncertainty in environmental
modeling , in Calibration of Watershed Models, Water Science Application Series (6), edited
by Duan, Q. et al., AGU, Washington, DC, 49–68.
Kavetski, D., G. Kuczera, and S. W. Franks (2006), Bayesian analysis of input uncertainty in hydro-
logical modeling: 1. Theory, Water Resour. Res., 42, W03407, doi:10.1029/2005WR004368.
Kavetski, D., G. Kuczera, and S. W. Franks (2006), Bayesian analysis of input uncertainty
in hydrological modeling: 2. Application, Water Resour. Res., 42, W03408, doi:10.1029/
2005WR004376.
Kirkpatrick, S., C. D. Gelatt, and M. P. Vechi (1983): Optimization by simulated annealing. Sci-
ence, 220, 671–680.
Kitanidis, P. K. and R. L. Bras (1980a): Real-time forecasting with conceptual hydrologic models,
1. Analysis of uncertainty. Water Resources Research, 16(6), 1025–1033.
Kitanidis, P. K. and R. L. Bras (1980b): Real-time forecasting with conceptual hydrologic models,
2. Applications and results. Water Resources Research, 16(6), 1034–1044.
Kivman, G. A. (2003): Sequential parameter estimation for stochastic systems. Nonlinear Pro-
cesses in Geophysics, 10, 253–259.
Koren, V., M. Smith, and Q. Duan (2001): Use of a priori parameter estimates in the derivation of
spatially consistent parameter sets of rainfall-runoff models, Calibration of Watershed Models,
Water Science and Applications Series edited by Duan, Q. et al., AGU.
Kuczera, G. and E. Parent (1998): Monte Carlo assessment of parameter uncertainty in conceptual
catchment models. The Metropolis algorithm, Journal of Hydrology, 211, 69–85.
Masri, S. F., G. A. Bekey, and F. B. Safford (1980): A global optimization algorithm using adaptive
random search. Applied mathematics and Computation, 7, 353–375.
Misirli, F., H. V. Gupta, S. Sorooshian, and M. Thiemann (2003): Bayesian recursive estimation
of parameter and output uncertainty for watershed models, Calibration of Watershed Mod-
els, Water Science Application Series (6), edited by Duan, Q. et al., AGU, Washington, DC,
pp. 113–124.
Moradkhani, H., K. Hsu, H. V. Gupta, and S. Sorooshian (2004): Improved streamflow forecast-
ing using self-organizing radial basis function artificial neural network. Journal of Hydrology,
295(1–4), 246–262.
Moradkhani, H. (2004): Improved uncertainty assessment of hydrologic models using data assim-
ilation and stochastic filtering, Ph.D. Dissertation, University of California, Irvine.
Moradkhani, H., S. Sorooshian, H. V. Gupta, and P. R. Houser (2005a): Dual state-parameter esti-
mation of hydrological models using ensemble Kalman filter. Advanced Water Resource, 28(2),
135–147.
Moradkhani, H., K.-L. Hsu, H. Gupta, and S. Sorooshian (2005b): Uncertainty assessment of
hydrologic model states and parameters: sequential data assimilation using the particle filter.
Water Resource Research, 41, W05012, doi:10.1029/2004WR003604.
Moradkhani, H., K. Hsu, Y. Hong, and S. Sorooshian (2006): Investigating the impact of remotely
sensed precipitation and hydrologic model uncertainties on the ensemble streamflow forecast-
ing. Geophysical Research Letters, 33, L12107, doi:10.1029/2006GL026855.
Nash, J. E. and J. V. Sutcliffe (1970): River flow forecasting through conceptual models, Part I- a
discussion of principles. Journal of Hydrology, 10(3), 282–290.
General Review of Rainfall-Runoff Modeling 23

Nelder, J. A. and R. Mead (1965): A simplex method for function minimization. Computer Journal,
7, 308–313.
Pickup, G. (1977): Testing the efficiency of algorithms and strategies for automatic calibration of
rainfall-runoff models. Hydrological Science Bulletin, 12(2) 257–274.
Pham, D. T. (2001): Stochastic methods for sequential data assimilation in strongly nonlinear sys-
tems. Monthly Weather Review, 129, 1194–1207.
Rajaram, H. and K. P. Georgakakos (1989): Recursive parameter estimation of hydrological mod-
els. Water Resources Research, 25(2), 281–294.
Refsgaard, J. C. (1996): Terminology, modeling Protocol and classification of hydrological model
codes. In: Abbott, M. B. and Refsgaard, J. C. (Eds.), Distributed Hydrological Modeling, Water
Science and Technology Library.
Refsgaard, J. C. and B. Storm (1995): MIKE SHE. In: Singh, V. J. (Ed.) Computer Models in
Watershed Hydrology, Water Resources Publications.
Rosenbrock, H. H. (1960): An automatic method for finding a greatest or least value of a function.
Computer Journal, 3, 175–184.
Sambridge, M. and K. Mosegaard (2002): Monte Carlo methods in geophysical inverse problems.
Reviews of Geophysics, 40(3), 1009, doi:10.1029/2000RG000089.
Singh, V. P. (1995): Computer Models of Watershed Hydrology. Water Resources Publications,
1130pp.
Singh, V. P. and D. K. Frevert (2002a): Mathematical Models of Large Watershed Hydrology.
Water Resources Publications, 891pp.
Singh, V. P. and D. K. Frevert (2002b): Mathematical Models of Large Watershed Hydrology and
Applications. Water Resources Publications, 950pp.
Smith, M. B., D. Seo, V. I. Koren, S. M. Reed, Z. Zhang, Q. Duan, F. Moreda, and S. Cong
(2004): The distributed model intercomparison project (DMIP): motivation and experiment
design. Journal of Hydrology (Amsterdam), 298, 4–26 ER.
Smith, E. E., D. C. Goodrich, D. A. Woolhiser, and C. L. Unkrich (1995): KINEROS: A KINematic
Runoff and EROSion Model. In: Singh, V. P. (Ed.), Computer Models of Watershed Hydrology,
Water Resources Publications, Highlands Ranch, CO, pp. 697–732.
Sorooshian, S. and J. A. Dracup (1980): Stochastic parameter estimation procedures for hydrologic
rainfall-runoff models: correlated and heteroscedastic error cases. Water Resource Research,
16(2), 430–442.
Sorooshian, S., Q. Duan, and V. K. Gupta (1993): Calibration of rainfall-runoff models: application
of global optimization to the soil moisture accounting model. Water Resource Research, 29(4):,
1185–1194.
Sorooshian S. and V. K. Gupta (1995): Model calibration. In: Singh, V. P. (Ed.), Computer
Models of Watershed Hydrology, Water Resources Publications, Highlands Ranch, CO,
pp. 23–67.
Thiemann, M., M. Trosset, H. Gupta, and S. Sorooshian (2001): Bayesian recur-
sive parameter estimation for hydrological models. Water Resource Research, 37(10),
2521–2535.
Todini, E., A. Szollosi-Nagy, and E. F. Wood (1976): Adaptive state-parameter estimation algo-
rithm for real time hydrologic forecasting: a case study, IISA/WMO workshop on the re-
cent developments in real time forecasting/ control of water resources systems; Laxemburg
(Austeria).
Todini, E. (1978): Mutually Interactive State-Parameter (MISP) estimation, Applications of
Kalman filters to hydrology, hydraulics and water resources, Proceedings of AGU Chapman
Conference, Edited by Chao-Lin Chiu.
P. Mantovan and E. Todini (2006), Hydrological forecasting uncertainty assessment: incoherence
of the GLUE methodology, J. Hydrol. 330 (2006) (1–2), pp. 368–381.
Vrugt, J. A., H. V. Gupta, W. Bouten, and S. Sorooshian (2003a):, A Shuffled Complex Evolu-
tion Metropolis algorithm for optimization and uncertainty assessment of hydrologic model
parameters. Water Resources Research, 39(8), 1201, doi:10.1029/2002WR001642.
24 H. Moradkhani, S. Sorooshian

Wheater, H. S., A. J. Jakeman, and K. J. Beven (1993): Progress and directions in rainfall-runoff
modeling. In: Jakeman, A. J., Beck, M. B., and McAleer, M. J. (Eds.), Modleing change in
environmental systems. John Wiley & Sons, Chichester, UK, pp. 101–132.
WMO (1992): Simulated real-time intercomparison of hydrological models, OHR-38. WMO
No. 779.
Zhao, R. J., Y. L. Zhuang, L. R. Fang, X. R. Liu, and Q. S. Zhang (1980): The Zinanjiang Model,
Hydrological Forecasting Proceedings Oxford Symposium, IAHS 129, pp. 351–356.