See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/275514783

MT3DMS: Model use, calibration, and validation

Article in Transactions of the ASABE (American Society of Agricultural and Biological Engineers) · July 2012
DOI: 10.13031/2013.42263

CITATIONS READS
57 6,551

4 authors:

Chunmiao Zheng Mary C. Hill

Southern University of Science and Technology University of Kansas
540 PUBLICATIONS 20,260 CITATIONS 189 PUBLICATIONS 8,526 CITATIONS

SEE PROFILE SEE PROFILE

Guoliang Cao Rui Ma

China University of Geosciences (Beijing) China University of Geosciences
37 PUBLICATIONS 2,027 CITATIONS 52 PUBLICATIONS 1,354 CITATIONS

SEE PROFILE SEE PROFILE

All content following this page was uploaded by Chunmiao Zheng on 06 August 2019.

The user has requested enhancement of the downloaded file.

 MT3DMS: MODEL USE, CALIBRATION,
AND VALIDATION
C. Zheng, M. C. Hill, G. Cao, R. Ma

ABSTRACT. MT3DMS is a three-dimensional multi-species solute transport model for solving advection, dispersion, and
chemical reactions of contaminants in saturated groundwater flow systems. MT3DMS interfaces directly with the U.S. Ge-
ological Survey finite-difference groundwater flow model MODFLOW for the flow solution and supports the hydrologic
and discretization features of MODFLOW. MT3DMS contains multiple transport solution techniques in one code, which
can often be important, including in model calibration. Since its first release in 1990 as MT3D for single-species mass
transport modeling, MT3DMS has been widely used in research projects and practical field applications. This article pro-
vides a brief introduction to MT3DMS and presents recommendations about calibration and validation procedures for field
applications of MT3DMS. The examples presented suggest the need to consider alternative processes as models are calibrat-
ed and suggest opportunities and difficulties associated with using groundwater age in transport model calibration.
Keywords. Contaminant transport modeling, Groundwater age simulation, Groundwater contamination, Groundwater
modeling, Groundwater remediation, MODFLOW, MT3DMS.

T
his article discusses the background and use of plored the applicability of the classical advection-
MT3DMS, a modular three-dimensional multi- dispersion model versus the alternative dual-domain mass
species solute transport model for application in transfer model at the Macrodispersion Experiment (MADE)
saturated groundwater flow systems. MT3DMS site; Barth et al. (2001b) compared the effect on solute
was originally referred to as MT3D and developed for sin- transport of independently measured and in situ values of
gle-species solute transport by S. S. Papadopulos and Asso- hydraulic conductivity; Mehl and Hill (2001) investigated
ciates, Inc., and subsequently documented as an open- the effects of numerical dispersion in subsurface transport
source code for the Robert S. Kerr Environmental Research models on sensitivity analysis and parameter estimation;
Laboratory of the U.S. Environmental Protection Agency Zheng and Wang (2002) analyzed the effectiveness of a
(Zheng, 1990). MT3D was significantly expanded and re- pump-and-treat system to restore a contaminated aquifer;
vised in 1999 with support from the U.S. Army Engineer Barth and Hill (2005a, 2005b) evaluated numerical issues
Research and Development Center (Zheng and Wang, and the importance of parameters and observations in the
1999). For more detailed theoretical background, numerical simulation of virus transport; and Bianchi et al. (2011) sim-
algorithms, and recent development, refer to Zheng (1990, ulated the effect of aquifer heterogeneity and resulting
1993, 2009), Zheng and Wang (1999), and Zheng and Ben- small-scale preferential flow paths on solute transport dur-
nett (2002). ing a dipole tracer test.
Since the initial release in 1990, the MT3D and
MT3DMS series of transport models have remained freely
accessible as open-source code and have been widely used MT3DMS DESCRIPTION
in research projects and practical field applications. For ex-
MT3DMS is intended to simulate changes in concentra-
ample, Wang and Zheng (1997) studied optimization of
tions of miscible contaminants in three-dimensional
groundwater remediation system design with contaminant
groundwater flow systems considering advection, disper-
concentrations as constraints; Feehley et al. (2000) ex-
sion, molecular diffusion, and some basic chemical reac-
tions, with various types of boundary conditions and exter-
nal sources or sinks. The chemical reactions included in
Submitted for review in January 2012 as manuscript number SW 9611; MT3DMS are equilibrium-controlled or rate-limited linear
approved for publication by the Soil & Water Division of ASABE in July or non-linear sorption, first-order radioactive decay or bio-
2012. degradation, and zero-order reaction or production.
The authors are Chunmiao Zheng, Professor, College of Engineering, MT3DMS interfaces directly with the U.S. Geological Sur-
Peking University, Beijing, China, and Department of Geological Scienc-
es, Tuscaloosa, Alabama; Mary C. Hill, Research Hydrologist, U.S. Geo- vey (USGS) finite-difference groundwater flow model
logical Survey, Boulder, Colorado; Guoliang Cao, Research Scientist, MODFLOW (McDonald and Harbaugh, 1988; Harbaugh
Center for Water Research, Peking University, Beijing, China; and Rui and McDonald, 1996; Harbaugh et al., 2000; Harbaugh,
Ma, Professor, School of Environmental Studies, China University of Ge-
osciences, Wuhan, China. Corresponding author: Chunmiao Zheng, De-
2005), which provides the flow solution required by the
partment of Geological Sciences, Tuscaloosa, AL 35487; phone: 205-348- transport simulation. MT3DMS supports transport through
0579; e-mail: [email protected].

Transactions of the ASABE

Vol. 55(4): 1549-1559 2012 American Society of Agricultural and Biological Engineers ISSN 2151-0032 1549
most of the hydrologic and discretization features of Groundwater Modeling System (GMS), Groundwater Vis-
MODFLOW. More sophisticated biological and geochemi- tas (GV), Processing MODFLOW (PM), and Visual
cal programs have been developed based on MT3D and MODFLOW (VM). Information on the USGS and com-
MT3DMS. For example, RT3D simulates kinetic reactions mercial products can be found on their respective websites.
related to hydrocarbon contamination (Clement, 1997);
SEAM3D considers complex biodegradation problems in- SOLUTION TECHNIQUES
volving multiple substrates and multiple electron acceptors MT3DMS contains multiple transport solution tech-
(Waddill and Widdowson, 1998); PHT3D combines niques in a single code, including the fully implicit finite-
PHREEQC-2, MT3DMS, and MODFLOW to accommo- difference method (FDM), the particle-tracking based
date a wide range of equilibrium-controlled and kinetic ge- method of characteristics (MOC) and its variants (modified
ochemical reactions (Parkhurst and Appelo, 1999; Prommer MOC and hybrid MOC), and a third-order total variation
et al., 2003); and SEAWAT couples MT3DMS and diminishing (TVD) scheme that conserves mass while lim-
MODFLOW into a single code to account for fluid density iting numerical dispersion and artificial oscillation. Since
dependence (Langevin et al., 2007). Based on an incom- no single numerical technique has been shown to be effec-
plete search, MT3DMS and its predecessor have been cited tive for all transport conditions, the combination of these
by several hundred journal articles that discuss a wide vari- solution techniques, each having its own strengths and limi-
ety of subsurface transport problems. tations, offers the best approach for solving the most wide-
MT3DMS is based on the classical advection-dispersion ranging transport problems with desired efficiency and ac-
conceptual model. In addition, it was implemented with an curacy (Zheng and Bennett, 2002; Konikow, 2011). For ex-
alternative dual-domain formulation for modeling solute ample, for advection-dominated transport problems, the
transport in highly heterogeneous media. With this formula- MOC solution scheme is most effective in eliminating the
tion, the porous medium is conceptualized as consisting of numerical dispersion error; however, it can lead to large
two distinct but overlapping domains: a mobile fluid do- mass discrepancy errors when the model grid is highly ir-
main where transport is predominantly by advection, and regular or when numerous sinks/sources are present. On the
an immobile fluid domain where transport is predominantly other hand, the finite-difference method is mass conserva-
by molecular diffusion (Zheng and Bennett, 2002). Instead tive and computationally efficient, but it is vulnerable to
of a single “effective” porosity for each model cell, two po- numerical dispersion errors for advection-dominated
rosities, one for the mobile domain and the other for the transport problems. The TVD scheme is an excellent com-
immobile domain, are used to characterize the porous me- promise between the MOC scheme and finite-difference
dium. The exchange between the mobile and immobile method, but it is often computationally demanding due to
domains is defined by a first-order mass transfer rate coef- the transport time step size restriction. Thus, only one of
ficient. The dual-domain mass transfer model may be more these solution techniques may work well under a unique
appropriate for modeling transport in fractured media or combination of field and numerical settings.
highly heterogeneous porous media than the single porosity MT3DMS includes an efficient and versatile matrix
advection-dispersion model, provided that the porosities solver based on generalized conjugate gradient (GCG)
and mass transfer rate coefficients can be properly charac- methods with three preconditioning options and the
terized (e.g., Feehley et al., 2000; Liu et al., 2010). Lanczos/ORTHOMIN acceleration scheme for non-
MT3DMS, in conjunction with MODFLOW, can ac- symmetrical matrices (see Zheng and Wang, 1999, Appen-
commodate very general spatial discretization schemes and dix A). The GCG solver is always activated to solve disper-
transport boundary conditions, including: (1) confined, un- sion, sink/source, and reaction terms implicitly without any
confined, or variably confined/unconfined aquifer layers; stability constraints. For the advection term, the user has
(2) inclined model layers and variable cell thickness within the option to select any of the solution schemes available,
the same layer; (3) specified concentration or mass flux including the implicit finite-difference method, the particle
boundaries; (4) the solute transport effects of external hy- tracking based Eulerian-Lagrangian methods (MOC and its
draulic sources and sinks, such as wells, drains, rivers, areal variants), and the third-order TVD method. The finite-
recharge, evapotranspiration; and (5) locally refined grids difference method is fully implicit without any stability
(Mehl et al., 2006). MT3DMS has been successfully ap- constraint to limit transport step sizes, but the particle
plied to field problems with spatial scales ranging from la- tracking based Eulerian-Lagrangian methods and the third-
boratory, plot, and site scales (measured in meters) over a order TVD method still have time step size constraints as-
temporal scale of hours to days (e.g., Barth et al., 2001a, sociated with particle tracking and TVD methodology.
2001b; Bianchi et al., 2011; Ma et al., 2010) to river basin
scales (measured in kilometers) over a temporal scale of PROGRAM STRUCTURE AND
decades (e.g., Cao, 2011). LINKAGE TO FLOW SOLVER
MT3DMS is freely available on-line as open-source MT3DMS is based on a modular structure similar to that
code. MT3DMS itself does not contain graphical user inter- implemented in the USGS modular three-dimensional fi-
faces. However, the USGS has developed a pre-processor nite-difference groundwater flow model MODFLOW
(ModelMuse) and a post-processor (Model Viewer) that (McDonald and Harbaugh, 1988; Harbaugh and McDonald,
support MT3DMS and are freely available. In addition, 1996; Harbaugh et al., 2000; Harbaugh, 2005). The modu-
several commercial vendors have developed and main- lar structure of the transport model makes it possible to
tained GIS-enabled graphical user interfaces, including

1550 TRANSACTIONS OF THE ASABE

simulate advection, dispersion/diffusion, source/sink mix- specific site. Deficiencies need to be understood and docu-
ing, and chemical reactions separately without reserving mented as much as possible, considering the specific pur-
computer memory space for unused options; furthermore, poses of the model and within specific ranges of field con-
new packages involving other transport processes and reac- ditions and performance criteria. For fundamental concepts,
tions can be added to the model readily without having to practical techniques, and field examples of model calibra-
modify the existing code. tion, refer to numerous discussions in the literature, includ-
MT3DMS was developed for use with any block- ing Sun (1994), Zheng and Bennett (2002), Hill and
centered finite-difference flow model and is based on the Tiedeman (2007), Clark et al. (2011), and Konikow (2011).
assumption that changes in the concentration field will not The following sections discuss three aspects of
affect the flow field significantly. MT3DMS has been most- MT3DMS model development and prediction. The first
ly used in conjunction with MODFLOW, but it can be section describes targets and the procedures by which they
linked to any other block-centered finite-difference flow are used to develop MT3DMS applications. The next sec-
model in a simple and straightforward fashion. After a flow tion addresses measures of goodness of model fit, sensitivi-
model is developed and calibrated, the information needed ty analysis, and uncertainty. The third section examines is-
by the transport model can be saved in external files, which sues of calibration parameters, including parameterization.
are then retrieved by the transport model. Since most poten-
tial users of a transport model are likely to be familiar with TARGETS AND PROCEDURES
MODFLOW or another block-centered finite-difference The targets used to calibrate an MT3DMS model are as-
flow model, MT3DMS provides an opportunity to simulate sociated with the flow model as well as the transport mod-
transport of contaminants without having to learn a new el. These might include, for example, heads and draw-
flow model or modify an existing flow model to fit the downs, flows such as streamflow gains and losses, point
transport model. The sequential simulation of flow and then concentrations, cumulative solute mass removed by extrac-
transport performed when using MT3DMS can result in tion wells or discharged to surface water bodies, and total
substantial savings in computer memory and execution mass remaining in the aquifer.
time. The increase in computational efficiency is especially After a solute transport model is constructed using the
significant when many transport runs are required while the MT3DMS code based on initial estimates of model struc-
flow solution remains the same. ture and input parameter values, the calibration takes place
by adjusting the structure and the values of input parame-
ters (including initial and boundary conditions) until the
MT3DMS CALIBRATION model results match the field observations to a reasonable
degree based on an analysis of data errors and model defi-
AND VALIDATION ciencies, while respecting what is known about system
To users of MT3DMS, the term “model calibration” characteristics, including the geology. Ideally, the user
generally describes a process in which the model structure starts the calibration process for a contaminant transport
and parameter values are adjusted, either manually or by model from pristine conditions everywhere in the aquifer.
using formal mathematical optimization procedures, until That is, with the measured plume distribution in the aquifer
the model output satisfactorily matches a set of targets and breakthrough curves at observation locations since the
(Zheng and Bennett, 2002). The suggestion by Hill and source release at time zero, the primary objective of the cal-
Tiedeman (2007) that “satisfactory” needs to be considered ibration is to achieve a satisfactory match between the field
in the context of data errors and model limitations is im- measurements and model results between the present and
portant to users of MT3DMS because of difficulties associ- time zero. The calibrated model may then be used for sce-
ated with simulating subsurface transport. Difficulties in- nario analysis or predictive assessment of future conditions.
clude the inaccessibility of subsurface systems and the However, this may be difficult to do, as the information on
many order of magnitude range of concentration values that contaminant sources is often uncertain or unavailable. As a
can occur in the data set for a single site. result, the existing plume distribution is sometimes treated
There is no consensus in the groundwater modeling com- as the initial condition to be calibrated and adjusted.
munity on the meaning and definition of “model valida- In some model applications, the targets may be used to
tion.” While some have referred to model validation as the implement formal optimization techniques of parameter es-
process by which the calibrated model is shown to be capa- timation using, for example, PEST (Doherty, 2010),
ble of reproducing a set of field observations or predicting UCODE (Poeter et al., 2005), and OSTRICH (Matotta and
future conditions without further adjustment to the calibrat- Rabideau, 2008). In other applications, a manual trial-and-
ed parameters, others have argued that, at least philosophi- error procedure may be used. When using optimization
cally, a groundwater model, like any scientific hypothesis, methods, concentration targets are useful as long as the
cannot be validated in the absolute sense and thus the term simulated plume overlaps the measured plume. If no over-
“model validation” should be avoided (Konikow and lap occurs, sensitivities will be zero and automated meth-
Bredehoeft, 1992). In this article, we do not attempt to sep- ods will not be able to use the concentration data to update
arate model validation from the overall model calibration the parameter values to obtain a better match. In such cir-
and testing process, which is viewed here as the process of cumstances, using the concentration to calculate the first
determining to what extent a calibrated model provides a moments of the plume and including the first moment as a
good representation of the actual processes occurring at a target provides a way for the optimization to correct the

55(4): 1549-1559 1551

flow field and move the plume toward the location reflect- and possibly more depending on the effects of unrepresent-
ed in the targets (for example, Anderman et al., 1996; Barth ed heterogeneity and other model inadequacies. The gen-
and Hill, 2005b). eral values stated here need to be adjusted for a given cir-
In developing the calibration strategy, decisions are cumstance. Once established, this context can be used to
needed as to whether the calibration is to be quasi-steady- identify poor model fit and possible model bias.
state (matching field observations in a snapshot), transient
(matching a time series), or both; what targets are to be MEASURES OF GOODNESS-OF-FIT, SENSITIVITY
matched; and what aspects of the model structure and ANALYSIS, AND UNCERTAINTY EVALUATION
which parameter values are to be adjusted. Decisions must Targets are used to judge the realism of model perfor-
also be made as to which parameters are known well mance. This is accomplished using measures of model fit,
enough or are unimportant enough to be treated as prede- i.e., the match between targets and simulated values. Sensi-
termined model input, and which should be estimated as tivity analysis identifies parameters important to model fit
part of the calibration. In general, the calibration often first and predictions, and observations important to parameters
proceeds iteratively between hydraulic (flow model) and and predictions. Uncertainty plagues models because model
chemical (transport model) results, and then all the pro- structure and parameter values are never completely known
cesses and data may be used simultaneously. Simultaneous and are, therefore, uncertain. As a result, model predictions
consideration is often important because, for example, both are uncertain.
flow-related data, such as heads and streamflow gains and The simulated values that are compared to targets de-
losses, and transport data, such as concentrations, are im- pend on the model structure and the values of model pa-
portant to calibrated hydraulic conductivity values (Barlebo rameters. Model structure refers to system geometry, pro-
et al., 1998, 2004; Hill and Tiedeman, 2007). cesses included in the simulation such as dispersion and
In some cases, evaluation of the data available for cali- reactions, and parameterization of spatially distributed and
bration may indicate that sufficient data simply do not exist time-varying characteristics such as hydraulic conductivity,
to obtain sufficient accuracy for the intended purpose of the porosity, and pumpage.
model. In these cases, a decision must be made as to The match between model responses and field observa-
whether the objectives of the simulation can be achieved in tions may be examined qualitatively and quantitatively. An
any meaningful way. If the objective is to gain an enhanced example of qualitative examination is a visual comparison
understanding of the transport regime, then calculations of contour maps of the calculated and observed heads or
with a model based even on very little data can often pro- concentrations. As noted above, groundwater transport
vide valuable insights. Frequently, these insights include models often include several kinds of targets, including
indications of what sorts of field data should be collected in heads, streamflow gains and losses, and concentrations. For
the future. If the goal is assessment of solute distributions different targets, the same difference between the target and
under various remedial alternatives, then calculation using simulated values may be important or unimportant. For ex-
a transport model with little or limited calibration, while ample, a difference of 1.0 is significant for a target concen-
certainly open to criticism, may sometimes represent the tration of 0.01 ppm, while it is insignificant for a target
best available approach. Clearly, any other type of calcula- concentration of 1,000 ppm. Overall measures of goodness-
tion will be subject to the same uncertainties in parameters, of-fit need to accommodate the range of targets used. One
will be less complete in the processes or geometries it can way this can be accomplished is with error-based
represent, and thus will be less suited for sensitivity analysis. weighting, as discussed by Foglia et al. (2009) and Hill and
The discussion on hydrologic model refutability pre- Tiedeman (2007, guideline 6), which is consistent with Re-
sented by Clark et al. (2011) applies to users of MT3DMS nard et al. (2011) and others. In error-based weighting, the
in its encouragement to define clear metrics by which mod- differences between targets and simulated values are
el inadequacy can be indicated. In groundwater flow mod- weighted before being combined to form a summary meas-
els such as the MODFLOW model that is often used by ure of model fit for all or part of a set of targets. Summary
MT3DMS, fit to heads with differences up to about 5% to measures include (1) the mean of weighted residuals (re-
10% of the total head loss through the system are thought siduals are the target minus simulated values; means that
to be typical. The error-based weighting arguments pre- differ from 0 indicate bias); (2) the variance or standard de-
sented by Hill and Tiedeman (2007) demonstrate how viation (also called standard error) of the weighted residu-
measurement error, unrepresented transient effects, and als errors or its square root (values close to 1.0 indicate
misrepresentation of subsurface geologic features contrib- model fit consistent with the evaluation of data and model
ute to justify this level of misfit. Streamflow gains and error used to determine the weights, smaller values may in-
losses are also important to many MT3DMS models and dicate overfitting, and larger values may indicate model er-
can be considered in the context of typical measurements ror); (3) the mean of the absolute values of the weighted re-
errors. If streamflows are measured to an accuracy of, for siduals (no guidance on value is available); and (4) the
example, 5%, the accuracy of the difference in streamflows linear correlation coefficient (r) (values close to 1.0 indicate
used to calculate streamflow gains from and losses to the close linear correspondence between observed and simulat-
groundwater system can be calculated by computing and ed values). These quantitative measures of goodness-of-fit
adding associated variances, as described by Hill and are defined by Hill (1998), Zheng and Bennett (2002), Hill
Tiedeman (2007, pp. 296-297). Concentrations are often and Tiedeman (2007), and Foglia et al. (2009).
thought to be measured to an accuracy of about 5% to 10%,

1552 TRANSACTIONS OF THE ASABE

 A frequently asked question is “What values of these Sensitivity analysis is an essential method for probing
statistical criteria are acceptable in model calibration?” The the behavior of a groundwater flow and transport system in
text above suggests that error-based weighting provides a response to changes in model structure and parameter val-
useful structure. In determining values for error-based ues. The information gained from the sensitivity analysis
weighting, it is important to keep in mind the magnitude of can greatly aid in the model calibration process and in un-
the observed head or concentration variation over the mod- derstanding the system. Manual sensitivity analysis is per-
el domain. If the variation in observed head or concentra- formed by changing a model input parameter and running
tion data over the model domain is small, it is obvious that the model again to determine the corresponding change in
a closer match between observed and calculated values the model response of interest. The ratio of the change in
would be expected. On the other hand, if the variation in the model response relative to a small change in model in-
observed head or concentration data over the model domain put yields the sensitivity coefficient (Zheng and Bennett,
is large, greater discrepancies would be expected. A simple 2002). Inverse parameter estimation codes such as PEST,
and crude measure of relative errors can be obtained by di- UCODE, and OSTRICH can be used to compute sensitivity
viding the statistical criteria by the difference between the coefficients for model parameters in an automated fashion.
maximum and minimum observed values over the model Greskowiak et al. (2010) provide an example of using sen-
domain. Results on the order of a few and sometimes up to sitivity analysis to understand the scale dependence of ura-
10% commonly are acceptable. nium reactive transport processes by comparing the sensi-
Because solute transport is greatly affected by signifi- tivity coefficients for laboratory column experiments versus
cant variabilities in the physical and chemical properties of field tracer test sites.
aquifer sediments, point measurements of the concentration It may be advantageous to conduct sensitivity analyses
field tend to exhibit considerable variations over space and before, during, and after model calibration. The reason for
time. Thus, basing the statistical measures of model calibra- doing so is to identify (1) targets important to parameters
tion purely on point concentration measurements may not and predictions of interest; (2) parameters for which the
be an effective calibration strategy, as it can miss the more available targets provide ample information, indicating the
important overall system behavior. Instead, other more sys- parameters that can be estimated; (3) parameters for which
tem-oriented measures, such as plume moments, frontal po- the targets provide little information, indicating that the pa-
sitions, mean arrival times, peak concentration values, and rameters cannot be estimated or are independent; and (4)
overall trends may be more effective in assessing the good- parameters that are important and unimportant to the pre-
ness-of-fit between field observations and model results dictions.
(e.g., Lu et al., 1999; Feehley et al., 2000; Julian et al., The uncertainty in model parameters can be quantified us-
2001; Barlebo et al., 2004). ing the parameter variance-covariance matrix and Bayesian
The objective of a particular model application also has a methods, and the resulting uncertainty in predictions can be
strong bearing on the consideration of model fit and on the quantified using linear and nonlinear confidence intervals
amount of effort it is wise to invest in the calibration process and Bayesian credible intervals (Lu et al., 2012). Statistics
(Zheng and Bennett, 2002). If the model is intended as a such as OPR (observation-prediction) and PPR (parameter-
management tool on which expensive remediation designs prediction), which can be calculated using UCODE_2005
may be based, it makes sense to refine the calibration, reduce and OPR-PPR (Poeter et al., 2005) and Tonkin et al.(2007),
the residual errors, and collect additional data as needed. On and PredUnc and PredVar (calculated using PEST) can be
the other hand, if the model is intended only as an analysis used to investigate the contributions to the measures of pre-
tool to help test hypotheses and “what-if” scenarios, relative- diction uncertainty and what data might be most useful in re-
ly large residuals may be acceptable as long as the basic sys- ducing measures of prediction uncertainty. Prediction uncer-
tem processes appear to be represented fairly well. tainty also arises because of model structure uncertainty. This
In many large-scale model applications, there are many can be evaluated by constructing models that represent the
degrees of freedom, and data available for calibration are different alternatives. Predictions and measures of prediction
sparse. A “perfect” calibration, as defined by minimal re- uncertainty produced by the alternative models can be com-
sidual errors, is an elusive goal and thus can go on forever, pared. Model-averaged values can be produced using model
as there are always some other parameter values to be ad- discrimination criteria such as AIC, AICc, BIC, and KIC us-
justed or new parameters to be added. Models can also ing, for example, the program MMA (Poeter and Hill, 2007).
match observations too closely, such that errors start to The use of such criteria in groundwater modeling is just be-
dominate the results (Hill, 2006, 2010). ing understood, and difficulties are being discovered and dis-
In addition to the statistical criteria, there are other, cussed in the literature, as indicated by Ye at al. (2008) and
harder to quantify factors that should be considered in as- Foglia et al. (2012). The calibration of a model may be con-
sessing the quality of model calibration. For example, an sidered adequate for a specific purpose if measures of predic-
important factor to consider in model calibration is the ro- tion uncertainty are consistent with the risk involved.
bustness of the model predictions relative to the model ob-
jectives or vice versa. It is possible that, under certain cir- CALIBRATION PARAMETERS
cumstances, changes in some model input parameters that For groundwater flow and transport modeling using
have a large effect on certain statistical criteria will have MODFLOW and MT3DMS, typically calibrated parame-
almost no effect on the model predictions relative to the ters are used to define hydraulic conductivity, storage prop-
model objectives. erties, dispersivities, and porosity. If the dual-domain mass

55(4): 1549-1559 1553

transfer formulation instead of the classical advection- reports to predict specifically when and where it might rain,
dispersion model is used, mass transfer rate coefficients be- transport models cannot precisely predict concentrations.
tween the mobile and immobile domains are often calibrat- However, they can often provide important insight. Wise
ed. If reactive transport is involved, sorption constants for use of transport model results is also discussed by Konikow
sorbing chemicals and possibly other chemical reaction rate (2011).
constants may be calibrated.
The calibrated parameters are used to define the associ-
ated property in a way defined by the system parameteriza- CASE STUDIES
tion. Parameterization options include, for example, zona-
Two practical examples are briefly presented in this sec-
tion, in which constant values are defined spatially and
tion. The first evaluates the effect of intraborehole flow and
temporally; interpolation using pilot points and other meth-
resulting solute mixing in long-screened wells on interpre-
ods; cell-by-cell variation; and so on. Models for which the
tation of tracer test results. The second considers the simu-
number of parameters defined is far fewer than the number
lation of groundwater flow and the mean age of groundwa-
of targets are called simply parameterized; models for
ter in the North China Plain.
which the number of parameters exceeds the number of tar-
gets are called highly parameterized models (Hill, 2006;
MODELING TRACER EXPERIMENTS AT THE
Hunt et al., 2007).
HANFORD SITE, WASHINGTON STATE
Given a constructed model, values of the input parame-
The first case study is adapted from the study by Ma et
ters can be assigned directly using applicable field infor-
al. (2011), which describes the application of the
mation or indirectly using calibration. Indirect methods de-
MODFLOW and MT3DMS codes to the Hanford IFRC
termine parameter values that produce simulated values
tracer experiment site. The Hanford IFRC site is located in
that are close to the targets. To make this easier,
the U.S. Department of Energy (DOE) Hanford 300 Area
MODFLOW and MT3DMS have observation processes
along the Columbia River in southeastern Washington State
that can produce files that list simulated values of interest.
(fig. 1). In this case study, a bromide anion (Br-) tracer ex-
Such files make it easy to read simulated values of interest
using “clip-on” model calibration and analysis programs
such as PEST, UCODE, and OSTRICH.
As in nearly all natural systems, calibration of models
constructed using MT3DMS is plagued by non-uniqueness,
i.e., different combinations of the parameter values and mod-
el structure alternatives lead to a similar value of goodness-
of-fit measures, such as the weighted least-squares objective
function. Non-uniqueness can be reduced by including mul-
tiple and independent targets to constrain the calibration to
the extent possible, including hydrologic, geophysical, and
geochemical data. In addition, the number of parameters to
be calibrated can be kept to the minimum initially, and ex-
panded subsequently in accordance with the availability of
field data suitable for use in the model calibration.
It is of interest to quote from a recent report issued by
the National Research Council: “Models will always be
constrained by computational limitations, assumptions, and
knowledge gaps. They can best be viewed as tools to help
inform decisions rather than as machines to generate truth
or make decisions” (NRC, 2007, p. 2). Thus, as pointed out
by Zheng (2011), as a simplification of reality, a numerical
model should never be expected to fully match the behavior
of a real system. Rather, it should be viewed as a tool de-
signed to fulfill clearly expressed tasks. As a result, model
calibration should not be an exercise to employ and fine-
tune many parameters to match every detail of the field ob-
servation. Instead, model calibration should clearly aim to
obtain a set of parameters that enables the numerical model
to capture the essential, overall behavior of the flow and
transport system under investigation. This is especially rel-
evant to the groundwater transport problems simulated us-
ing MT3DMS, in which much of the system is often unob-
servable, and simulated results are sensitive to those
Figure 1. (a) Plan view of the Hanford IFRC site showing the model
unobservable details. In such circumstances, it is general domain as a green rectangle and (b) the IFRC experimental well field
trends that tend to be reliable. Like the inability of weather (after Ma et al., 2011).

1554 TRANSACTIONS OF THE ASABE

periment conducted in March 2009 at the IFRC site was the wellbore head, the flow is toward the wellbore from the
numerically simulated and evaluated. The purpose of the aquifer. Conversely, if the head in the aquifer at a specific
case study was to evaluate the effect of intraborehole flow elevation is lower than the wellbore head, then the flow is
and resulting solute mixing in long-screened wells on inter- away from the wellbore into the aquifer. Thus, if vertical
pretation of tracer test results. hydraulic gradients exist across different model layers that
A three-dimensional groundwater flow model was de- are intersected by the well screen, the well can act as a
veloped with MODFLOW to simulate saturated flow with- conduit for intraborehole flow.
in the model domain shown in figure 1a. The model do- MT3DMS accounts for intraborehole solute mixing in
main was discretized into 60 columns (x-direction), response to flow exchanges between the wellbore and aqui-
61 rows (y-direction), and 27 layers (z-direction), covering fer by calculating a single composite concentration inside
a horizontal distance 120 m in the x-direction, 122 m in the the wellbore at the end of each transport time step. The
y-direction, and a vertical thickness of 22.5 m. Boundary composite concentration calculation is based on the mass
conditions were defined for the sides of model domain balance of fluxes and associated concentrations into and out
(looking in plan view) using the time-varying constant- of the wellbore over the course of the preceding transport
head (CHD) package for MODFLOW. The bottom of the time step. The composite flux-weighted concentration in-
grid was represented as a no-flow boundary. The uppermost side the wellbore is updated at the end of each transport
layer is an unconfined aquifer underneath a no-flow bound- time step.
ary because vertical groundwater recharge is negligible in Figure 2 compares the observed Br- concentrations
this semi-arid environment. Measured hourly water level (open circles) with the model-calculated Br- tracer break-
data from the existing wells within and outside the IFRC through curves obtained without (red lines) and with (green
site (fig. 1a) were used to interpolate time-varying head lines) consideration of intraborehole flow at six selected
values at the four lateral model boundaries. Three- well locations from the IFRC site. Agreement between
dimensional distribution of hydraulic conductivity was ini- simulated (green lines) and observed concentrations was
tially set based on data from the constant-rate and electro- markedly improved by explicitly simulating intraborehole
magnetic borehole flowmeter (EBF) tests (Newcomer et al., flow. The RMS (root mean square) error of 939 unweighted
2010), and these values were used as fitting parameters in residuals in 35 wells was reduced from 7.0 mg L-1 in the
the subsequent calibration. model that did not account for intraborehole flow to 5.1 mg
Br- transport was simulated with the MT3DMS code. In- L-1 in the model that accounted for intraborehole flow. This
itially, a longitudinal dispersivity (αL) of 1 m was used for case study highlights the importance of considering addi-
the highly permeable Hanford Formation, and 0.5 m was tional physical processes that might be overlooked during
assigned to the less permeable Ringold Unit. The horizontal model calibration.
and vertical transverse dispersivities were assumed to be
10% and 1% of the longitudinal dispersivity in all geologi- MODELING GROUNDWATER AGE DISTRIBUTION
cal units, respectively, based on previous studies (Ma et al., IN THE NORTH CHINA PLAIN
2010). Dispersivities defined as being constant throughout The second case study is adapted from Cao (2011), who
the two units were estimated during model calibration, applied MODFLOW and MT3DMS to simulate groundwa-
along with values of hydraulic conductivity. ter flow and mean age distribution in the quaternary aquifer
Manual adjustment and inverse modeling were em- system of the North China Plain (NCP). The NCP is
ployed for both flow and transport model calibration. The bounded in the north by Yan Mountain, in the west by Tai-
inverse modeling was based on the “pilot point” method, as hang Mountain, in the south by the Yellow River, and in the
implemented in the PEST code (Doherty, 2010). Over east by the Bohai Sea (fig. 3). For the flow model, the aqui-
500 pilot points were used to parameterize the hydraulic fer system receives recharge from rainfall and irrigation re-
conductivity at predefined locations, from which the three- turn flow, as well as laterally from the mountain fronts and
dimensional hydraulic conductivity distribution for the en- leakage from the Yellow River. The discharge terms include
tire model was interpolated using kriging. Although the pumping, evapotranspiration, and outflow to the Bohai Sea.
model domain extended beyond the IFRC site to minimize For the transport model, the mean age of all inflow terms is
the effects of boundary conditions, hydraulic conductivity set to zero, as it is considered negligible relative to the
values were estimated only within the IFRC site, where de- transport time scale. The mean age is simulated using the
tailed hydraulic conductivity measurements were made. An concentration as a proxy, based on the method described by
average hydraulic conductivity value, which was based on Goode (1996).
a few available measurements and knowledge of the site Figure 4 illustrates the distribution of calculated mean
geology and texture of subsurface material, was assigned groundwater ages for the NCP under the boundary condi-
where detailed hydraulic conductivity data were not availa- tions described above. The age distribution was obtained by
ble. running the age transport model in steady-state mode. It can
The multi-node well (MNW) package for MODFLOW be seen that the calculated groundwater ages range from
(Halford and Hanson, 2002) was used to assess intrabore- relatively young (less than 100 years old) near the moun-
hole vertical flow in the fully-screened wells. The MNW tain fronts to very old (up to 30,000 years old) in some
package calculates a single head inside the fully screened deeper parts of the aquifer system. The calibration of the
wellbore, which cuts through multiple model layers. When age transport model was carried out through a trial-and-
the head in the aquifer at a specific elevation is higher than error procedure by matching the calculated ages with

55(4): 1549-1559 1555

Figure 2. Comparison of observed concentrations and simulated concentrations calculated with and without the multi-node well package
(MNW) at six well locations of the Hanford IFRC site (after Ma et al., 2011).

measured carbon-14 age dating values. In addition, the cal- challenging. First, the number of wells used for calibration
culated age distribution, as shown in figure 4, was visually (~200) is small relative to the size of the regional model
and qualitatively compared with a contour map derived (~140,000 km2), as is inevitably the case in any regional
from the measured carbon-14 age data. It is noteworthy that basin-scale modeling. Second, direct field measurement of
the calibration of the calculated age distribution is very groundwater ages based on such techniques as carbon-14
dating is subject to significant uncertainty and ambiguity,
and thus it is not easy to obtain high-quality data points for
calibration purposes (also noted by Sanford, 1997). Finally,
regional-scale transport parameters such as dispersivities
are not well defined and may not be unique. These issues
underscore the difficulties of model calibration discussed in
the previous section. As a result, a modeling study such as
this example should be viewed more as conceptual than

Figure 4. Distribution of calculated mean groundwater ages in the

Figure 3. Location of the North China Plain (after Cao, 2011). North China Plain (after Cao, 2011).

1556 TRANSACTIONS OF THE ASABE

quantitative. Still, a study of this nature improves the un- Zheng and Wang (1999): (1) ability to account for the effect
derstanding of the three-dimensional regional flow system of intraborehole solute mixing in a long monitoring well
and provides a basis for assessing the field measurements. that intercepts multiple model layers, (2) support for more
types of chemical reactions, (3) the option to define arbi-
trary time-varying mass loading sources and boundary con-
SUMMARY AND DISCUSSION ditions, (4) the addition of new input/output features,
(5) the capability for direct simulation of groundwater age
MT3DMS is a powerful and versatile solute transport
distributions subject to hydrodynamic mixing, and (6) a
simulator that has been successfully applied to hundreds of
procedure for modeling heat transport based on its mathe-
sites and research problems over the past 20 years. On-
matical equivalence to solute transport. In addition, active
going development continues to enhance its functionality
development work is on-going to improve the variable-
and applicability in solving complex reactive transport
density flow and transport capability through the SEAWAT
problems. The greatest strengths of MT3DMS are the
code (Langevin et al., 2007) and to add the variably satu-
availability of multiple transport solvers in a single pro-
rated flow and transport capability on the basis of the UZF
gram, a large number of case studies demonstrating its reli-
package of MODFLOW (Niswonger et al., 2006).
ability and flexibility, and availability of numerous com-
Model calibration, sensitivity analysis, and uncertainty
panion and supporting software packages. The current
evaluation capabilities continue to expand with computa-
limitations of MT3DMS include that (1) it only applies to
tional resources and experience. On-going work developing
the saturated groundwater flow system, (2) it is restricted to
and comparing computationally frugal (10s to 100s of
the block-centered finite-difference grid, and (3) it only
model runs) and expensive (10,000s to 1,000,000s of model
supports classical advection-dispersion and alternative du-
runs) methods, as represented by Foglia et al. (2007, 2009)
al-domain mass transfer conceptual models.
and Lu et al. (2012), is important to users of MT3DMS be-
Transport models developed on the basis of the
cause transport modeling can produce lengthy execution
MT3DMS code can be calibrated using standard approach-
times. These issues become increasingly important when
es and procedures discussed in previous sections. The two
conducting simulations with density dependence and multi-
case studies illustrate and underscore the challenges in suc-
component reactions.
cessful transport model calibration. The first example
shows that, regardless of how much calibration effort is un-
dertaken, if a physical process is not properly included in ACKNOWLEDGEMENTS
The senior author is supported by grants from the China
the transport model, the calibration process can only yield
Geological Survey (No. 1212011121174) and the National
misleading results. The second example highlights the dif-
Natural Science Foundation of China (No. 41140017).
ficulty in regional basin-scale transport model calibration,
as point-by-point comparison of calculated and measured
values is unlikely to lead to reliable calibration. Instead,
evaluation and assessment of overall system behaviors sup- REFERENCES
ported by multiple sources of data is preferred. Anderman, E. R., M. C. Hill, and E. P. Poeter. 1996. Two-
The data issue remains the bottleneck for contaminant dimensional advective transport in ground-water flow
transport modeling. Pervasive subsurface heterogeneities parameter estimation. Ground Water 34(6): 1001-1009.
arising from the large variabilities in physical and chemical Barlebo, H. C., M. C. Hill, D. Rosbjerg, and K. H. Jensen. 1998.
Concentration data and dimensionality in groundwater models:
properties control the fate and transport of contaminants in
Evaluation using inverse modeling. Nordic Hydrol. 29(3): 149-
the porous media. Yet field characterization of physical and 178.
chemical heterogeneities at sufficiently high resolution is Barlebo, H. C., M. C. Hill, and D. Rosbjerg. 2004. Investigating
difficult and expensive, resulting in often inadequate data the macrodispersion experiment (MADE) site in Columbus,
for model development and calibration. Based on long-term Mississippi, using a three-dimensional inverse flow and
experimental data sets and insights from an intensively in- transport model. Water Resources Res. 40(4): W04211, doi:
strumented field tracer test site, Zheng et al. (2011) summa- 10.1029/2002WR001935.
rized the major challenges and future directions in quantita- Barth, G. R., and M. C. Hill. 2005a. Numerical methods for
tive prediction of contaminant transport processes. New improving sensitivity analysis and parameter estimation of
virus transport simulated using sorptive-reactive processes. J.
field techniques are needed to permit rapid, cost-effective,
Contam. Hydrol. 76(3-4): 251-277.
and highly resolved characterization of aquifer properties Barth, G. R., and M. C. Hill. 2005b. Parameter and observation
for improved model calibration and validation. New importance in modeling virus transport in saturated systems:
transport models are also needed to account for and repre- Investigations in a homogenous system. J. Contam. Hydrol.
sent “non-ideal” transport behaviors resulting from the 80(3-4): 107-129.
presence of connected high-conductivity preferential flow Barth, G. R., T. H. Illangasekare, M. C. Hill, and H. Rajaram.
pathways. 2001a. A new tracer-density criterion for heterogeneous porous
media. Water Resources Res. 37(1): 21-31.
Barth, G. R., M. C. Hill, T. H. Illangasekare, and H. Rajaram.
2001b. Predictive modeling of flow and transport in a two-
FUTURE DEVELOPMENTS dimensional intermediate-scale, heterogeneous porous media.
The most recent release of MT3DMS is 5.3 (Zheng, Water Resour. Res. 37(10): 2503-2512.
2010), which includes the following features added since Bianchi, M., C. Zheng, G. R. Tick, and S. M. Gorelick. 2011.

55(4): 1549-1559 1557

 Investigation of small-scale preferential flow with a forced- Hill, M. C. 2010. Comment on “Two statistics for evaluating
gradient tracer test. Ground Water 49(4): 503-514. parameter identifiability and error reduction” by John Doherty
Cao, G. 2011. Recharge estimation and sustainability assessment and Randall J. Hunt. J. Hydrol. 380(3-4): 481-488.
of groundwater resources in the North China Plain. Hill, M. C., and C. R. Tiedeman. 2007. Effective Groundwater
Unpublished PhD diss. Tuscaloosa, Ala.: University of Model Calibration: With Analysis of Sensitivities, Predictions,
Alabama, Department of Geological Sciences. and Uncertainty. New York, N.Y.: Wiley.
Clark, M. P., D. Kavetski, and F. Fenicia. 2011. Pursuing the Hunt, R. J., J. Doherty, and M. Tonkin. 2007. Are models too
method of multiple working hypotheses for hydrological simple? Arguments for increased parameterization. Ground
modeling. Water Resour. Res. 47: W09301, doi: 10.1029/ Water 45(3): 254-262.
2010WR009827. Julian, H. E., J. M. Boggs, C. Zheng, and C. E. Feehley. 2001.
Clement, T. P. 1997. RT3D: A modular computer code for Numerical simulation of a natural gradient tracer experiment
simulating reactive multi-species transport in 3-dimensional for the natural attenuation study: Flow and physical transport.
groundwater aquifers. PNNL-11720. Richland, Wash.: Pacific Ground Water 39(4): 534-545.
Northwest National Laboratory. Available at: http://bioprocess. Konikow, L. F. 2011. The secret to successful solute-transport
pnnl.gov/rt3d.publications.htm. Accessed 8 January 2012. modeling. Ground Water 49(2):144-159.
Doherty, J. 2010. PEST User’s Guide. Bethesda, Md.: S.S. Konikow, L. F., and J. D. Bredehoeft. 1992. Ground-water models
Papadopulos and Associates, Inc. cannot be validated. Adv. Water Resour. 15(1): 75-83.
Feehley, C. E., C. Zheng, and F. J. Molz. 2000. A dual-domain Langevin, C. D., D. T. Thorne Jr., A. M. Dausman, M. C. Sukop,
mass transfer approach for modeling solute transport in and W. Guo. 2007. SEAWAT version 4: A computer program
heterogeneous porous media: Application to the MADE site. for simulation of multi-species solute and heat transport.
Water Resour. Res. 36(9): 2501-2515. USGS Techniques and Methods, Book 6, Chapter A22. Reston,
Foglia, L., S. W. Mehl, M. C. Hill, P. Perona, and P. Burlando. Va.: U.S. Geological Survey. Available at:
2007. Testing alternative ground water models using cross- http://water.usgs.gov/ogw/seawat/. Accessed 8 January 2012.
validation and other methods. Ground Water 45(5): 627-641. Liu, G., C. Zheng, G. R. Tick, and S. M. Gorelick. 2010. Relative
Foglia, L., M. C. Hill, S. W. Mehl, and P. Burlando. 2009. importance of dispersion and rate-limited mass transfer in
Sensitivity analysis, calibration, and testing of a distributed highly heterogeneous porous media: Analysis of a new tracer
hydrological model using error-based weighting and one test at the macrodispersion experiment (MADE) site. Water
objective function. Water Resour. Res. 45: W06427, doi: Resour. Res. 46: W03524, doi: 10.1029/2009WR008430.
10.1029/2008WR007255. Lu, D., M. C. Hill, and M. Ye. 2012. Analysis of regression
Foglia, L., S. W. Mehl., M. C. Hill, and P. Burlando. 2012. confidence intervals and Bayesian credible intervals for
Analysis of model discrimination techniques: The case of the uncertainty quantification. Water Resour. Res. (in press).
Maggia Valley, southern Switzerland. Water Resour. Res. (in Lu, G., T. P. Clement, C. Zheng, and T. H. Wiedemeier. 1999.
review). Natural attenuation of BTEX compounds: Model development
Goode, D. J. 1996. Direct simulation of groundwater age. Water and field-scale application. Ground Water 37(5): 707-717.
Resour. Res. 32(2): 289-296. Ma, R., C. Zheng, H. Prommer, J. Greskowiak, C. Liu, J. Zachara,
Greskowiak, J., H. Prommer, C. Liu, V. E. A. Post, R. Ma, C. and M. Rockhold. 2010. A field-scale reactive transport model
Zheng, and J. M. Zachara. 2010. Comparison of parameter for U(VI) migration influenced by coupled multi-rate mass
sensitivities between a laboratory and field-scale model of transfer and surface complexation reactions. Water Resour.
uranium transport in a dual-domain, distributed-rate reactive Res. 46: W05509, doi: 10.1029/2009WR008168.
system. Water Resour. Res. 46: W09509, doi: 10.1029/ Ma, R., C. Zheng, M. Tonkin, and J. M. Zachara. 2011.
2009WR008781. Importance of considering intraborehole flow in solute
Halford, K. J., and R. T. Hanson. 2002. User guide for the transport modeling under highly dynamic flow conditions. J.
drawdown-limited, multimode well (MNW) package for the Contam. Hydrol. 123(1-2): 11-19.
U.S. Geological Survey’s modular three-dimensional finite- Matotta, L. S., and A. J. Rabideaub. 2008. Calibration of complex
difference ground-water flow model, versions MODFLOW-96 subsurface reaction models using a surrogate-model approach.
and MODFLOW-2000. USGS Open-File Report 02-293. Advances in Water Resources 31(12): 1697-1707.
Reston, Va.: U.S. Geological Survey. McDonald, M. G., and A. W. Harbaugh. 1988. A modular three-
Harbaugh, A. W. 2005. MODFLOW-2005, the U.S. Geological dimensional finite-difference ground-water flow model. USGS
Survey modular ground-water model: The ground-water flow Techniques of Water-Resources Investigations, Book 6,
process. USGS Techniques and Methods 6-A16. Reston, Va.: Chapter A1. Reston, Va.: U.S. Geological Survey.
U.S. Geological Survey. Mehl, S. W., and M. C. Hill. 2001. A comparison of solute-
Harbaugh, A. W., and M. G. McDonald. 1996. User’s transport solution techniques and their effect on sensitivity
documentation for MODFLOW-96, an update to the U.S. analysis and inverse modeling results. Ground Water 39(2):
Geological Survey modular finite-difference ground-water 300-307.
flow model. USGS Open-File Report 96-485. Reston, Va.: Mehl, S. W., M. C. Hill, and S. A. Leake. 2006. Comparison of
U.S. Geological Survey. local grid refinement methods for MODFLOW. Ground Water
Harbaugh, A. W., E. R. Banta, M. C. Hill, and M. G. McDonald. 44(6): 792-796.
2000. MODFLOW-2000, The U.S. Geological Survey Newcomer, D. R., B. N. Bjornstad, and V. R. Vermeul. 2010.
modular ground-water model: User guide to modularization Vertical wellbore flow monitoring for assessing spatial and
concepts and the ground-water flow process. Reston, Va.: U.S. temporal flow relationships with a dynamic river boundary.
Geological Survey. Ground Water Monit. and Remed. 30(4): 123-135.
Hill, M. C. 1998. Methods and guidelines for effective model Niswonger, R. G., D. E. Prudic, and R. S. Regan. 2006.
calibration. USGS Water-Resources Investigations Report 98- Documentation of the unsaturated-zone flow (UZF1) package
4005. Reston, Va.: U.S. Geological Survey. for modeling unsaturated flow between the land surface and
Hill, M. C. 2006. The practical use of simplicity in developing the water table with MODFLOW-2005. USGS Techniques and
ground water models. Ground Water 44(6): 775-781. Methods 6-A19. Reston, Va.: U.S. Geological Survey.

1558 TRANSACTIONS OF THE ASABE

 NRC. 2007. Models in environmental regulatory decision making. Wang, M., and C. Zheng. 1997. Optimal remediation policy
Washington, D.C.: National Research Council, Committee on selection under general conditions. Ground Water 35(5): 757-
Models in the Regulatory Decision Process. 764.
Parkhurst, D. L., and C. A. J. Appelo. 1999. User’s guide to Ye, M., P. D. Meyer, and S. P. Neuman. 2008. On model selection
PHREEQC (version 2): A computer program for speciation, criteria in multimodel analysis. Water Resour. Res. 44:
batch-reaction, one-dimensional transport, and inverse W03428, doi: 10.1029/2008WR006803.
geochemical calculations. USGS Water-Resources Zheng, C. 1990. MT3D: A modular three-dimensional transport
Investigations Report 99-4259. Reston, Va.: U.S. Geological model for simulation of advection, dispersion, and chemical
Survey. reactions of contaminants in groundwater systems. Report to
Poeter, E. P., and M. C. Hill. 2007. MMA: A computer code for the U.S. Environmental Protection Agency. Available at:
multi-model analysis. USGS Techniques and Methods 6-E3. www.epa.gov/ada/csmos.html.
Reston, Va.: U.S. Geological Survey. Available at: Zheng, C. 1993. Extension of the method of characteristics for
http://typhoon.mines.edu/freeware/mma/. simulation of solute transport in three dimensions. Ground
Poeter, E. P., M. C. Hill, E. R. Banta, S. Mehl, and S. Christensen. Water 31(3): 456-465.
2005. UCODE_2005 and six other computer codes for Zheng, C. 2009. Recent developments and future directions for
universal sensitivity analysis, calibration, and uncertainty MT3DMS and related transport codes. Ground Water 47(5):
evaluation. USGS Techniques and Methods 6-A11. Reston, 620-625.
Va.: U.S. Geological Survey. Available at: http://typhoon. Zheng, C. 2010. MT3DMS 5.3 supplemental user’s guide.
mines.edu/freeware/ucode/. Tuscaloosa, Ala.: University of Alabama, Department of
Prommer, H., D. A. Barry, and C. Zheng. 2003. Geological Sciences. Available at: http://hydro.geo.ua.
MODFLOW/MT3DMS-based multicomponent reactive edu/mt3d/.
transport model. Ground Water 41(2): 247-257. Zheng, C. 2011. Reflections: 2002-2009. Ground Water 49(2):
Renard, B., D. Kavetski, E. Leblois, M. Thyer, G. Kuczera, and S. 129-132.
W. Franks. 2011. Toward a reliable decomposition of Zheng, C., and G. D. Bennett. 2002. Applied Contaminant
predictive uncertainty in hydrological modeling: Transport Modeling. 2nd ed. New York, N.Y.: John Wiley and
Characterizing rainfall errors using conditional simulation. Sons.
Water Resour. Res. 47: W11516, doi: 10.1029/2011WR010643. Zheng, C., and P. P. Wang. 1999. MT3DMS: A modular three-
Sanford, W. E. 1997. Correcting for diffusion in carbon-14 dating dimensional multi-species transport model for simulation of
of ground water. Ground Water 35: 357-361. advection, dispersion, and chemical reactions of contaminants
Sun, N.-Z. 1994. Inverse Problems in Groundwater Modeling. in groundwater systems. Documentation and User’s Guide,
Boston. Mass.: Kluwer Academic. Contract Report SERDP-99-1. Vicksburg, Miss.: U.S. Army
Tonkin, M., C. R. Tiedeman, D. M. Ely, and M. C. Hill. 2007. Engineer Research and Development Center. Available at:
OPR-PPR: A computer program for assessing data importance http://hydro.geo.ua.edu/mt3d/.
to model predictions using linear statistics. USGS Techniques Zheng, C., and P. P. Wang. 2002. A field demonstration of the
and Methods 6-E2. Reston, Va.: U.S. Geological Survey. simulation-optimization approach for remediation system
Available at: http://water.usgs.gov/software/OPR-PPR.html. design. Ground Water 40(3): 258-265.
Waddill, D. W., and M. A. Widdowson. 1998. A three-dimensional Zheng, C., M. Bianchi, and S. M. Gorelick. 2011. Lessons learned
model for subsurface transport and biodegradation. ASCE J. from 25 years of research at the MADE site. Ground Water
Environ. Eng. 124(4): 336-344. 49(5): 649-662.

55(4): 1549-1559 1559

V i e w p u b l i c a t i o n s t a t s