Geosci. Model Dev.

, 12, 4955–4997, 2019

https://doi.org/10.5194/gmd-12-4955-2019
© Author(s) 2019. This work is distributed under
the Creative Commons Attribution 4.0 License.

The Lagrangian particle dispersion model FLEXPART version 10.4

Ignacio Pisso1 , Espen Sollum1 , Henrik Grythe1 , Nina I. Kristiansen1,a , Massimo Cassiani1 , Sabine Eckhardt1 ,
Delia Arnold2,3 , Don Morton4 , Rona L. Thompson1 , Christine D. Groot Zwaaftink1 , Nikolaos Evangeliou1 ,
Harald Sodemann5 , Leopold Haimberger6 , Stephan Henne7 , Dominik Brunner7 , John F. Burkhart8 , Anne Fouilloux8 ,
Jerome Brioude9 , Anne Philipp6,10 , Petra Seibert11 , and Andreas Stohl1
1 Norwegian Institute for Air Research (NILU), Kjeller, Norway
2 Central Institute for Meteorology and Geodynamics (ZAMG), Vienna, Austria
3 Arnold Scientific Consulting, Manresa, Spain
4 Boreal Scientific Computing, Fairbanks, Alaska, USA
5 Geophysical Institute, University of Bergen and Bjerknes Centre for Climate Research, Bergen, Norway
6 Department of Meteorology and Geophysics, University of Vienna, Vienna, Austria
7 Empa, Swiss Federal Laboratories for Materials Science and Technology, Dübendorf, Switzerland
8 Department of Geosciences, University of Oslo, Oslo, Norway
9 Laboratoire de l’Atmosphère et des Cyclones (LACy), UMR8105, Université de la Réunion – CNRS – Météo-France,

Saint-Denis de La Réunion, France

10 Aerosol Physics & Environmental Physics, University of Vienna, Vienna, Austria
11 Institute of Meteorology, University of Natural Resources and Life Sciences, Vienna, Austria
a now at: Met Office, FitzRoy Road, Exeter, EX1 3PB, UK

Correspondence: Ignacio Pisso ([email protected])

Received: 21 December 2018 – Discussion started: 28 January 2019

Revised: 25 July 2019 – Accepted: 7 August 2019 – Published: 2 December 2019

Abstract. The Lagrangian particle dispersion model FLEX- ters of Environmental Prediction (NCEP) Global Forecast
PART in its original version in the mid-1990s was designed System (GFS). Since the last publication of a detailed FLEX-
for calculating the long-range and mesoscale dispersion of PART description (version 6.2), the model has been im-
hazardous substances from point sources, such as those re- proved in different aspects such as performance, physico-
leased after an accident in a nuclear power plant. Over the chemical parameterizations, input/output formats, and avail-
past decades, the model has evolved into a comprehensive able preprocessing and post-processing software. The model
tool for multi-scale atmospheric transport modeling and anal- code has also been parallelized using the Message Passing
ysis and has attracted a global user community. Its applica- Interface (MPI). We demonstrate that the model scales well
tion fields have been extended to a large range of atmospheric up to using 256 processors, with a parallel efficiency greater
gases and aerosols, e.g., greenhouse gases, short-lived cli- than 75 % for up to 64 processes on multiple nodes in runs
mate forcers like black carbon and volcanic ash, and it has with very large numbers of particles. The deviation from
also been used to study the atmospheric branch of the wa- 100 % efficiency is almost entirely due to the remaining non-
ter cycle. Given suitable meteorological input data, it can be parallelized parts of the code, suggesting large potential for
used for scales from dozens of meters to global. In particu- further speedup. A new turbulence scheme for the convective
lar, inverse modeling based on source–receptor relationships boundary layer has been developed that considers the skew-
from FLEXPART has become widely used. In this paper, we ness in the vertical velocity distribution (updrafts and down-
present FLEXPART version 10.4, which works with meteo- drafts) and vertical gradients in air density. FLEXPART is
rological input data from the European Centre for Medium- the only model available considering both effects, making it
Range Weather Forecasts (ECMWF) Integrated Forecast highly accurate for small-scale applications, e.g., to quantify
System (IFS) and data from the United States National Cen- dispersion in the vicinity of a point source. The wet deposi-

Published by Copernicus Publications on behalf of the European Geosciences Union.

4956 I. Pisso et al.: FLEXPART version 10.4

tion scheme for aerosols has been completely rewritten and sity. An important characteristic of LPDMs is their ability
a new, more detailed gravitational settling parameterization to run backward in time in a framework that is theoretically
for aerosols has also been implemented. FLEXPART has had consistent with both the Eulerian flow field and LPDM for-
the option of running backward in time from atmospheric ward calculations. This was discussed by Thomson (1987,
concentrations at receptor locations for many years, but this 1990), further developed by Flesch et al. (1995), and ex-
has now been extended to also work for deposition values tended to global-scale dispersion by Stohl et al. (2003) and
and may become useful, for instance, for the interpretation Seibert and Frank (2004). The more practical aspects and ef-
of ice core measurements. To our knowledge, to date FLEX- ficiency of LPDMs were discussed by Zannetti (1992) and
PART is the only model with that capability. Furthermore, the Uliasz (1994). A history of their development was provided
temporal variation and temperature dependence of chemical by Thomson and Wilson (2013).
reactions with the OH radical have been included, allowing Lagrangian models exhibit much less numerical diffu-
for more accurate simulations for species with intermediate sion than Eulerian or semi-Lagrangian models (e.g., Reith-
lifetimes against the reaction with OH, such as ethane. Fi- meier and Sausen, 2002; Cassiani et al., 2016), even though
nally, user settings can now be specified in a more flexible some numerical errors also arise in the discretization of their
namelist format, and output files can be produced in NetCDF stochastic differential equations (Ramli and Esler, 2016).
format instead of FLEXPART’s customary binary format. In Due to their low level of numerical diffusion, tracer filaments
this paper, we describe these new developments. Moreover, generated by dispersion in the atmosphere (Ottino, 1989) are
we present some tools for the preparation of the meteorolog- much better captured in Lagrangian models than in Eulerian
ical input data and for processing FLEXPART output data, models. It has been noticed, for instance, that Eulerian mod-
and we briefly report on alternative FLEXPART versions. els have difficulties simulating the fine tracer structures cre-
ated by intercontinental pollution transport (Rastigejev et al.,
2010), while these are well preserved in LPDMs (e.g., Stohl
et al., 2003). Furthermore, in Eulerian models a tracer re-
1 Introduction leased from a point source is instantaneously mixed within
a grid box, whereas Lagrangian models are independent of a
Multi-scale offline Lagrangian particle dispersion models computational grid and can account for point or line sources
(LPDMs) are versatile tools for simulating the transport and with potentially infinitesimal spatial resolution. When com-
turbulent mixing of gases and aerosols in the atmosphere. bined with their capability to run backward in time, this
Examples of such models are the Numerical Atmospheric- means that LPDMs can also be used to investigate the his-
dispersion Modelling Environment (NAME) (Jones et al., tory of air parcels affecting, for instance, an atmospheric
2007), the Stochastic Time-Inverted Lagrangian Transport measurement site (e.g., for in situ monitoring of atmospheric
(STILT) model (Lin et al., 2003), the Hybrid Single-Particle composition).
Lagrangian Integrated Trajectory (HYSPLIT) model (Stein The computational efficiency of LPDMs depends on the
et al., 2015) and the FLEXible PARTicle (FLEXPART) type of application. One important aspect is that their com-
model (Stohl et al., 1998, 2005). LPDMs are stochastic mod- putational cost does not increase substantially with the num-
els that compute trajectories for a large number of notional ber of species transported (excluding aerosol particles with
particles that do not represent real aerosol particles but points different gravitational settling, for which trajectories deviate
moving with the ambient flow. The trajectories represent from each other), making multispecies simulations efficient.
the transport by mean flow as well as turbulent, diffusive On the other hand, the computational time scales linearly
transport by unresolved parameterized subgrid-scale trans- with the number of particles used, while the statistical error
port processes (e.g., turbulence, meandering, deep convec- in the model output decreases only with the square root of the
tion, etc.) and can also include gravitational settling. Each particle density. Thus, it can be computationally costly to re-
particle carries a certain mass, which can be affected by loss duce statistical errors, and data input/output can require sub-
processes such as radioactive decay, chemical loss, or dry and stantial additional resources. Generally, a high particle den-
wet deposition. sity can be achieved with a small number of released parti-
The theoretical basis for most currently used atmospheric cles in the vicinity of a release location, where statistical er-
particle models was laid down by Thomson (1987). He in- rors, relative to simulated concentrations, are typically small.
troduced the criterion to formulate Lagrangian stochastic However, particle density and thus the relative accuracy of
models that produce particle trajectories consistent with pre- the results decrease with distance from the source. Methods
defined Eulerian probability density functions in physical should therefore be used to reduce the statistical error (e.g.,
and velocity space. Rodean (1996) and Wilson and Sawford Heinz et al., 2003), such as kernels or particle splitting, and
(1996) provided detailed descriptions of the theory and for- it is important to quantify the statistical error.
mulation of LPDMs in constant density flows and under dif-
ferent atmospheric stability conditions. Stohl and Thomson
(1999) extended this to flows with vertically variable air den-

Geosci. Model Dev., 12, 4955–4997, 2019 www.geosci-model-dev.net/12/4955/2019/

I. Pisso et al.: FLEXPART version 10.4 4957

1.1 The Lagrangian particle dispersion model models. For the model version described here, v10.4, data
FLEXPART from the European Centre for Medium-Range Weather Fore-
casts (ECMWF) Integrated Forecast System (IFS) and data
One of the most widely used LPDMs is the open-source from the United States National Centers of Environmental
model FLEXPART, which simulates the transport, diffusion, Prediction (NCEP) Global Forecast System (GFS) can be
dry and wet deposition, radioactive decay, and 1st-order used. Common spatial resolutions for IFS depending on the
chemical reactions (e.g., OH oxidation) of tracers released application include 1◦ ×1◦ at 3 h (standard for older products,
from point, line, area or volume sources, or filling the whole e.g., ERA-Interim), 0.5◦ × 0.5◦ at 1 h (standard for newer
atmosphere (Stohl et al., 1998, 2005). FLEXPART develop- products, e.g., ERA5) and 0.1◦ ×0.1◦ at 1 h (current ECMWF
ment started more than 2 decades ago (Stohl et al., 1998) operational data). The ECMWF IFS model currently has
and the model has been free software ever since it was first 137 vertical levels. NCEP GFS input files are usually used
released. The status as a free software is formally estab- at 1◦ × 1◦ horizontal resolution, with 64 vertical levels and
lished by releasing the code under the GNU General Public 3 h time resolution. NCEP GFS input files are also avail-
License (GPL) Version 3. However, the last peer-reviewed able at 0.5◦ × 0.5◦ and 0.25◦ × 0.25◦ horizontal resolution.
publication describing FLEXPART (version 6.2) was pub- Other FLEXPART model branches have been developed for
lished as a technical note about 14 years ago (Stohl et al., input data from various limited-area models, for example
2005). Since then, while updates of FLEXPART’s source the Weather Research and Forecasting (WRF) meteorolog-
code and a manual were made available from the web page at ical model (Brioude et al., 2013) and the Consortium for
https://flexpart.eu/ (last access: 30 October 2019), no citable Small-scale Modeling (COSMO) model (Oney, 2015), which
reference was provided. In this paper, we describe FLEX- extend the applicability of FLEXPART down to the meso-
PART developments since Stohl et al. (2005), which led to gamma scale. Notice that the turbulence parameterizations of
the current version 10.4 (subsequently abbreviated as v10.4). FLEXPART are valid at even smaller scales. Another FLEX-
FLEXPART can be run either forward or backward in PART model version, FLEXPART–NorESM/CAM (Cassiani
time. For forward simulations, particles are released from et al., 2016), uses the meteorological output data generated
one or more sources and concentrations (or mixing ratios) by the Norwegian Earth System Model (NorESM1-M) with
are determined on a regular latitude–longitude–altitude grid. its atmospheric component CAM (Community Atmosphere
In backward mode, the location where particles are released Model). The current paper does not document these other
represents a receptor (e.g., a measurement site). Like in the model branches, but most share many features with FLEX-
forward mode, particles are sampled on a latitude–longitude– PART v10.4 and some are briefly described in Appendix C.
altitude grid, which in this case corresponds to potential A key aspect of these model branches is the ability to read
sources. The functional values obtained represent the source– meteorological input other than that from ECMWF or NCEP.
receptor relationship (SRR) (Seibert and Frank, 2004), also
called source–receptor sensitivity (Wotawa et al., 2003) or 1.2 FLEXPART and its history
simply emission sensitivity, and are related to the particles’
residence time in the output grid cells. Backward modeling FLEXPART’s first version (v1) was a further development
is more efficient than forward modeling for calculating SRRs of the trajectory model FLEXTRA (Stohl et al., 1995) and
if the number of receptors is smaller than the number of (po- was coded in Fortran 77. It provided gridded output of con-
tential) sources. Seibert and Frank (2004) explained in de- centrations of chemical species and radionuclides. Its mete-
tail the theory of backward modeling, and Stohl et al. (2003) orological input data were based on the ECMWF-specific
gave a concrete backward modeling example. FLEXPART GRIB-1 (gridded binary) format. The model was first ap-
can also be used in a domain-filling mode whereby the entire plied in an extensive validation study using measurements
atmosphere is represented by (e.g., a few million) particles from three large-scale tracer experiments (Stohl et al., 1998).
of equal mass (Stohl and James, 2004). The number of parti- A deposition module was added in version 2. Version 3 saw
cles required for domain-filling simulations, not unlike those improvements in performance and the addition of a subgrid-
needed for other types of simulations, depends on the scien- scale terrain effects parameterization. In v3.1 the output for-
tific question to be answered. For instance, a few million par- mat was optimized (sparse matrix) and mixing ratio output
ticles distributed globally are often enough to investigate the could optionally be produced. It also allowed for the output
statistical properties of air mass transport (e.g., monthly aver- of particle positions. Furthermore, a density correction was
age residence times in a particular area that is not too small) added to account for decreasing air density with height in
but would not be enough for a case study of airstreams re- the boundary layer (Stohl and Thomson, 1999). Further v3
lated to a particular synoptic situation (e.g., describing flow releases included the addition of a convection scheme (Seib-
in the warm conveyor belt of a particular cyclone). ert et al., 2001) based on Emanuel and Živković-Rothman
FLEXPART is an offline model that uses meteorological (1999), the option to calculate mass fluxes across grid cell
fields (analyses or forecasts) as input. Such data are available faces and age spectra, and free format input (v3.2). The pre-
from several different numerical weather prediction (NWP) liminary convection scheme of v3.2 was revised in v4 (see

www.geosci-model-dev.net/12/4955/2019/ Geosci. Model Dev., 12, 4955–4997, 2019

4958 I. Pisso et al.: FLEXPART version 10.4

Forster et al., 2007). In v5 the output unit of backward calcu- space for inverse modeling applications was also introduced
lations was changed to seconds and improvements in the in- (Thompson and Stohl, 2014) (see Sect. 2.6).
put/output handling were made. Comprehensive validation of
these early FLEXPART versions was done during interconti- 1.3 FLEXPART version 10.4
nental air pollution transport studies at the end of the 1990s
and early 2000s (Stohl and Trickl, 1999; Forster et al., 2001; For v10.4 of FLEXPART, described in this paper, several
Spichtinger et al., 2001; Stohl et al., 2002, 2003). Special de- more changes and improvements were made. First, an op-
velopments were also made in order to extend FLEXPART’s tional new scheme applying more realistic skewed rather than
forecasting capabilities for large-scale field campaigns (Stohl Gaussian turbulence statistics in the convective atmospheric
et al., 2004). Version 6.0 saw corrections to the numerics boundary layer (ABL) was developed (Sect. 2.1). Second, the
in the convection scheme, the addition of a domain-filling wet deposition scheme for aerosols was totally revised (Gry-
option used, for instance, in water cycle studies (Stohl and the et al., 2017), introducing dependencies on aerosol size,
James, 2004) and the possibility to use nested output. Version precipitation type (rain or snow), and distinguishing between
6.2, which added the ability to model sources and receptors in in-cloud and below-cloud scavenging (see Sect. 2.4). The
both mass and mixing ratio units (Seibert and Frank, 2004), is code now also allows for the reading of three-dimensional (3-
currently the last version described in a publication (Stohl et D) cloud water fields from meteorological input files. Third,
al., 2005). A separate sister model branch (v6.4) was adapted a method to calculate the sensitivity of deposited quantities to
to run with NCEP GFS meteorological input data. The cur- sources in backward mode was developed (Sect. 2.5) Fourth,
rent paper describes the most important model developments chemical reactions with the hydroxyl radical (OH) are now
since v6.2 (for ECMWF) and v6.4 (for GFS). made dependent on the temperature and vary sub-monthly
Version 8.0 unified the model branches based on ECMWF (Sect. 2.7). Fifth, large parts of the code were parallelized us-
IFS and NCEP GFS input data in one source package but ing the Message Passing Interface (MPI), thus facilitating a
still required the building of two different executables. Im- substantial speedup for certain applications (see Sect. 3), and
portantly, Fortran 90 constructs were introduced in parts of the code was unified so that a single executable can now use
the code, such as initial support for dynamic memory allo- both ECMWF and GFS input data. Sixth, a dust mobiliza-
cation. Furthermore, a global land use inventory was added, tion scheme that can be run as a FLEXPART preprocessor
allowing for more accurate dry deposition calculations ev- was developed (Sect. 2.8). Seventh, the software used to re-
erywhere on the globe (before, land use data were provided trieve data from the ECMWF has been modernized and can
only for Europe). The reading of the – at the time – newly in- now also be used by scientists from non-ECMWF member
troduced GRIB-2 format with the ECMWF grib_api library states (Sect. 5.2.1). Finally, a testing environment was cre-
was implemented in v8.2. An option to calculate the sensi- ated that allows users to verify their FLEXPART installation
tivity to initial conditions in backward model runs (in addi- and compare results (Sect. 7).
tion to the emission sensitivity calculations) was also imple- Despite the many changes and additions, in large part the
mented in v8.2. Version 8 was also the first version that dis- operation of FLEXPART v10.4 still resembles the original
tinguished between in-cloud and below-cloud scavenging for version 1 design. Throughout this paper, we avoid repeating
washout, relying on simple diagnostics for clouds based on information on aspects of the model that have not changed
grid-scale relative humidity. With a growing number of pa- since earlier model descriptions. The paper should there-
rameters defining removal processes, each species was given fore always be considered together with the publications of
its own definition file, whereas in previous versions the prop- Stohl et al. (1998, 2005). To provide the necessary context
erties for all species were contained in a single file. The grav- for the rest of this paper, we provide a brief overview of the
itational settling scheme was improved in v8.2.1 (Stohl et FLEXPART v10.4 directory structure in Table 1. The source
al., 2011), and this is briefly described in this paper in sec- code is contained in directory src. The pathnames of the in-
tion 2.3. put and output directories are stated in the file pathnames
For v9, the code was transformed to the Fortran 90 free- read by the FLEXPART executable. The directory options
form source format. The option to read the run specifica- contains the parameters that define a run in files such as
tions from Fortran namelists instead of the standard input COMMAND (e.g., start and end times of the simulation, out-
files was introduced, as described in Sect. 5 of this paper. put frequency, etc.), RELEASES (definition of the particle
This change was motivated by the resulting greater flexibil- releases), OUTGRID (output grid specifications) and others.
ity, in particular with regard to setting default values, optional All the output is written in a directory unique for each run.
arguments, when new developments require adding new pa- There are also other directories containing the model testing
rameters and when specifying parameter lists. In addition, an environment and example runs, as well as preprocessing and
option to produce output in compressed NetCDF 4 format post-processing software (see Table 1).
was provided (see Sect. 6.3). Another option to write some Sensu stricto FLEXPART consists of the (Fortran) source
model output only for the first vertical layer to save storage files required to build an executable, not including external
libraries such as those needed for input reading. The make-

Geosci. Model Dev., 12, 4955–4997, 2019 www.geosci-model-dev.net/12/4955/2019/

I. Pisso et al.: FLEXPART version 10.4 4959

files and the sample input as provided in the “options” may

also be included under this term. However, in order to do real
work with FLEXPART, one also needs to obtain meteorolog-
ical input data (in the case of the ECMWF this is not trivial,
so the flex_extract package is provided), and one needs
to do post-processing. This is the reason why we include a
selection of such tools here.

2 Updates of the model physics and chemistry

This section gives an overview of the main updates of the

model physics and chemistry since the last published FLEX-
PART version, v6.2 (Stohl et al., 2005). Some developments
have been published already separately, and in such cases
we keep the description short, focusing on technical aspects
of the implementation in FLEXPART that are important for
model users or demonstrating applications not covered in the
original papers.

2.1 Boundary layer turbulence

Subgrid-scale atmospheric motions unresolved by the me- Figure 1. Comparison of FLEXPART results obtained with the
teorological input data need to be parameterized in FLEX- skewed turbulence parameterization (a) and with the Gaussian tur-
PART. This is done by adding stochastic fluctuations based bulence parameterization (b). Shown are the tracer concentrations
on Langevin equations for the particle velocity components integrated over all latitudes as a function of altitude and longitude.
(Stohl et al., 2005). In the ABL, the stochastic differential The simulations used a point source emitting 100 kg of tracer per
equations are formulated according to the well-mixed cri- hour for a period starting on 1 July 2017 at 12:00 UTC and end-
teria proposed by Thomson (1987). Until FLEXPART ver- ing at 13:30 UTC. The source was located near Vienna (Austria) at
sion 9.2, the Eulerian probability density functions (PDFs) 47.9157◦ N and 16.3274◦ E, 250 m above ground level. Results are
for the three velocity components were assumed to be three averaged for the time period 12:40 to 13:00 UTC. Notice that the
maximum ground-level concentration in panel (a) (ca. 7 mg m−2 )
independent Gaussian PDFs. However, for the vertical veloc-
is about 30 % higher than in panel (b) (5 mg m−2 ).
ity component, the Gaussian turbulence model is well suited
only for stable and neutral conditions. In the convective ABL
(CBL), turbulence is skewed since a larger area is occupied
by downdrafts than by updrafts (e.g., Stull, 1988; Luhar and the standard Gaussian formulation, the CBL option is much
Britter, 1989). In such conditions, the Gaussian turbulence more computationally demanding and not recommended for
model is not appropriate for sources within the ABL, as it large-scale applications. However, for studies of tracer dis-
cannot reproduce the observed upward bending of plumes persion in the vicinity of individual point sources, the CBL
from near-ground sources or the rapid downward transport option is essential to reproduce the characteristic features of
of plumes from elevated sources (Venkatram and Wyngaard, CBL dispersion (Weil, 1985), while the additional computa-
1988). However, the Gaussian approximation has negligi- tional burden remains tolerable.
ble influence once the tracer is mixed throughout the whole Figure 1 shows a comparison between two simulations
ABL. of dispersion from an elevated source, with the skewed and
Cassiani et al. (2015) developed an alternative Langevin with the Gaussian turbulence model. It can be seen that
equation model for the vertical particle velocity includ- the maximum time-averaged ground-level concentration is
ing both skewed turbulence and a vertical density gradi- about 30 % higher for the skewed turbulence parameteriza-
ent, which is now implemented in FLEXPART v10.4. This tion. This is the result of the plume centerline tilting down-
scheme can be activated by setting the switch CBL to 1 ward to the surface in the vicinity of the source for the
in the file COMMAND. In this case, the time step require- skewed turbulence case due to downdrafts being more fre-
ment for numerical stability is much more stringent than for quent than updrafts. The plume also spreads faster in this
the standard Gaussian turbulence model (typically, values of case. These results are similar to those obtained by others
CTL=10 and IFINE=10 are required, also set in the file (e.g., Luhar and Britter, 1989).
COMMAND). Therefore, also considering that the computation It is important to note that the CBL formulation smoothly
time required for each time step is about 2.5 times that of transits to a standard Gaussian formulation when the stabil-

www.geosci-model-dev.net/12/4955/2019/ Geosci. Model Dev., 12, 4955–4997, 2019

4960 I. Pisso et al.: FLEXPART version 10.4

Table 1. Directory structure overview of the FLEXPART v10.4 software distribution. All listed directories are subdirectories of the installa-
tion root directory $flexhome/.

Subdirectory or file Contents Description and comments

pathnames $options/ path to options directory
(file) $output/ path to output directory
$flex_winds/ path to meteorological input files
$AVAILABLE path to AVAILABLE file
$flexhome/

src/ *.f90 Fortran source files

makefile see Sect. 4 and Appendix A
FLEXPART executable file (see Sect. 4)
options/ COMMAND, RELEASES, OUTGRID, SPECIES, user input files
AGECLASSES, OUTGRID_NEST, RECEPTORS, see Sect. 5 and Table 7
IGBP_int1.dat, surfdata.t, surfdepo.t, OH_variables.bin
AVAILABLE list of meteorological input data files file containing list, see Sect. 5
output/ FLEXPART output files see Sect. 6 and Table 11
preprocess/ flex_extract/ see Sect. 5.2
postprocess/ read_flex_fortran/, read_flex_matlab/, see Sect. 6.4
tests/ development tests for FLEXPART and ancillary software see Sect. 7
tests/examples/ example runs illustrating various FLEXPART functionalities and Appendix C

ity changes towards neutral (Cassiani et al., 2015). However, to the value of the horizontal diffusivity Dh by the square
the actual equation solved inside the model for the Gaussian of the typical atmospheric aspect ratio for tracer structures
condition is still different from the standard version as actual κ ≈ 100–250 (Haynes and Anglade, 1997).
particle velocities rather than the scaled ones are advanced FLEXPART uses by default a constant vertical diffusiv-
(see, e.g., Wilson et al., 1981; Rodean, 1996). Full details ity Dz = 0.1 m2 s−1 in the stratosphere, following Legras et
of the CBL implementation can be found in Cassiani et al. al. (2003), whereas a horizontal diffusivity Dh = 50 m2 s−1
(2015). is used in the free troposphere. In general in the atmosphere,
To date, FLEXPART has mainly been used for large-scale the values of the turbulent diffusivity depend on time and lo-
applications. With this new CBL option, FLEXPART is now cation, showing in particular seasonal, latitudinal and altitude
also well suited for the simulation of small-scale tracer dis- variability: e.g., Dv = 10−2 m2 s−1 in the stratosphere (Bal-
persion or for the inverse modeling of point source emissions luch and Haynes, 1997) and Dh = 104 m2 s−1 (Pisso et al.,
from near-field measurements – at least if the resolved winds 2009) in the troposphere. The values can be modified by the
are representative of the situation considered. In fact, to our user in the COMMAND file (namelist variables d_trop and
knowledge FLEXPART is the only particle model consider- d_strat, in m2 s−1 ). As mentioned above, Dh ≈ κ 2 Dz ,
ing both skewness in the vertical velocity distribution and and therefore both values can be used interchangeably.
vertical gradients in air density. Both these effects are par- In FLEXPART version 6.2, the stratosphere and tropo-
ticularly important in deep CBLs and can be additive with sphere were distinguished based on a threshold of 2 pvu (po-
respect to simulated ground-level concentrations. tential vorticity units), with a maximal height of 18 km in
the tropics and a minimal height of 5 km elsewhere. Such a
2.2 Turbulent diffusivity above the boundary layer threshold is well suited to midlatitudes, but it can differ from
the thermal tropopause in the polar regions and close to the
Above the atmospheric boundary layer, turbulent fluctuations Equator. In FLEXPART 10.4, the thermal tropopause defini-
can be represented with a turbulent diffusivity. The value of tion is used and is calculated using the lapse rate definition
the diffusivity tensor controls the size and lifetimes of the (Hoinka, 1997).
filamentary structures caused by advection.qDiffusivities are
converted into velocity scales using σvi = 2D i
dt , where i is
2.3 Gravitational settling
the direction. This
√ corresponds to a mean diffusive displace-
ment of σxi = 2Di dt, characteristic of Brownian motion. Gravitational settling of aerosols is implemented in FLEX-
For i, only the vertical (v) and horizontal (h) directions are PART as a process that changes the particle trajectories. The
considered. The value of the vertical diffusivity Dz is related settling velocity is determined at each time step and added

Geosci. Model Dev., 12, 4955–4997, 2019 www.geosci-model-dev.net/12/4955/2019/

I. Pisso et al.: FLEXPART version 10.4 4961

to the vertical wind velocity. In simulations in which a parti- cloud scavenging and also has different parameterizations
cle represents several species, all species are transported with of 3 for gases and particles. For the latter, it also dis-
the settling velocity of the first species. If this is not intended, tinguishes between liquid-phase and ice-phase states. This
simulations for the different species must be run separately. yields in total six different parameterizations for 3, de-
Gravitational settling velocities are also used in the calcula- scribed in Sect. 2.4.2 and 2.4.3.
tion of dry deposition. In the third step, the removal of particle mass due to wet
In older FLEXPART versions, gravitational settling was deposition is calculated. It takes the form of an exponential
calculated using a single dynamic viscosity of air. With decay process (McMahon, 1979),
FLEXPART 8.2.1, the gravitational settling calculation was
generalized to higher Reynolds numbers and it takes into ac- m(t + 1t) = m(t) exp(−31t) , (1)
count the temperature dependence of dynamic viscosity. This
is done in subroutine get_settling.f90 in an iterative where m is the particle mass (kg) (it can also be a mass
loop, wherein both the Reynolds number and the settling ve- mixing ratio, depending on settings in file COMMAND). This
locity are determined (Naeslund and Thaning, 1991). For ini- removal of particle mass and corresponding accumulation
tialization of the loop, Stokes’ law and a constant viscos- of deposition at the surface is calculated in subroutine
ity estimate is used. The dynamic viscosity is calculated as wetdepo.f90 and has not been changed since earlier ver-
a function of temperature using the formula of Sutherland sions.
(1893). A spherical shape of the particles is assumed in the
settling calculation, which could be further extended in the 2.4.1 Definition of clouds, cloud water content and
future to allow for more complex particle shapes. For particle precipitation
sizes of about 10 µm, settling velocities in the new FLEX-
The location of clouds, the total cloud column wa-
PART version are not much different from earlier versions
ter content and phase, and precipitation intensity and
using the old settling calculation, typically less than 20 %.
phase are needed in the calculation of the wet scaveng-
However, the differences are largest in the cold upper tropo-
ing. Therefore, a three-dimensional cloud mask is de-
sphere, implying, for instance, changes in the residence time
fined in subroutine verttransform_ecmwf.f90 (or
of volcanic ash particles at heights relevant for aviation. The
verttransform_gfs.f90 for GFS data). In previous
residence times in the upper troposphere are increased with
FLEXPART versions, the cloud definition scheme was very
the new scheme, but the effect is not particularly large, typi-
simple and based on relative humidity (RH). In grid columns
cally on the order of 20 %.
with precipitation, grid cells with RH > 80 % were defined as
in-cloud, and those with RH < 80 % were set as below-cloud
2.4 Wet deposition
up to the bottom of the uppermost layer with RH > 80 %.
This was appropriate for the first version of FLEXPART,
In FLEXPART, the calculation of wet scavenging is di-
as the ECMWF had a similarly simple definition of clouds
vided into three parts. First, where scavenging occurs and
and more detailed information was not available from the
which form it takes is determined (e.g., below- or within-
ECMWF archives at the time.
cloud scavenging). Second, the scavenging coefficient is de-
If no cloud information is available from the meteorolog-
termined. Third, the actual removal of particle mass is calcu-
ical data, the old RH-based scheme is still used in FLEX-
lated.
PART. However, nowadays, specific cloud liquid water con-
With respect to the first part, it is important to un-
tent (CLWC; kg kg−1 ) and specific cloud ice water content
derstand that wet scavenging occurs only in the pres-
(CIWC; kg kg−1 ) are available as 3-D fields in meteorolog-
ence of clouds and where precipitation occurs. In air
ical analyses from the ECMWF, and NCEP also provides
columns without clouds, above the top of the clouds, and
the 3-D cloud water (liquid plus ice) mixing ratio (CLWMR;
where neither the large-scale nor the convective precip-
kg kg−1 ), from here on referred to as qc . A cloudy grid cell
itation rate exceeds 0.01 mm h−1 , no scavenging occurs.
is defined when qc > 0. FLEXPART v10.4 can ingest the
To quickly know where a particle is located relative to
ECMWF CLWC and CIWC either separately or as a sum
the clouds, in subroutines verttransform_ecmwf.f90
(qc =CLWC+CIWC). However, to save storage space, we
andverttransform_gfs.f90 each grid cell is catego-
recommend retrieving only the sum, qc , from the ECMWF,
rized as being in a cloud-free column, above a cloud, inside a
as the relative fractions of ice and liquid water can be param-
cloud or below a cloud. This cloud definition has been com-
eterized quite accurately using Eq. (4).
pletely revised compared to earlier versions and is described
The column cloud water (cl ; kg m−2 ), which is needed for
in Sect. 2.4.1.
the in-cloud scavenging parameterization, is calculated by in-
With respect to the second step, the scavenging coefficient
tegrating qc over all vertical z levels:
3 (s−1 ) is determined in subroutine get_wetscav.f90.
After a series of updates, in particular Grythe et al. (2017), cl =
X
qc (z)ρair (z)1z , (2)
FLEXPART now distinguishes between below-cloud and in- z

www.geosci-model-dev.net/12/4955/2019/ Geosci. Model Dev., 12, 4955–4997, 2019

4962 I. Pisso et al.: FLEXPART version 10.4

where ρair (z) is the density of the air in the grid cell, and 1z 2.4.2 Below-cloud scavenging
is the vertical extent of the grid cell. In older FLEXPART
versions, cl was parameterized based on an empirical equa- For gases, the scavenging coefficient, 3, for below-cloud
tion given in Hertel et al. (1995) using the subgrid (see below scavenging is calculated as described in Asman (1995),
for a description of how subgrid is defined) surface precipi- 3 = AIsB , (5)
tation rate Is (mm h−1 ). While such a parameterization is not
needed anymore if qc is available, it is still activated in the where the scavenging parameters A and B depend on the
case that cloud water input data are missing. However, in or- chemical properties of the gas and are specified in the
der to ensure that cl from the parameterization is statistically SPECIES_nnn file as described in Sect. 5.1.3 (nnn repre-
consistent with the cloud data, we derived the modified ex- sents the species number (0–999) used for the simulation).
pression In older FLEXPART versions, this scheme was used also
for aerosols; however, Grythe et al. (2017) developed a new
cl = 0.5 × Is0.36 (3) aerosol scavenging scheme that is implemented in FLEX-
PART v10.4 and briefly summarized here.
using a regression analysis between existing cloud and pre-
The relevant processes of collision and attachment of am-
cipitation data.
bient aerosol particles to falling precipitation depend mainly
Precipitation is not uniform within a grid cell. To account
on the relationship between the aerosol and hydrometeor size
for subgrid variability, it is assumed that precipitation is en-
and type (rain or snow) as well as to a lesser degree on the
hanced within a subgrid area and that no precipitation (and
density and hygroscopicity of the aerosol. In FLEXPART
thus no scavenging) occurs outside this subgrid area. The
v10.4, the dependence of scavenging on the sizes of both the
subgrid area fraction and precipitation rate (Is ) are estimated
aerosol and falling hydrometeors are taken into account by
from the grid-scale precipitation rate (It ) based on values tab-
the schemes of Laakso et al. (2003) for rain and Kyrö et al.
ulated in Hertel et al. (1995). This parameterization of sub-
(2009) for snow. Both schemes follow the equation
grid variability is used for all scavenging processes in FLEX-
PART and maintained from previous FLEXPART versions as 3
log10 ( ) =C∗ (a + b Dp−4 + c Dp−3 + d Dp−2
described in Stohl et al. (2005). 30
The precipitation phase needed for the below-cloud scav- Is
enging scheme is simply based on ambient grid-scale tem- + e Dp−1 + f ( )0.5 ) , (6)
I0
perature, with snow occurring below 0 ◦ C and rain above. For
d
cloud water, cl , we assume a temperature-dependent mixture where Dp = log10 dp0p , dp is the particle dry diameter pro-
of liquid and solid particles, for which the liquid fraction (αL ) vided in the SPECIES_nnn file, dp0 = 1 m, 30 = 1 s−1 and
is calculated based on the local temperature T , I0 = 1 mm h−1 . Coefficients for factors a–f are different for
rain and snow scavenging and are given in Table 2. The C∗
T − TI 2
 
αL = , (4) values are collection efficiencies that reflect the properties of
TL − TI the aerosol and must be given for both rain (C∗ = Crain ) and
where TL = 0◦ C and TI = −20 ◦ C. For T > TL , αL = 1 and snow scavenging (C∗ = Csnow ) in the SPECIES_nnn file.
for T < TI , αL = 0. Even when CLWC and CIWC are avail- Notice that by setting Csnow = 0, below-cloud scavenging by
able as separate fields, we derive the liquid fraction (αL ) of snowfall is switched off (similarly, Crain = 0 for rain).
cloud water from the local temperature. The ice fraction αI
is 1 − αL . Comparisons have shown that CLWC is very accu- 2.4.3 In-cloud scavenging
rately reproduced by αL qc .
For in-cloud scavenging of both aerosols and gases, 3 is cal-
The cloud information should be linearly interpolated like
culated as described in Grythe et al. (2017):
the other variables, and situations in which the diagnosed
cloud is incompatible with the precipitation rate (be it be- 3 = icr Si Is , (7)
cause of interpolation or because of convective precipitation
where icr = 6.2 is the cloud water replenishment factor,
accompanied by too shallow or lacking grid-scale clouds)
which was determined empirically in Grythe et al. (2017)
need to receive special treatment. This is planned for a ver-
(there it was named icr ), and Si is proportional to the in-
sion upgrade in the near future in conjunction with a bet-
cloud scavenging ratio, which is derived differently for gases
ter interpolation scheme for precipitation (see Hittmeir et al.,
and aerosols.
2018). In certain cases, the deposition calculation of FLEX-
For gases, Si = 1−c1l , where H is Henry’s constant
PART might be improved by using higher-resolution precip- H RT +cl
itation data from other sources such as from radar obser- (describing the solubility of the gas and specified in the
vations (Arnold et al., 2015); however, as the precipitation SPECIES_nnn file), R is the gas constant and T is tem-
and ECMWF cloud data, and also the precipitation and wind perature. Notice that this is applied for both liquid-phase and
fields, may not match, this does not guarantee better results. ice clouds.

Geosci. Model Dev., 12, 4955–4997, 2019 www.geosci-model-dev.net/12/4955/2019/

I. Pisso et al.: FLEXPART version 10.4 4963

Table 2. Parameters used in Eq. (6) for below-cloud scavenging.

C∗ a b c d e f Reference
Rain scavenging Crain 274.36 332 839.6 226656 58 005.9 6588.38 0.24498 Laakso et al. (2003)
Snow scavenging Csnow 22.7 0 0 1321 381 0 Kyrö et al. (2009)

For aerosols, the in-cloud scavenging is dominated by ac- and v10.4. A simulation performed with v10.4 but which em-
tivated particles forming cloud droplets or ice nuclei. Those ulated the in-cloud scavenging of v9 showed that the differ-
may eventually combine to form a hydrometeor that falls out ence is mainly due to the decreased in-cloud scavenging in
of the cloud, thus removing all aerosol particles contained in the new removal scheme compared to the old one. Notice
it. Therefore, Si depends on the nucleation efficiency (Fnuc ) that the lifetime obtained with v10.4 is much closer to the
and cl : observation-based lifetimes. Turning off the below-cloud re-
moval has a relatively small effect, increasing the lifetime to
Fnuc 11 d, whereas turning off the in-cloud removal extends the
Si = . (8)
cl lifetime to the unrealistic value of 66 d (see bottom two pan-
els in Fig. 2). This highlights the dominant role of in-cloud
Fnuc describes how efficiently the aerosol particles are ac- removal for accumulation-mode particles in FLEXPART.
tivated as cloud droplet condensation nuclei (CCN) or ice Notice that compared to older versions of FLEXPART, the
nuclei (IN): SPECIES_nnn files now include additional parameters re-
lated to the wet deposition scheme. Old input files, therefore,
Fnuc = αL CCNeff + αI INeff , (9) need to be updated for use with FLEXPART v10.4. The re-
quired format changes are detailed in Sect. 5.1.3.
where the relative abundances of the liquid and ice phase
are accounted for by the factor αL . Values for the efficien-
cies, CCNeff and INeff , are available from the literature for 2.5 Source–receptor matrix calculation of deposited
many different types of aerosols (e.g., black carbon, mineral mass backward in time
dust particles or soluble particles) and some have been col-
lected in SPECIES_nnn files distributed with FLEXPART When running FLEXPART forward in time for a deposit-
(see Sect. 5.1.3). The CCNeff and INeff values are set for an ing species with a given emission flux (kilograms per re-
aerodynamic particle radius of 1 µm, but CCN and IN effi- lease as specified in file RELEASES), the accumulated
ciencies increase with increasing particle size. The in-cloud wet and dry deposition fluxes (ng m−2 ) are appended to
parameterization takes this into account. For further details the FLEXPART output files (grid_conc_date and/or
on the wet scavenging scheme used in FLEXPART, see Gry- grid_pptv_date, for which date represents the date
the et al. (2017). and time in format YYYYMMDDhhmmss; see Sect. 6) con-
taining the atmospheric concentration and/or volume mix-
2.4.4 Influence of wet scavenging on the aerosol lifetime ing ratio output. The deposition is always given in mass
units, even if atmospheric values are given in mixing ratio
Aerosol wet scavenging controls the lifetime of most units. In contrast to concentration values, deposition quanti-
aerosols. In Fig. 2, we compare modeled e-folding lifetimes ties are accumulated over the time of the simulation, so the
from a number of FLEXPART simulations using different deposited quantities generally increase during a simulation
model versions and switching off in-cloud and below-cloud (except when radioactive decay is activated, which also af-
scavenging in FLEXPART v10.4 with measured lifetimes. fects deposited quantities and can decrease them).
The parameter settings in FLEXPART used for these com- As discussed in Sect. 1, running FLEXPART backward
parisons were the same as used by Grythe et al. (2017). To in time for calculating SRRs is more efficient than run-
derive aerosol lifetimes in a consistent way from both mea- ning it forward if the number of (potential) sources is larger
surements and model simulations, a radionuclide attached than the number of receptors. For atmospheric concentra-
to ambient aerosol and a noble gas radionuclide were used. tions (or mixing ratios), the backward mode has been avail-
Kristiansen et al. (2016) used the same method to compare able from the very beginning and in an improved form since
many different aerosol models, and we refer to their paper FLEXPART v5 (Stohl et al., 2003; Seibert and Frank, 2004).
for more details on the method. For our model simulations, This has proven very useful for the interpretation of ground-
several size bins of aerosols were used, though total con- based, shipborne or airborne observations (e.g., to character-
centrations and lifetimes are largely controlled by 0.4 and ize sources contributing to pollution plumes). Furthermore,
0.6 µm particles (see Grythe et al., 2017). E-folding life- the inversion scheme FLEXINVERT (Thompson and Stohl,
times increase from 5.8 to 10.0 d between FLEXPART v9 2014) that is used to determine the fluxes of greenhouse gases

www.geosci-model-dev.net/12/4955/2019/ Geosci. Model Dev., 12, 4955–4997, 2019

4964 I. Pisso et al.: FLEXPART version 10.4

the calculation of, for example, 100 years of seasonally re-

solved deposition data in less than 24 h of computation time.
We illustrate the different backward modes and explain
the required settings with an example. The calculations were
run for a single receptor location, Ny-Ålesund in Spitsber-
gen (78.93◦ N, 11.92◦ E) and for the 24 h period from 18 Au-
gust 2013 at 20:00 UTC to 19 August 2013 at 20:00 UTC.
SRR values are calculated for the atmospheric concentration
averaged over the layer 0–100 m a.g.l., as well as for wet
and dry deposition. The substance transported is black car-
bon (BC), which is subject to both dry and wet deposition.
Backward simulations for wet and dry deposition must al-
ways be run separately. In order to obtain SRR values for
total deposition, results for wet and dry deposition need to be
summed.
The backward mode is activated by setting the simulation
direction, LDIRECT in file COMMAND (see Sect. 5), to −1.
The three simulations are obtained by consecutively setting
IND_RECEPTOR to 1 (for concentration), 3 (wet deposi-
tion) and 4 (dry deposition). IND_SOURCE is always set to
1, meaning that the sensitivities (SRR values) are calculated
with respect to physical emissions in mass units. A complete
list of possible options is reported in Table 1 of Eckhardt et
al. (2017).
Figure 3 shows the resulting SRR (i.e., emission sensitiv-
ity) fields for the concentration as well as dry and wet de-
position at the receptor. Dry deposition occurs on the Earth’s
surface, and therefore particles are released in a shallow layer
adjacent to the surface. Its height is consistent with the shal-
low depth over which dry deposition is calculated in forward
Figure 2. Aerosol lifetimes estimated from the decrease in radionu- mode (user settings for the release height are ignored for dry
clide ratios (aerosol-bound 137 Cs and noble gas 133 Xe as a passive deposition backward calculations). Dry deposition rates are
tracer) with time after the Fukushima nuclear accident, as measured the product of the surface concentration and the deposition
and modeled at a number of global measurement stations. For de- velocity. Therefore, the SRR fields for surface concentration
tails on the method, see Kristiansen et al. (2016). E-folding life- (Fig. 3a) and dry deposition (Fig. 3b) show similar patterns,
times, τe , are estimated based on fits to the data and reported in in this case indicating high sensitivity for sources over Scan-
each panel. In the top panel, the observed values are shown and dinavia and northwestern Russia. The differences in the spa-
in subsequent panels from the top, modeled values are given for tial patterns are mainly due to temporal variability in the dry
FLEXPART v9, FLEXPART v10.4, FLEXPART v10.4 with param- deposition velocity at the receptor caused by varying mete-
eter settings to emulate removal as in v9, FLEXPART v10.4 with
orological conditions (e.g., stability) and surface conditions
no below-cloud removal and FLEXPART v10.4 with no in-cloud
removal.
during the 24 h release interval.
Wet deposition, on the other hand, can occur anywhere in
the atmospheric column from the surface to the top of the
precipitating cloud. FLEXPART automatically releases par-
ticles in the whole atmospheric column (again, user settings
is based on backward simulations. However, there are also for the release height are ignored), but particles for which no
measurements of deposition on the ground, e.g., in precipita- scavenging occurs (e.g., those above the cloud top or when
tion samples or ice cores, and for this type of measurement no precipitation occurs) are immediately terminated. There-
no backward simulations were possible until recently. There- fore, and because of the vertical variability of the scaveng-
fore, Eckhardt et al. (2017) introduced the option to calculate ing process, the sensitivity for the deposited mass can de-
SRR values in backward mode also for wet and dry deposi- viate significantly from the sensitivity corresponding to sur-
tion, and a first application to ice core data was presented by face concentration. Here (Fig. 3c), the sensitivity is high over
McConnell et al. (2018). It is anticipated that quantitative in- Scandinavia and northwestern Russia, as was already seen
terpretation of ice core data will be a major application of the for surface air concentrations and dry deposition. However,
new backward mode, which is efficient enough to allow for in addition, sources located in North America and eastern

Geosci. Model Dev., 12, 4955–4997, 2019 www.geosci-model-dev.net/12/4955/2019/

I. Pisso et al.: FLEXPART version 10.4 4965

the specification of the RELEASES file. The emission fluxes

must be volume averages over the output grid cells specified
in the OUTGRID file, typically surface emission fluxes (in
kg m−2 s−1 ) divided by the height of the lowermost model
layer.

2.6 Sensitivity to initial conditions

Backward simulations with FLEXPART in the context of in-

verse modeling problems typically track particles for several
days up to a few weeks. This is sufficient to estimate con-
centrations at the receptor only for species with atmospheric
lifetimes shorter than this period. Many important species
(e.g., greenhouse gases such as methane) have considerably
longer lifetimes. For such long-lived species, most of the at-
mospheric concentration variability is still caused by emis-
sion and loss processes occurring within the last few days
before a measurement because the impact of processes occur-
ring at earlier times is smoothed out by atmospheric mixing.
This leads to a relatively smooth “background” (in time se-
ries analyses sometimes also called a baseline) that is often a
dominant fraction of the total concentration but that does not
vary much with time, with short-term fluctuations on top of
Figure 3. Source–receptor relationships (for emissions occurring in
it. The signal of the regional emissions around the measure-
the lowest 100 m a.g.l.) for black carbon observed at Ny-Ålesund in
ment site is mostly contained in the short-term concentration
Svalbard for a 24 h period starting on 18 August 2013 at 20:00 UTC.
The sensitivities were calculated for (a) concentrations (s) in the fluctuations but in order to use it in inverse modeling, the
layer 0–100 m a.g.l., (b) dry deposition (mm) and (c) wet deposition background still needs to be accounted for, as otherwise no
(mm). direct comparison to measurements is possible.
One simple method is to estimate the background from
the measurements as, e.g., in Stohl et al. (2009). A better ap-
Siberia also contribute strongly to wet deposition. The max- proach is to use a concentration field taken from a long-term
imum over the ocean close to the North American east coast forward simulation with an Eulerian model or with FLEX-
is likely due to lifting in a warm conveyor belt, followed by PART itself, especially if nudged to observations (Groot
fast transport at high altitude. Zwaaftink et al., 2018), as an initial condition for the back-
Concentration, dry deposition and wet deposition at the re- ward simulation. This field needs to be interfaced with the
ceptor can be calculated from the SRR fields shown in Fig. 3 FLEXPART backward simulation by calculating the receptor
as follows. sensitivity to the initial conditions (see Eqs. 2–6 in Seibert
and Frank, 2004). For instance, for a 10 d backward simu-
c = mc · q lation, the concentration field needs to be sampled at those
dd = md · q points in time and space when and where each particle tra-
dw = mw · q (10) jectory terminates 10 d back in time. Furthermore, it is nec-
essary to quantify the effects of deposition or chemical loss
Here, c is the modeled concentration (in kg m−3 ), dd the during the backward simulation on this background (the fac-
dry deposition rate and dw the wet deposition rate (both in tor p(0) in Seibert and Frank, 2004). For example, chemical
kg m−2 s−1 ). In this specific case with only a single scalar reactions with hydroxyl radicals will reduce initial concen-
receptor, the source–receptor matrix degenerates to a vector trations of methane en route to the receptor, even though this
of the SRR values, one for each of the three types of receptor is not much during a 10 d period.
(mc for concentration in units of seconds, md for dry depo- Since version 8.2, FLEXPART has provided an option
sition and mw for wet deposition, both in units of meters). to quantify the influence of initial conditions on the recep-
In order to obtain the concentration or the deposition rates, tor in backward simulations, which is activated with the
these vectors need to be multiplied with the vector of emis- switch LINIT_COND in file COMMAND. Then, gridded fields
sions q (kg m−3 s−1 ). If the total deposition is desired, the de- containing the sensitivities to background mixing ratios (or
position rates dd and dw can be multiplied with the receptor concentrations, depending on user settings for the switch
time interval 1Tr , in our case 86 400 s (24 h). Note that this LINIT_COND in file COMMAND) are produced and stored in
is the period during which particles are released according to the output files grid_initial_nnn (nnn stands for the

www.geosci-model-dev.net/12/4955/2019/ Geosci. Model Dev., 12, 4955–4997, 2019

4966 I. Pisso et al.: FLEXPART version 10.4

species number) on the same 3-D grid as the regular output, wherein initial conditions were taken from a gridded ob-
defined in the files OUTGRID and OUTGRID_NEST. In this servation product. A further output option, which was
case, a concentration would be calculated as also introduced for practical considerations of inverse
modeling, is the LINVERSIONOUT option in the file
c = mi · cb + mc · q, (11)
COMMAND. If LINVERSIONOUT is set to 1, then the
where mi denotes the sensitivity to the initial condition and grid_time_date_nnn and grid_initial_nnn files
cb the background concentration when and where particles are written per release with a time dimension of footprints
are terminated. instead of the default per footprint with a time dimension
Figure 4 shows an example of the use of the sensitivities of releases. Since inverse modeling assimilates atmospheric
of receptor mixing ratios (here, of methane) to both surface observations and each observation is represented by a single
emissions and initial conditions. The panel (b) shows the sen- release, it is computationally more efficient to read in the grid
sitivity to surface emissions on one particular day, and the files separated by release. This output format also interfaces
panels (c) and (d) show the sensitivity to initial conditions be- directly with FLEXINVERT.
low and above 3000 m for the same day. Both results are from
an 8 d backward simulation from one receptor site in Sweden. 2.7 Chemical reactions with the hydroxyl radical (OH)
It can be seen that the sensitivity to emissions is highest close
to the station, but there is also substantial sensitivity to emis- The hydroxyl (OH) radical reacts with many gases and is
sion uptake over large parts of central and eastern Europe. the main cleansing agent in the atmosphere. While it is in-
The particles terminated 8 d before arrival at the receptor in volved in highly nonlinear atmospheric chemistry, for many
a roughly croissant-shaped area covering large parts of Eu- substances (e.g., methane) a simplified linear treatment of
rope and the North Atlantic, as indicated by the sensitivity loss by OH is possible using prescribed OH fields. For this,
to initial conditions. Most of the sensitivity is located below monthly averaged 3◦ × 5◦ resolution OH fields for 17 atmo-
3000 m but there is also some influence from higher levels. spheric layers are used in FLEXPART. The fields were ob-
Notice that only two layers are shown in Fig. 4, whereas the tained from simulations with the GEOS-Chem model (Bey et
real model output has much higher vertical resolution. al., 2001) and are read from the file OH_variables.bin
The sensitivity to initial conditions was interfaced with a by the subroutine readOHfield.f90.
domain-filling methane forward simulation as described in Tracer mass is lost by reaction with OH if a positive value
Groot Zwaaftink et al. (2018) (not shown), while the emis- for the OH reaction rate is given in the file SPECIES_nnn.
sion sensitivity was interfaced with an emission inventory for In FLEXPART v10.4, the OH reaction scheme was modified
methane (not shown), as given by Eq. (11). This was done to account for (i) hourly variations in OH and (ii) the temper-
for daily simulations throughout 1 month, thus generating a ature dependence of the OH reaction rate (Thompson et al.,
time series of background mixing ratios (from the first term 2015). This makes the chemical loss calculations more accu-
in Eq. 11 only) and total mixing ratios (Fig. 4a). The latter rate, especially for substances with shorter lifetimes (of the
include the contributions from emissions during the 8 d back- order of weeks to months), for example ethane. Hourly OH
ward simulation. It can be seen that the methane background fields are calculated from the stored monthly fields by cor-
advected from 8 d back varies relatively little between about recting them with the photolysis rate of ozone calculated with
1910 and 1940 ppbv, while the emission contributions vary a simple parameterization for cloud-free conditions based on
from 0 (on 29 October) to about 200 ppbv (on 19 October, the solar zenith angle (gethourlyOH.f90):
the date for which the sensitivity plots are shown).
In practical applications for inverse modeling, source– j
OH = OH∗ , (12)
receptor sensitivities are often only needed at the sur- j∗
face (as most emissions occur there), while sensitivities to
the background are needed in 3-D. By setting the option where j represents the hourly photolysis rates calculated for
SURF_ONLY to 1 in the COMMAND file, the regular out- all 3-D locations in the field, while j ∗ represents the cor-
put files grid_time_date_nnn containing the source– responding monthly mean rates, precalculated and stored in
receptor sensitivities will include only the first vertical level file OH_variables.bin together with the monthly mean
as defined in the file OUTGRID, while the full vertical fields OH∗ (see Sect. 5.1.8). The motivation for this is that
resolution is retained in grid_initial_nnn files con- OH production closely follows the production of O(1 D) by
taining the sensitivities to the initial conditions. Since the the photolysis of ozone, allowing for this simple parameter-
data amounts stored in the grid_time_date_nnn files ization of OH variability. At any time, two hourly OH fields
can be much larger than in the grid_initial_nnn are in memory and are interpolated to the current time step.
files, this is a highly efficient way to save storage space. Figure 5 shows the annual and daily variation of OH for two
This setup also interfaces directly with the inverse model- locations as obtained with this simple parameterization.
ing package FLEXINVERT (Thompson and Stohl, 2014). The OH reaction rate κ (s−1 ) is calculated in
An application can be found in Thompson et al. (2017) ohreaction.f90 using the temperature-dependent

Geosci. Model Dev., 12, 4955–4997, 2019 www.geosci-model-dev.net/12/4955/2019/

I. Pisso et al.: FLEXPART version 10.4 4967

Figure 4. Example of FLEXPART 8 d backward runs for methane from a site in southern Sweden (Hyltemossa) demonstrating the combined
use of sensitivities to emissions and initial conditions. (a) Time series of methane background mixing ratios and total mixing ratios in Octo-
ber 2016. (b) Sensitivity of the methane mixing ratio at Hyltemossa on 19 October 2016 to methane emissions at the surface. (c) Sensitivity
of the methane mixing ratio at Hyltemossa on 19 October 2016 to methane initial conditions below 3000 m. (d) Sensitivity of the methane
mixing ratio at Hyltemossa on 19 October 2016 to methane initial conditions above 3000 m. Blue asterisks on the maps mark the receptor
location.

formulation where A is the activation energy, R is the gas constant and

N −D/T κr is the former OH reaction rate (referring to Tr = 298 K),
κ =CT e [OH], (13) which were specified in the SPECIES_nnn file for earlier
where C, N and D are species-specific constants (assigned versions.
in the SPECIES_nnn files), T is the absolute temperature, OH fields other than those provided with the model
and [OH] the OH concentration (Atkinson, 1997). As the code have been tested in FLEXPART. These fields
OH concentration in file OH_variables.bin is given may have higher spatial and temporal resolution (e.g.,
in units of molecules per cubic centimeter, the unit of C Fang et al., 2016), which is important for chemical
needs to be in cubic centimeters per molecule per second species with short lifetimes. Users are required to modify
(cm3 molecule−1 s−1 ). The mass m of a given species after readOHfield.f90 and gethourlyOH.f90 to read in
reaction with OH is determined as other OH fields and be aware that expressions of the OH re-
0 action rate or reaction with OH might differ from those in
m(t + 1t 0 ) = m(t) e−κ1t , (14) the above equations. If this is the case users need to modify
ohreaction.f90, too.
where 1t 0 is the reaction time step (given by lsynctime).
Backwards compatibility with the former temperature-
independent specification of the OH reaction (version 9 and 2.8 Dust mobilization scheme
before) can be achieved by setting the constant N in the
SPECIES_nnn file to zero. The constants C and D can be Desert dust is a key natural aerosol with relevance for both
derived from the former parameters as follows: climate and air quality. FLEXPART has been used earlier
with preprocessors to initialize dust amounts from wind
C = κr eD/Tr (15) speed and surface properties following Tegen and Fung
and (1994) (Sodemann et al., 2015). Now a dust mobilization
routine has been included as a preprocessing tool in FLEX-
D = A/R, (16) PART v10.4. The scheme, called FLEXDUST, was devel-

www.geosci-model-dev.net/12/4955/2019/ Geosci. Model Dev., 12, 4955–4997, 2019

4968 I. Pisso et al.: FLEXPART version 10.4

Figure 6. Average annual dust emission for the period 1990–2012

estimated with FLEXDUST driven with ERA-Interim meteorology.

ERA-Interim reanalysis data, Global Land Cover by National

Mapping Organizations version 2 (Tateishi et al., 2014), and
and sand and clay fractions from the Global Soil Data Task
(2014). Erodibility is enhanced in topographic depressions,
and dust emissions are modified by soil moisture and snow
cover. The module includes high-latitude dust sources in the
Northern Hemisphere. These sources are rarely included in
global dust models, even though they appear important for
the climate system and substantially contribute to dust in the
Arctic (Bullard et al., 2016; Groot Zwaaftink et al., 2016).
Icelandic deserts in particular are known to be highly ac-
tive, and a high-resolution surface type map for Iceland can
therefore be included in FLEXDUST simulations (Arnalds et
al., 2016; Groot Zwaaftink et al., 2017). Like in FLEXPART,
nested meteorological fields can be used for specific regions
Figure 5. Annual and daily OH concentration variation as obtained of interest. The size distribution of emitted dust follows Kok
with the simple parameterization based on photolysis rates of ozone
(2011), is independent of friction velocity and is by default
for two locations, one in the Northern Hemisphere (a) and one in
represented by 10 size bins. This can be changed depending
the Southern Hemisphere (b). Line labels correspond to the time of
day. on known properties or assumptions of dust sources. The dust
particles are assumed to be spherical in FLEXPART. An ex-
ample of annual mean dust emissions from 1990 to 2012 cal-
culated with FLEXDUST driven with ERA-Interim meteo-
oped to simulate mineral dust transport with FLEXPART in rology is shown in Fig. 6. Further details on FLEXDUST, in-
forward or backward simulations (Groot Zwaaftink et al., cluding model evaluation, are given by Groot Zwaaftink et al.
2016). This module runs independently from FLEXPART (2016). The source code is available from the git repository:
and produces gridded output of mineral dust emissions as https://git.nilu.no/christine/flexdust.git (last access: 30 Octo-
well as input files (RELEASES) that can be used for FLEX- ber 2019).
PART simulations of atmospheric transport. It can thus be
considered a preprocessing (for forward simulations) or post-
processing tool (for backward simulations) for FLEXPART 3 Parallelization
v10.4.
In FLEXDUST, emission rates are estimated according to In a Lagrangian model like FLEXPART, particles move to-
the emission scheme proposed by Marticorena and Berga- tally independently of each other. This facilitates efficient
metti (1995). We thereby assume that sandblasting occurs parallelization of the code. The most simple and often most
in the case that sand is present and a minimum threshold effective way is running several instances of the model in par-
based on the size-dependent threshold friction velocity fol- allel. For example, if the model is to be run backwards (for
lowing Shao and Lu (2000) can be applied. The following are 10 d, for example) at regular intervals from a measurement
used as input for the model: ECMWF operational analysis or site for a year, one could run the model separately, in par-

Geosci. Model Dev., 12, 4955–4997, 2019 www.geosci-model-dev.net/12/4955/2019/

I. Pisso et al.: FLEXPART version 10.4 4969

Figure 7. Computational time (a, b) and speedup (c, d) for up to 16 processes on a single node. In panels (a, c), all processes read meteoro-
logical input data, whereas in panels (b, d), a dedicated process reads and distributes input data for Np ≥ 4.

Figure 8. Computational time (a) and speedup (b) for up to 256 processes on 16 nodes. Logarithmic scaling along both axes. For n ≥ 4 a
dedicated process reads and distributes input data.

allel, for monthly subperiods. The total computation time of Even if a task cannot easily be decomposed into runs for
the 12 monthly processes together is nearly the same as if the different periods or sources, trivial parallelization is still pos-
model is run as one process for the whole year. Some over- sible if a large number of particles is desired, for example
head in processing input data occurs because, in the above in a domain-filling simulation for which tens of millions of
example, 10 extra days of data per process are needed to cal- particles may be used. The strategy in this case would be to
culate trajectories 10 d back into the preceding month. One assign a fraction of the particles to each run. Note that differ-
disadvantage of that approach is that the memory needed for ent random seeds should be used for each run, which requires
holding the meteorological input data and the model output a manual change and recompilation of the code.
fields is multiplied. However, this overhead is often small; As a user-friendly alternative, FLEXPART v10.4 has
thus, this approach has been used very often by FLEXPART been parallelized using standard parallelization libraries.
users in the past. Common parallelization libraries are Open Multi-Processing
(OpenMP; http://www.openmp.org/, last access: 30 Octo-

www.geosci-model-dev.net/12/4955/2019/ Geosci. Model Dev., 12, 4955–4997, 2019

4970 I. Pisso et al.: FLEXPART version 10.4

ber 2019), which is designed for multicore processors with simulation time t = Ti+1 the dedicated “reader process” will
shared memory, and Message Passing Interface (MPI, 2015) distribute the newest data to the other processes and imme-
for distributed memory environments. Examples of other diately start reading fields for time Ti+3 , while the other pro-
Lagrangian particle models that have been parallelized are cesses continue doing trajectory calculations. A hard-coded
NAME (Jones et al., 2007) and FLEXPART–WRF (Brioude integer (read_grp_min in file mpi_mod.f90) is used to
et al., 2013), which use a hybrid approach (OpenMP+MPI). set the minimum number of total MPI processes for which
For FLEXPART v10.3 we decided to use a pure MPI ap- this separate process will be reserved for reading input data.
proach for the following reasons. For the examples shown in Sect. 3.2 a value of 4 was used
(Figs. 7 and 8).
– It is simpler to program than a hybrid model and more
flexible than a pure OpenMP model. 3.2 Performance aspects
– While OpenMP in principle may be more effective in
To assess the performance of the parallel code we performed
a shared memory environment, MPI can often perform
three scaling experiments of various size on different com-
equally well or better provided there is not excessive
putational platforms.
communication between the processes.

– MPI offers good scalability and potentially low over- 3.2.1 40 million particles, single 16-core node
head when running with many processes.
In the following we present the results from running the code
3.1 Implementation on a machine equipped with an Opteron 6174 processor with
16 cores. Compilation was done using gfortran version 4.9.1
The FLEXPART code contains several computational loops and OpenMPI version 1.8.3. For the experiment, 40 million
over all the particles in the simulation, which is where particles were released and propagated 48 h forward in time.
most of the computational time is spent for simulations with We ran with this setup with an increasing number of pro-
many particles. The basic concept behind our parallel code cesses, from 1 to 16. All time measurements in the code were
closely resembles the “trivial parallelization” concept de- made with the MPI_wtime() subroutine.
scribed above. When launched with a number of processes, For the first experiment, every process separately pro-
Np , each process will separately calculate how many parti- cessed the meteorological input data. Figure 7a and c show
cles to release per location, attempting to achieve an approx- the CPU time Tn used in the case of n processes and the rela-
imately even distribution of particles among the processes tive speedup factor S(n) = T1 /Tn . Time and speedup shown
while keeping the total number of particles the same as for a for “particle loops” includes the three most computationally
simulation with the serial version. Each running process will demanding particle loops (integration of the Langevin equa-
generate an independent series of random numbers and sepa- tion, wet deposition and concentration calculations), but, in
rately calculate trajectories and output data for its set of par- addition, FLEXPART contains a few smaller loops over par-
ticles. Explicit communications between processes are only ticles that exhibit similar performance improvements. We see
used when the output fields are combined at the master pro- that for 40 million particles, the loops over particles take the
cess (MPI rank 0) using MPI_Reduce operations, before largest share, at least 87 % of the total time when run with
writing the output. Also, in the case in which the output of one process. Close-to-perfect speedup is expected and ob-
all individual particle properties is desired (option IPOUT1 served for these loops (compare results for “particle loops”
= 1 or 2 in file COMMAND), we let each process append its and “ideal (particle loops)” in Fig. 7a and c). The major bot-
data to the same file. We thus avoid the costly operation of tleneck for overall performance in this case is that each pro-
transferring particle properties between processes. The per- cess reads the same input files from disk, thus forcing the
formance of the implementation is discussed in Sect. 3.2 (see others to wait. This bottleneck causes the speedup to deviate
Fig. 7). substantially from the ideal situation when more than a few
Some parts of the code are not simply loops over parti- processes are used (compare results for “total” and “ideal” in
cles, most notably the routines for reading and transforming Fig. 7a and c).
the input meteorological data. It follows that the performance Next we repeated the experiment above but set aside a ded-
gain of using parallel FLEXPART in general is better for sim- icated process for reading the meteorological data whenever
ulations with a larger number of particles. We have, however, n ≥ 4. The results are shown in Fig. 7b and d. Numerical val-
implemented a feature whereby instead of having each MPI ues for the speedup factors for selected numbers of processes
process read and process the same input data, one dedicated are given in Table 3. We observed that with n ≥ 7 there was
MPI process is set aside for this purpose. When the simula- consistently a benefit to setting aside the dedicated reader
tion time t lies in the interval between wind field time Ti and process, whereas for n < 7 it was more effective to have all
Ti+1 , all other processes calculate particle trajectories, while processes read data and thus an extra process available for
this dedicated process ingests input fields from time Ti+2 . At doing the trajectory calculations. These results will of course

Geosci. Model Dev., 12, 4955–4997, 2019 www.geosci-model-dev.net/12/4955/2019/

I. Pisso et al.: FLEXPART version 10.4 4971

Table 3. Computational speedup S for up to 16 processes (single- amples. Still, we see that a speedup of 2.38 on a 4-core lap-
node experiment) for the two different MPI modes, with 40 million top and 5.25 on a 16-core machine is attainable. We also note
released particles. that for practical applications, users would likely use the se-
rial version for applications with so few particles and, if there
Number of processes 1 2 4 8 16 are many such runs to be done, use trivial parallelization by
All processes read 1 1.89 3.30 5.76 9.16 submitting many separate serial runs in parallel. The paral-
Dedicated read process 1 1.91 3.09 6.72 13.10 lelization feature is most useful for cases with a very large
number of particles that cannot so easily be split in many
separate runs, such as domain-filling simulations.
vary with the resolution of the input data, the number of par-
3.3 Validation
ticles and the system on which the program is run.
In order to ensure that the parallel version produces results
3.2.2 500 million particles, multiple 16-core nodes
with the same accuracy as the serial version, we have per-
We performed a larger-scale experiment at the Abel com- formed a set of tests and validation experiments. A direct
puter cluster1 using up to 256 cores on 16 nodes with Intel comparison between the versions can only be performed
Xeon E5-2670 CPUs. For each node, up to 16 cores were in statistical terms because FLEXPART uses Gaussian-
used, and then the number of nodes was determined by the distributed random numbers for calculating the turbulent ve-
total number of processes launched. The FLEXPART setup locities of the particles. For the parallel version we let each
was similar to the previous single-node experiment, but we process independently calculate a set of random numbers,
increased the number of particles to 500 million and reduced which leads to small numeric differences (arising from the
the simulated time to 12 h. Compilation was done with Intel random “noise”) between the versions.
Fortran v16.0.1 and OpenMPI v1.10.2. To confirm that the only source of differences between the
Run time and speedup factors are shown in Table 4 and serial and parallel code is in the random number generation,
Fig. 8. As before we see essentially perfect speedup of the we first observe that when the parallel executable is run using
computationally intensive parts (the particle loops), which is only one process, it produces results identical to the serial
expected. Table 4 also gives the parallel efficiency, which is version. This is as expected, as the first MPI process (rank
seen decreasing for larger Np . This is partly due to the in- 0) always uses the same random number seeds as the serial
creased cost of MPI communications and also because the version.
nonparallel parts of the code have relatively higher impact. Next, we have done tests in which all random numbers are
With 256 processes there are only about 2 million particles set to zero in both codes, corresponding to switching off the
per process and the CPU time is not as clearly dominated turbulent displacements, and we run the parallel version us-
by the particle loops as when 500 million particles all run ing multiple processes. The outputs from the serial and par-
in one process. In addition, the initialization of the code (al- allel versions of the code when run this way are identical
location of arrays, reading configuration files) takes around except for small differences due to round-off errors (e.g., in
20 s for this run, which is significant given a total run time concentration calculations – these round-off errors are typi-
of 299 s. Thus, parallel efficiency would increase for longer cally larger in the serial version due to the larger number of
simulation times and/or for simulations with more particles particles).
per process, i.e., realistic cases that are more likely to be run
with such a large number of processes.
4 Installation, compilation and execution
3.2.3 900 000 particles, laptop and single 16-core node
FLEXPART is usually used in a Linux environment, which
Finally, we examined a small-scale experiment in which we we also assume for the following instructions. However, the
released 900 000 particles and simulated 15 d of transport. model has also been implemented successfully under MacOS
The performance was tested on two systems; a ThinkPad and MS Windows. The default Fortran compiler for FLEX-
P52s laptop (Intel i7-8550U CPU with four cores; results PART v10.4 is gfortran, but ifort, Absoft and PGI compilers
in Table 5) and a machine equipped with an AMD Opteron have been used as well.
6386 SE processor (16 cores; results in Table 6). With this
4.1 Required libraries and FLEXPART download
relatively lower number of particles it is not surprising to see
that the parallel efficiency is lower than in the preceding ex- As the meteorological data from numerical weather predic-
1 Owned by the University of Oslo and Uninett/Sigma2, oper- tion models are usually distributed in GRIB format, a li-
ated by the Department for Research Computing at USIT, the Uni- brary for reading GRIB data is required. It is recommended
versity of Oslo IT department; https://www.uio.no/english/services/ to use ecCodes (https://software.ecmwf.int/wiki/display/
it/research/hpc/abel/ (last access: 30 October 2019). ECC, last access: 30 October 2019), the primary GRIB

www.geosci-model-dev.net/12/4955/2019/ Geosci. Model Dev., 12, 4955–4997, 2019

4972 I. Pisso et al.: FLEXPART version 10.4

Table 4. Run time and speedup for the multi-node experiment with 500 million particles. Up to 16 nodes in the Abel cluster (University of
Oslo).

Number of processes n 1 2 4 8 16 32 64 128 256

Total run time (s) 39 536 19 681 14 123 6380 3061 1568 828 491 299
Speedup factor S 1 2.01∗ 3.37 6.20 12.92 25.22 47.76 80.53 132.12
Parallel efficiency (S/n) 1 1.004 0.843 0.775 0.807 0.788 0.746 0.629 0.516
∗ Superlinear speedup (efficiency greater than 100 %) as seen here is usually attributed to memory and/or cache effects.

Table 5. Run time and speedup using up to four cores on a ThinkPad The directory src contains the code and a makefile. The
P52s laptop (900 000 particles). makefile needs to be adapted to the compiler and libraries
present on the local system. Appendix A4 describes these
Number of processes n 1 2 3 4 steps in detail, including manual installation of the libraries.
Total run time (s) 3504 2035 1790 1470 This was tested for Ubuntu 16.04.3 as well as 18.04.3 LTS
Speedup factor S 1 1.72 1.97 2.38 Linux and MacOS (OS versions 14.5.0 and 18.6.0). Both a
Parallel efficiency (S/n) 1 0.86 0.66 0.775 serial and a parallel executable can be built from the FLEX-
PART v10.4 source files.

4.2 Compiling and running the serial version

encoding–decoding package used at the ECMWF (recent
enough versions of its predecessor grib_api, no longer After correctly setting the library paths in the makefile, the
supported after 2018, can also be used). Data in GRIB-2 for- command make produces the executable called FLEXPART.
mat can be compressed. If this is the case for the input data, It can be executed from the command line by ./FLEXPART
the jasper library is needed2 . If it is desired to produce and then expects a file pathnames to exist in the current
FLEXPART output in the NetCDF format, the NetCDF For- working directory. This file contains information on where
tran Library (https://www.unidata.ucar.edu/software/netcdf/, input data are located and where output data shall be pro-
last access: 30 October 2019) is also required. duced (see Sect. 5). Note that pathnames is expected in
In order to obtain the FLEXPART source code, download the directory from which FLEXPART is started, which can
the appropriate v10.4 tarball from the FLEXPART website3 be different from where the executable file is located. A dif-
and unpack it. ferent name of a pathnames file can be also given as an
argument. FLEXPART can thus be invoked according to the
tar -xvf flexpart10.4.tar
following generic syntax:
To obtain the latest available model version, clone the
path_to_flexpart/flexpart_executable
FLEXPART git repository from the FLEXPART community
path_to_pathnames/pathnames_file.
website.
Using an optional argument, -v (verbose mode), will dis-
git clone https://www.flexpart.eu/ play additional information during the run. Even more in-
gitmob/flexpart formation, including clock time between different program
This repository mirrors https://transport.nilu.no units, will be printed with -v2. Invoking FLEXPART with
(https://git.nilu.no/flexpart/flexpart/-/releases, last ac- the flags -i and -i2 (info mode) will provide detailed run-
cess: 18 November 2019). Additional mirrors exist, specific information while reading input files. However, in
e.g., at GitHub (https://github.com/flexpart/flexpart, this mode FLEXPART then stops before particle trajectories
last access: 30 October 2019) and BitBucket are calculated.
(https://bitbucket.org/flexpart/flexpart, last access: 30 Oc-
4.3 Compiling and running the parallel version
tober 2019). See the “Code and data availability” section
for additional information. After unpacking the tarball or Most subroutines calling MPI functions are in a single mod-
cloning the repository, a local directory structure as shown ule named mpi_mod.f90. Other FLEXPART source files
in Table 1 is created. that depend on this module are given the _mpi.f90 suffix
2 The jasper package is available in Linux distributions; https: to distinguish them from the serial version. During compi-
//github.com/mdadams/jasper (last access: 30 October 2019). lation the makefile selects the source files automatically de-
3 The website, https://flexpart.eu/ (last access: 30 October 2019), pending on whether the parallel or serial version is built.
provides additional information that can be used to supplement In order to compile and run the parallel version, an
these instructions. MPI library must be installed, either as a package from

Geosci. Model Dev., 12, 4955–4997, 2019 www.geosci-model-dev.net/12/4955/2019/

I. Pisso et al.: FLEXPART version 10.4 4973

Table 6. Run time and speedup using up to 16 cores on a machine equipped with an AMD Opteron 6386 SE processor (900 000 particles).

Number of processes n 1 2 4 6 8 10 12 16
Total run time (s) 3788 2337 1376 976 840 765 747 717
Speedup factor S 1 1.62 2.75 3.88 4.51 4.95 5.07 5.29
Parallel efficiency (S/n) 1 0.81 0.69 0.65 0.56 0.50 0.42 0.33

the distribution or built from source code. Both Open- files are created; third, the path to the meteorological
MPI (http://www.open-mpi.org/software/ompi/v1.8/, last ac- input GRIB files; and, fourth, the path to the so-called
cess: 30 October 2019) and mpich2 (http://www.mpich.org/ AVAILABLE file (see point 3). The last two lines can
downloads/, last access: 30 October 2019) work, but test- be repeated if nested input data shall be used. For each
ing on some systems indicates slightly better performance nesting level, one line for the GRIB data directory and
with OpenMPI. As for the other libraries, the MPI li- one for the corresponding AVAILABLE file are needed.
brary names and paths need to be adapted in the make-
file. The MPIF90 variable sets the Fortran compiler wrap- 2. The files containing the run-defining settings are located
per (usually mpifort or mpif90; in the case of coex- in a subdirectory (given in line 1 of pathnames) by
isting OpenMPI and mpich installations, wrappers called default called options (see Table 1). The settings,
mpif90.openmpi or mpif90.mpich may be defined). which control FLEXPART’s physics and program flow,
Compilation of the parallel version should then be done by are stored in different text files listed in Table 7 and de-
the following. scribed in Sect. 5.1. In addition, the options directory
contains data files that are not usually changed by the
make mpi user.
This will produce an executable file FLEXPART_MPI. If 3. The meteorological input data, one file for each input
executed, this will run on a single processor and should pro- time, are stored in GRIB format in a common direc-
duce results identical to the serial version. To activate the tory (specified in line 3 of pathnames). To enable
parallel features, the executable must be run through an MPI FLEXPART to find these files, a file usually named
launcher (here it is important to use the launcher correspond- AVAILABLE (given in line 4 of pathnames) contains
ing to the MPI library that was used for the compilation). a list of all available meteorological input files and their
corresponding time stamps. Additional files containing
mpirun -n <number> FLEXPART_MPI
nested input data may also be provided. In this case, a
In this example, <number> specifies the number of separate file containing the input file names (e.g., named
processes one wishes to launch. For some installations, AVAILABLE_NESTED) must be given. Date and time
mpirun is called mpiexec, or, in the case of coexisting entries in the AVAILABLE* files for mother and nested
OpenMPI and mpich2 installations, mpiexec.openmpi fields must be identical. Details on the meteorological
or mpiexec.mpich2, respectively. Many command-line input data are given in Sect. 5.2.
options exist for mpirun that can be helpful for improving
performance, e.g., processor binding. For a list of these op- 5.1 Run-defining settings: the options directory
tions, see mpirun -help. In practice, the optimal number
of cores for a given simulation will depend on the size of the Here, we give an overview of the information provided in
problem and the hardware availability among other factors. the run-defining FLEXPART user input files listed in Ta-
ble 7. In previous versions of FLEXPART, these files were
formatted text files (coming alternatively in a long and a
5 FLEXPART input
short format). For backward compatibility, these plain text
In this section, we describe the different FLEXPART in- formats are still supported. However, FLEXPART v10.4 also
put files and, where appropriate, changes that have occurred allows for the use of namelists, a standard Fortran feature
since the last publication (Stohl et al., 2005). FLEXPART whereby values are provided in a list with elements of the
needs the following three types of input files. form name=value. When FLEXPART is started, it tries to
open the files as namelists, and if this is not working, it ex-
1. The text file pathnames is located by default in the pects the files to be in one of the two old plain text formats.
directory where FLEXPART is executed. It must con- We encourage users to update their input files to namelists
tain at least four lines: first, the path to the directories for two reasons. Firstly, FLEXPART now has default user op-
where run-defining input files are located (the so-called tions for all input settings so that users only need to set those
options directory); second, the path where output options that they want to deviate from the defaults. Secondly,

www.geosci-model-dev.net/12/4955/2019/ Geosci. Model Dev., 12, 4955–4997, 2019

4974 I. Pisso et al.: FLEXPART version 10.4

Table 7. Alphabetical list of the run-defining input files (upper part) and static input files (lower parts), usually contained in a directory called
options. Processing of files marked with ∗ depends on the run specifications. The other files are always read in.

File name Content

AGECLASSES∗ Age class definitions
COMMAND Main control parameters
OUTGRID Output grid definition
OUTGRID_NEST∗ Nested output grid definition
RECEPTORS∗ Receptor locations for receptor kernel output
RELEASES Specification of the sources (forward run) or receptors (backward run)
SPECIES/ Directory containing files with definitions of physical and chemical parameters of species referenced in RELEASES
IGBP_int1.dat Land cover input data
surfdata.t Roughness length, leaf area index for different land cover types
surfdepo.t Seasonal surface resistances for different land cover types
OH_variables.bin∗ OH field

namelists make it easier to add new user options, which may 5.1.2 File RELEASES
be required in future versions of FLEXPART. Thus, plain text
input files may not be supported in future versions of FLEX- The RELEASES file contains information related to when
PART. Examples for all formats of the user input files are and where the particles are introduced in the simula-
contained in the FLEXPART distribution. tion and other properties of the release points (e.g., the
To convert user input files of any format to namelist for- chemical species simulated). It consists of a namelist
mat, the switch nmlout=.TRUE. (in file com_mod.f90) &RELEASES_CTRL that specifies header information. The
must be set before compilation. Then, run-defining user in- header gives the total number of different species (i.e.,
put files are written out in namelist format in the output different substances) to be released, followed by a corre-
directory, with the appendix .namelist added to the input sponding list of the FLEXPART species numbers nnn, and
file name (e.g., COMMAND.namelist). This feature also SPECIES_nnn files define the species’ physical properties
improves the traceability of FLEXPART model results and (see Sect. 5.1.3). Following the header, there is an arbitrary
makes simulations easily reproducible by documenting the number of namelists &RELEASE defining each release. For
settings used for the model run. each such release, the following is given: the starting and
In the following, we provide reference tables of the run- ending time, the location and extension, the masses released
defining user input files including default settings (in bold) (one value for each released species), and the number of par-
when using the namelist format. Notice that the default val- ticles to be released, as well as a comment string. The content
ues are appropriate for regional-scale simulations, but simu- of the RELEASES file is summarized in Table 9.
lations on smaller scales or with higher accuracy may need
adjustments (in particular, shorter time steps and the use of
the new CBL scheme). 5.1.3 SPECIES files

5.1.1 File COMMAND The subdirectory options/SPECIES/ needs to con-

tain one or more files named SPECIES_nnn. For each
The COMMAND file contains the user settings controlling species nnn listed in the header section of the RELEASES
the simulation and the behavior of the run. The default file, such a SPECIES_nnn file must exist. The parame-
COMMAND file contains a namelist &COMMAND, for which ters in the SPECIES_nnn file, contained in the namelist
Table 8 provides a complete listing of all settings with their &SPECIES_PARAMS, set the species name and define the
meaning and preset default values. It is important that users physicochemical properties of the species; they are described
of previous FLEXPART versions who choose to use plain in Table 10. These are important for simulating radioactive or
text input files update their COMMAND file, since new param- chemical decay, wet deposition (scavenging) for gases and
eters have been added. However, the cblflag (and any op- aerosols, dry deposition for gases and aerosols, particle set-
tion added afterwards) must be provided in namelist format tling, and chemical reaction with the OH radical. Some pa-
in any case. rameters are only necessary for gas tracers and some are only
necessary for aerosol tracers; thus, a namelist does not need
to contain all parameters for both gases and particles. Option-
ally, since FLEXPART version 6.0, information about tem-
poral emission variations can be added at the end of the file.

Geosci. Model Dev., 12, 4955–4997, 2019 www.geosci-model-dev.net/12/4955/2019/

I. Pisso et al.: FLEXPART version 10.4 4975

Table 8. Contents of the user input file COMMAND. Variable names with their meaning and all possible values are listed. Where appropriate,
default values are given in bold. Note that not all input parameter combinations are allowed.

Variable name Description Value (default)

1 LDIRECT Simulation direction in time 1 (forward) or −1 (backward)
2 IBDATE Start date of the simulation YYYYMMDD: YYYY=year, MM=month,
DD=day
3 IBTIME Start time of the simulation HHMISS: HH hours, MI=minutes,
SS=seconds. UTC zone.
4 IEDATE End date of the simulation Same format as IBDATE
5 IETIME End time of the simulation Same format as IBTIME
6 LOUTSTEP Interval of model output Average concentrations are calculated every
LOUTSTEP (10 800 s)
7 LOUTAVER Concentration averaging interval, instantaneous (10 800 s)
for value of zero
8 LOUTSAMPLE Numerical sampling rate of output, higher sta- (900 s)
tistical accuracy with shorter intervals
9 ITSPLIT Time constant for particle splitting (particles are (999 999 999 s)
split in two after a given time)
10 LSYNCTIME All processes are synchronized to this time in- (900 s)
terval; it has to divide all values above
11 CTL Factor by which particle transport time step in > 1 for time steps shorter than tl ; if CTL < 0, a
the ABL must be smaller than the Lagrangian purely random walk simulation is done (–5.0)
timescale tl ; resulting time steps can be shorter
than LSYNCTIME; LSYNCTIME is used if
CTL < 0
12 IFINE Additional reduction factor for time step used Positive integer (4)
for vertical transport only considered if CTL >
1
13 IOUT Switch determining the output type (1) mass concentration (residence time back-
wards),
2 volume mixing ratio,
3 both 1 and 2,
4 plume trajectories,
5 both 1 and 4.
Add 8 for NetCDF output
14 IPOUT Switch for particle position output (0) no particle output,
1 particle output every output interval,
2 only at the end of the simulation (useful, e.g.,
for warm start)
15 LSUBGRID Increase in ABL heights due to subgrid-scale (0)=off, 1=on
orographic variations
16 LCONVECTION Switch for convection parameterization 0=off, (1)=on
17 LAGESPECTRA Switch for calculation of age spectra (needs file (0)=off, 1=on
AGECLASSES)
18 IPIN Warm start simulation, restarted from a particle (0)=no, 1=yes
dump (needs partposit_end file from pre-
vious simulation)
19 IOER Separate output fields for each location in the (0)=no, 1=yes
RELEASE file
20 IFLUX Output of mass fluxes through output grid box (0)=off, 1=on
boundaries (northward, southward, eastward,
westward, upward and downward)

www.geosci-model-dev.net/12/4955/2019/ Geosci. Model Dev., 12, 4955–4997, 2019

4976 I. Pisso et al.: FLEXPART version 10.4

Table 8. Continued.

21 MDOMAINFILL Switch for domain-filling calculations: particles (0)=no, 1 like air density, 2 stratospheric ozone
are initialized to reproduce air density or strato- tracer
spheric ozone density; for limited-area simu-
lations, particles are generated at the domain
boundaries
22 IND_SOURCE Unit to be used at the source; see Seibert and (1)=mass, 2=mass mixing ratio
Frank (2004); Eckhardt et al. (2017)
23 IND_RECEPTOR Unit to be used at the receptor; see Seibert and (1)=mass, 2=mass mixing ratio, 3=bwd. wet.
Frank (2004); Eckhardt et al. (2017) dep., 4=bwd. dry. dep.
24 MQUASILAG Quasi-Lagrangian mode to track individual (0)=off, 1=on
numbered particles
25 NESTED_OUTPUT Switch to produce output also for a nested do- (0)=no, 1=yes
main
26 LINIT_COND Switch to produce output sensitivity to initial (0)=no, 1=mass concentration, 2=mass mixing
conditions given in concentration or mixing ra- ratio
tio units (in backwards mode only)
27 SURF_ONLY Output of SRR for fluxes only for the low- (0)=no, 1=yes
est model layer, most useful for backward runs
when LINIT_COND set to 1 or 2
28 CBLFLAG Skewed rather than Gaussian turbulence in the (0)=no, 1=yes
convective ABL; when turned on, very short
time steps should be used (see CTL and IFINE)
29 OHFIELDS_PATH Default path for OH file
30 d_trop Tropospheric horizontal turbulent diffusivity (50 m2 s-1 )
Dh
31 d_strat Stratospheric vertical turbulent diffusivity Dz (0.1 m2 s-1 )

Table 9. Contents of the user input file RELEASES.

Variable name Description Format, valid values, variable type

Header (written only once and valid for all releases)
NSPEC Total number of species Integer number
SPECNUM_REL Species numbers in dir. SPECIES Integer array of size NSPEC
For each release
IDATE1 Release start date YYYYMMDD: YYYY=year, MM=month, DD=day
ITIME1 Release start time in UTC HHMISS: HH hours, MI=minutes, SS=seconds; integer
IDATE2 Release end date Same format as IDATE1
ITIME2 Release end time in UTC Same format as ITIME1
LON1 Left longitude of release box −180 < LON1 < 180, or according to input winds; real
LON2 Right longitude of release box Same format as LON1; real
LAT1 Lower latitude of release box −90 < LAT1 < 90, or according to input winds; real
LAT2 Upper latitude of release box Same format as LAT1; real
ZKIND Reference level 1: meters above ground, 2: meters above sea level, 3: pressure (hPa); integer
Z1 Lower height of release box Meters above reference level; real
Z2 Upper height of release box Meters above reference level; real
PARTS Total number of particles to be released Integer ≥ 1
For each species (NSPEC times)
MASS Total mass emitted in, e.g., kilograms or unitless for mixing ratio; 1 in backward mode; real
COMMENT Comment 40-character string (e.g., name of release point)

Geosci. Model Dev., 12, 4955–4997, 2019 www.geosci-model-dev.net/12/4955/2019/

I. Pisso et al.: FLEXPART version 10.4 4977

Notice that the format of the SPECIES_nnn files has further explanation, see Stohl et al. (1995). Example files can
changed from previous FLEXPART versions and users need be found in the options directory in the FLEXPART dis-
to update their files accordingly. The use of SPECIES_nnn tribution.
files from older FLEXPART versions may lead to run time
errors or erroneous results. 5.1.5 File OUTGRID_NEST
The following specifies the parameters associated with
each physicochemical process simulated. Output can also be produced on one nested output grid
with higher horizontal resolution, defined in the file
– Radioactive or chemical decay: set with pdecay; off if OUTGRID_NEST, but with the same vertical resolution as
pdecay<0. given in OUTGRID. The OUTGRID_NEST file contains a
namelist &OUTGRIDN specifying all parameters.
– Wet deposition for gases: set with pweta_gas,
pwetb_gas (for below-cloud) and phenry (for in- 5.1.6 File AGECLASSES
cloud). Switch off for both in- and below-cloud if either
pweta_gas or pwetb_gas is negative. The option to produce age class output can be activated in the
– Wet deposition for aerosols: set with pccn_aero, COMMAND file. The file AGECLASSES then allows for the
pin_aero for in-cloud scavenging and definition of a list of times (in seconds, in increasing order)
pcrain_aero, pcsnow_aero and pdquer that define the age classes used for model output. With this
for below-cloud scavenging. option, the model output (e.g., concentrations) is split into
contributions from particles of different age, defined as the
– Dry deposition for aerosols: set with pdensity, time passed since the particle release. Particles are dropped
pdquer and psigma; off if pdensity< 0. from the simulation once they exceed the maximum age, al-
lowing their storage locations to be reused for new particles.
– Dry deposition for gases: set with phenry, pf0 and This is an important technique to limit the memory usage for
preldiff; off if preldiff< 0. Alternatively, a con- long-term simulations. Thus, even if the user is not interested
stant dry deposition velocity pdryvel can be given. in age information per se, it may often be useful to set one
– Settling of particles: set with pdensity and pdquer. age class to define a maximum particle age.

– OH reaction: chemical reaction with the OH radical can 5.1.7 File RECEPTORS
be turned on by giving parameter pohcconst
(cm3 molecule−1 s−1 ), pohdconst (K) and In addition to gridded model output, it is also possible to de-
pohnconst (no unit) positive values; defined by fine receptor points. With this option output can be specifi-
Eq. (13). cally produced for certain points at the surface in addition to
gridded output. The RECEPTORS file contains a list with the
– Emission variation: emission variation during the hours definitions of the receptor name, longitude and latitude. If no
(local time) of the day and during the days of the week such file is present, no receptors are written to output.
can be specified. Factors should be 1.0 on average to ob-
tain unbiased emissions overall. The area source factors 5.1.8 Static data input files
(useful, e.g., for traffic emissions) are applied to emis-
sions with a lower release height below 0.5 m above Several files contain static input data that are not usu-
ground level (a.g.l.) and the point source factors (use- ally modified by the user. These are (by default) also lo-
ful, e.g., for power plant emissions) to emissions with a cated in the options directory. If modeling a species
lower release height than 0.5 m a.g.l. Default values are requires calculating OH reactions, an OH field stored
1.0. in file OH_variables.bin needs to be present. The
file IGBP_int1.dat is a land cover inventory; file
5.1.4 File OUTGRID surfdata.t gives the roughness length and leaf area in-
dex of the different land cover types, and file surfdepo.t
The OUTGRID file specifies the domain and grid spacing of contains surface resistances for dry deposition calculations.
the three-dimensional output grid. Note that in a Lagrangian
model, the domain and resolution of the gridded output are 5.2 Meteorological data and preprocessing routines
totally independent from those of the meteorological input
(apart from the fact that the output domain must be contained FLEXPART can be run with meteorological input data for
within the computational domain). The OUTGRID file con- global domains or for smaller, limited-area domains. The
tains a namelist &OUTGRID specifying all parameters. The FLEXPART computational domain always corresponds to
variables read in for this file and all the following input files this mother domain set by the input data, while the out-
have not changed in recent FLEXPART versions; thus, for put domain can be smaller. FLEXPART can also ingest

www.geosci-model-dev.net/12/4955/2019/ Geosci. Model Dev., 12, 4955–4997, 2019

4978 I. Pisso et al.: FLEXPART version 10.4

Table 10. FLEXPART variables set in the user input file SPECIES_nnn for species number nnn. Note that the variable names given in the
input namelist are the same as used subsequently in FLEXPART but with a prepended letter p (for parameter). For instance, pspecies
corresponds to species.

Variable name Description Unit Type

1 pspecies Tracer name no units string
2 pdecay Species half-life for radioactive or chemical decay; off if pdecay<0 seconds real
3 pweta_gas Gases wet deposition: below-cloud scavenging parameter A (for precip. of 1 s−1 real
1 mm h−1 )
4 pwetb_gas Gases wet deposition: below-cloud scavenging parameter B (dependency on 1 real
precip. rate)
5 pcrain_aero Aerosols wet deposition: below-cloud scavenging rain collection efficiency 1 real
moderator for rain Crain
6 pcsnow_aero Aerosols wet deposition: below-cloud scavenging rain collection efficiency 1 real
moderator for snow Csnow
7 pccn_aero Aerosols wet deposition: in-cloud scavenging, cloud condensation nuclei effi- 1 real
ciency CCNeff
8 pin_aero Aerosols wet deposition: in-cloud scavenging, ice nuclei efficiency INeff 1 real
9 preldiff Gases dry deposition: ratio D = DH2 O /Di , DH2 O of the diffusivity of H2 O 1 real
to the diffusivity of the component Di (the diffusivity of the species in the
SPECIES_nnn file); dry deposition of gases is switched off by negative D
10 phenry Gases dry deposition and in-cloud scavenging: Henry’s constant H M atm−1 real
11 pf0 Gases dry deposition: reactivity factor for oxidation of biological substances 1 real
relative to that of ozone; (0 ≤ f0 ≤ 1)
For nonreactive species f0 is 0, for slightly reactive species it is 0.1 and for
highly reactive species it is 1
12 pdensity Aerosols dry deposition and settling: particle density ρ kg m−3 real
13 pdquer Aerosol dry deposition, aerosol wet deposition: below-cloud scavenging: parti- m real
cle mean diameter d̄
Decides whether its gas(<=0) or aerosol (> 0)
14 psigma Aerosol dry deposition: species diameter normalized deviation (σ > 1) 1 real
15 pdryvel Gases dry deposition: dry deposition velocity (only used if preldiff and m s−1 real
pdensity < 0 )
16 pweightmolar Gases: species molar weight, used for volume mixing ratio (pptv) output g mol−1 real
17 pohcconst Gases OH reaction: C cm3 molecule s−1 real
18 pohdconst Gases OH reaction: D K real
19 pohnconst Gases OH reaction: N 1 real
20 parea_hour Emission variation factor (area source) for hour of the day, starting with 00:00– 1 real
01:00 local time, 24 values
Local time from longitude, no correction for summer time
21 parea_dow Emission variation factor (area source) for day of the week, starting with Mon- 1 real
day, 7 values
22 ppoint_hour Emission variation factor (point source) for hour of the day, starting with 00:00– 1 real
01:00 local time, 24 values
Local time from longitude, no correction for summer time
23 ppoint_dow Emission variation factor (point source) for day of the week, starting with Mon- 1 real
day, 7 values

higher-resolution meteorological input data in subdomains able from the FLEXPART website (https://flexpart.eu/wiki/
of the mother domain. Such nested data must be avail- FpInputMetMkavail, last access: 25 June 2018) that checks
able for the exact same times as those for the mother which input files are present and then creates this file in the
domain, checked by FLEXPART by comparing the time required format.
stamps in the two AVAILABLE(_NESTED) files. There Compilation of FLEXPART v10.4 produces a single ex-
is no nesting in the vertical direction and the poles must ecutable that automatically detects whether the meteorolog-
not be contained in any nest. To automatically produce the ical input data come from the ECMWF IFS or NCEP GFS
AVAILABLE(_NESTED) files, a Python script is avail- and whether they are in GRIB-1 or in GRIB-2 format. Nev-

Geosci. Model Dev., 12, 4955–4997, 2019 www.geosci-model-dev.net/12/4955/2019/

I. Pisso et al.: FLEXPART version 10.4 4979

ertheless, certain parameters may need to be adapted in some of the most important ones for driving FLEXPART.
par_mod.f90 to the size of the meteorological input files These are, in particular, the reanalysis datasets ERA-Interim
(array dimensions), and the input grid may need to be shifted (Dee et al., 2011), CERA-20C (the coupled climate reanal-
relative to the output grid (parameter nxshift). In the fol- ysis of the 20th century; Laloyaux et al., 2018) and the
lowing, we describe how meteorological input data appropri- latest reanalysis ERA5, as well as data from the opera-
ate for FLEXPART can be retrieved from the ECMWF and tional IFS stream. Each file (one for each time step) pre-
NCEP. pared by flex_extract for FLEXPART consists of a set
of model-level and surface data as a combination of anal-
5.2.1 ECMWF data retrieval ysis and forecast fields depending on availability. For ex-
ample, certain variables such as precipitation may only be
ECMWF data can be comprised of analysis and/or forecast available in forecast fields, whereas other data are also con-
data from the operational IFS data stream or specific re- tained in analysis fields; flex_extract seeks an optimum
analysis projects. For operational data, the meteorological combination of such data. Note that some parameters are
fields can currently have a maximal temporal resolution of stored as time-accumulated fields in the ECMWF archives
1 h (more frequent data are not available), a vertical resolu- and flex_extract calculates the instantaneous fluxes out
tion of 137 model levels and 0.1◦ ×0.1◦ horizontal resolution of them (e.g., precipitation fluxes). For more details on this
on a regular latitude–longitude grid. Other ECMWF datasets process of de-accumulation, see Hittmeir et al. (2018). Since
are not available at such high horizontal resolution. For ex- FLEXPART needs the pressure hybrid coordinate vertical ve-
ample, ERA-Interim reanalysis data (Dee et al., 2011) with locity as used in the ECMWF model, an important feature
1◦ × 1◦ latitude–longitude resolution and 60 vertical levels of flex_extract is the computation of this parameter
can be retrieved 3-hourly by mixing 6-hourly analysis and from the horizontal wind field (see Stohl et al., 2001) for
3 h forecast fields, but higher resolution is not available from ERA-Interim and for the years when it was not operationally
the standard archive. The new Copernicus reanalysis ERA5 archived in MARS (before 2009).
provides 1-hourly analysis fields with 137 model levels and a The ECMWF is a European intergovernmental organi-
horizontal resolution of 31 km (0.28125◦ ). Notice that access zation that grants full access to its multi-petabyte MARS
to some datasets, in particular the operational forecasts, is re- archive for their member and cooperating states. Users with
stricted and requires specific access (https://www.ecmwf.int/ a full-access account can run flex_extract v7.0.4 di-
en/forecasts/accessing-forecasts, last access: 23 June 2018). rectly on ECMWF servers or via a local gateway server. This
However, reanalysis data (https://software.ecmwf.int/wiki/ mode is also required to retrieve the most recent operational
display/WEBAPI/Available+ECMWF+Public+Datasets, last data from the ECMWF. Users from member or cooperat-
access: 23 June 2018) are publicly available. ing states interested in this mode should contact the com-
The IFS is a global model that uses spectral representa- puting representative from their national meteorological ser-
tion with spherical harmonics for the dynamical part and a vice to obtain an account. Users from other countries world-
grid-point representation on a reduced Gaussian grid for the wide can self-register at the ECMWF for a public account
physical part. However, FLEXPART needs the input data on to be able to retrieve the public datasets (i.e., most reanal-
a regular latitude–longitude grid, and thus IFS data have to be ysis products); flex_extract v7.0.4 makes use of the
preprocessed. With respect to the vertical coordinate system, WebAPI (https://software.ecmwf.int/wiki/display/WEBAPI/
the data need to be on the native ECMWF model levels (η ECMWF+Web+API+Home, last access: 24 June 2018) tool
levels), which are subsequently transformed within FLEX- provided by the ECMWF to access the data from outside
PART to a terrain-following vertical coordinate system. their systems. This tool can distinguish between public and
As explained above, each ECMWF dataset has its own member state users. Therefore, it is also a convenient option
specific temporal and spatial resolution, and the meteoro- for member and cooperating state users who only need data
logical parameters provided can be different from dataset to older than a few days from the operational stream or reanal-
dataset. To produce meteorological GRIB files suitable for ysis data. A full-access account to ECMWF servers is no
FLEXPART input from these different datasets, a software longer needed in this case.
called flex_extract (current version 7.0.4) has been de- The flex_extract software v7.0.4 is a set of Python
veloped specifically for this purpose. In order to prepare routines combined with a Fortran program for faster com-
the GRIB files from the ECMWF Meteorological Archival putation of grid transformations and vertical velocity calcu-
and Retrieval System (MARS; https://software.ecmwf.int/ lation. A Python 2.7 interpreter with several common mod-
wiki/download/attachments/45759146/mars.pdf, last access: ules, such as NumPy and date–time, are required and usu-
24 June 2018), several retrieval requests using the MARS ally included in the Anaconda distribution (https://www.
command language and some further processing steps are anaconda.com/download/, last access: 25 June 2018). Ad-
needed. Since all ECMWF datasets need to be handled dif- ditionally, a Fortran compiler, the ECMWF WebAPI tool,
ferently and some may not even contain all information the GRIB-API or ecCodes module, and the Emoslib inter-
needed for FLEXPART, flex_extract has a focus on polation library have to be available. Note that the GRIB-

www.geosci-model-dev.net/12/4955/2019/ Geosci. Model Dev., 12, 4955–4997, 2019

4980 I. Pisso et al.: FLEXPART version 10.4

API (or ecCodes) module has to be available for Python as analysis datasets are also available from NCEP, e.g., the Cli-
well as for Fortran. Installation instructions can be found mate Forecast System Reanalysis (CFSR) Selected Hourly
at ECMWF websites directly or in the Software Installation Time-Series Products (http://rda.ucar.edu/datasets/ds093.1/,
Plan for flex_extract. Knowledge of Python, although last access: 30 October 2019) for the period January 1979
helpful, is not necessary for using the retrieval scripts. A to March 2011.
certain knowledge of the ECMWF dataset to be retrieved
is useful to understand the composition of retrievals, but
many basic examples of CONTROL files are provided in the 6 FLEXPART output
flex_extract distribution. These CONTROL files deter-
mine the key parameters for the flex_extract MARS 6.1 Output files overview
retrievals and can be adapted to change domain as well as
spatial and temporal resolution. Even for these few parame- In the following we describe the FLEXPART output files to-
ters the user should check for availability upfront. For exam- gether with changes made since the last documented FLEX-
ple, ERA-Interim data have a maximum grid resolution of PART version (Stohl et al., 2005). An overview of all pos-
0.75◦ × 0.75◦ and 6-hourly temporal resolution for the pub- sible output files is provided in Table 11. Notice that not all
lic dataset. these files are written out in every model run; the user set-
The flex_extract v7.0.4 software is included in tings control which files are produced. At the beginning of a
the FLEXPART v10.4 file tree under the directory run, FLEXPART records descriptive metadata in the binary
preprocess (see Table 1). It can also be downloaded from file header. This information is also written into the plain
https://flexpart.eu/ (last access: 30 October 2019) (https: text files header_txt (with the exception of the orogra-
//flexpart.eu/downloads/62, last access: 30 October 2019) phy data and release information). The release information is
as a tarball. For more details the reader is referred to the written in header_txt_releases. Corresponding files
flex_extract v7.0.4 user documentation (e.g., the Soft- header_nest are produced if nested output is selected.
ware Installation Plan – SIP.pdf – and the Software User Tu- At each output time, FLEXPART produces files con-
torial – SUT.pdf) in preprocess/flex_extract. taining the gridded output. Separate files are created
for every species and domain (mother and, if re-
5.2.2 NCEP data retrieval quested, nest). The naming convention for these files is
grid_type_date_nnn. For forward runs, type can be
Meteorological data from the NCEP GFS are freely avail- conc or pptv for concentrations and mixing ratios or flux
able, easily accessible and ingested by FLEXPART on pres- for 3-D mass fluxes across the grid cell faces (Stohl et al.,
sure levels, unlike ECMWF data. These pressure-level data 2005, Sect. 8.5). For backward runs, type can be time for
have lower resolution than model-level data but offer the the sensitivity of receptor concentrations to emission fluxes,
advantage of great consistency between different datasets. drydep for the sensitivity of receptor dry deposition to
Therefore, preprocessing NCEP data is much simpler than emissions or wetdep for the sensitivity of receptor wet de-
ECMWF data and limited to precipitation data, which are position to emissions. For backward runs, there can also be
available only in forecast fields. an output file grid_initial_nnn, which gives the re-
Both operational analysis data and several reanalysis ceptor sensitivity to initial conditions; date denotes the date
datasets are available. Notice that NCEP also provides fore- and time for which the output is valid, and nnn is the species
cast data for free, which are not available from the ECMWF number as specified in RELEASES. The list of the output
even for member state users except for national meteorologi- times is progressively written to the text file dates. For the
cal services or users with a special contract. The data retrieval nested output, grid is replaced by grid_nest.
from NCEP is described in a wiki page on the FLEXPART Wet and dry deposition fields in forward runs are calcu-
website (https://flexpart.eu/wiki/FpInputMetGfs, last access: lated on the same horizontal output grid and are appended to
8 July 2018), where a script for downloading NCEP data can grid_conc_date_nnn and grid_pptv_date_nnn
also be found. Operational GFS data can be downloaded by files. The deposited matter is accumulated over the course
simple FTP or wget from a rolling archive of the meteoro- of a model run. It generally increases with model time, but
logical forecast and analysis data (http://www.ftp.ncep.noaa. for species with radioactive decay, losses are possible. As for
gov/data/nccf/com/gfs/prod/, last access: 8 July 2018) un- long simulations small deposition amounts may be added to
der the catalog gfs.YYYYMMDDHH, which contains fields in already large deposited quantities, the default precision of the
GRIB-2 format. Six-hourly NCEP FNL (Final) Operational deposition fields was changed from single (in older FLEX-
Model Global Tropospheric Analyses (http://rda.ucar.edu/ PART versions) to double precision to avoid numerical in-
datasets/ds083.2/, last access: 30 October 2019) are avail- accuracies when deriving instantaneous fluxes from accumu-
able in near-real time since July 1999. These data are sim- lated quantities.
ilar to the operational analyses, but NCEP also ingests late- For a list of points at the surface, concentrations or mixing
incoming observation data for their production. Archived re- ratios in forward simulations can also be calculated indepen-

Geosci. Model Dev., 12, 4955–4997, 2019 www.geosci-model-dev.net/12/4955/2019/

 Table 11. List of FLEXPART output files and for which user settings (“switches”) they are produced. See Table 12 for details on the units in the gridded output.

Name Format Switches Description of contents

header binary default output run metadata + ancillary data
header_txt text default output human-readable run metadata (from COMMAND)
header_txt_releases text default output human-readable run metadata (from RELEASES)
dates text default output time series: dates of output files
grid_conc_date_nnn binary (sparse array) LDIRECT=1 IOUT=1,3,5 3-D tracer mass density + 2-D deposition
grid_pptv_date_nnn binary (sparse array) LDIRECT=1 IOUT=2,3 3-D tracer volume mixing ratio + 2-D deposition
grid_time_date_nnn binary (sparse array) LDIRECT=-1 IOUT=1 3-D sensitivity of atmospheric receptor to emissions
grid_drydep_date_nnn binary (sparse array) LDIRECT=-1 IOUT=1, IND_RECEPTOR=3 3-D sensitivity of dry deposition receptor to emissions
I. Pisso et al.: FLEXPART version 10.4

www.geosci-model-dev.net/12/4955/2019/
grid_wetdep_date_nnn binary (sparse array) LDIRECT=-1 IOUT=1, IND_RECEPTOR=4 3-D sensitivity of wet deposition receptor to emissions
grid_conc_date_nnn.nc binary (NetCDF) LDIRECT=1 IOUT=9,11,13 3-D tracer + 2-D wet and dry deposition
grid_time_date_nnn.nc binary (NetCDF) LDIRECT=-1 IOUT=9 3-D sensitivity of atmospheric receptor to emissions
grid_drydep_date_nnn.nc binary (NetCDF) LDIRECT=-1 IOUT=9, IND_RECEPTOR=3 3-D sensitivity of dry deposition receptor to emissions
grid_wetdep_date_nnn.nc binary (NetCDF) LDIRECT=-1 IOUT=9, IND_RECEPTOR=4 3-D sensitivity of wet deposition receptor to emissions
grid_initial_nnn binary (sparse array) LDIRECT=-1, LINIT_COND>0 3-D sensitivity of receptor concentrations and deposi-
tion to initial conditions
partposit_date binary IPOUT=1,2 particle positions and meteorological data
IPOUT=1,2 MQUASILAG=1 particle positions and meteorological data numbered
consecutively
partposit_average_date binary IPOUT=3 time-averaged particle positions and meteorological
data
trajectories.txt text IOUT=4, 5 clustered trajectories
receptor_conc binary LDIRECT=1 IOUT=1,3,5,9,11,13 mass density at receptors
receptor_pptv binary LDIRECT=1 IOUT=2,3,10,11 volume mixing ratio at receptors
header_nest binary NESTED_OUTPUT=1 nest metadata + ancillary data
grid_conc_nest_date_nnn binary (sparse array)
grid_pptv_nest_date_nnn binary (sparse array) as for mother grid as for mother grid in a
grid_time_nest_date_nnn binary (sparse array) + NESTED_OUTPUT=1 higher-resolution latitude–longitude grid
grid_drydep_nest_date_nnn binary (sparse array)
grid_wetdep_nest_date_nnn binary (sparse array)
grid_conc_nest_date_nnn.nc binary (NetCDF)
grid_time_nest_date_nnn.nc binary (NetCDF)
grid_drydep_nest_date_nnn.nc binary (NetCDF)
grid_wetdep_nest_date_nnn.nc binary (NetCDF)

Geosci. Model Dev., 12, 4955–4997, 2019

4981
4982 I. Pisso et al.: FLEXPART version 10.4

dently from the grid using a kernel method and recorded in ray, sparse_dump_i, using a one-dimensional represen-
the files receptor_conc and/or receptor_pptv. tation of the output field. The total number of nonzero val-
If the particle dump option is activated, in addition to ues is stored in sp_count_r and the nonzero values them-
the gridded output, the particle coordinates together with selves in the real vector sparse_dump_r. Since all physi-
additional variables such as pressure, humidity, density, cal output quantities of FLEXPART are greater than or equal
tropopause height, ABL height and orography height are to zero, nonzero sequences are stored in sparse_dump_r
recorded in the binary files partposit_date. These with alternating signs, which allows for the separation of dif-
data can be useful for a variety of different purposes, ferent sequences upon reading. Finally, all four variables are
for instance diagnostics of the water cycle (Stohl and written out to the unformatted output file. This format re-
James, 2004). FLEXPART version 10.4 also has the places the compression used up to version 7 (the smallest of
new option to write out time-averaged particle positions a full dump and a simple sparse matrix format), saving up
and meteorological data. These are recorded in the files to 60 % of disk space. The sparse matrix data can be read,
partposit_average_date. Such output may be useful for example, with the functions readgrid.f (Fortran) and
to obtain, for instance, more representative heights for parti- flex_read.m (MATLAB) described in Sect. 6.4
cles in the boundary layer, where particle positions change
rapidly and this is not sampled sufficiently with instanta- 6.3 NetCDF output
neous output. If plume trajectory mode is activated, for ev-
ery release the positions of trajectory clusters representing FLEXPART v10.4 can also support output in NetCDF
the centers of mass of all released particles are recorded format if the NetCDF libraries are available. To activate
in the file trajectories.txt (Stohl et al., 2002, 2005, NetCDF support, append ncf=yes to the make command.
Sect. 10). If FLEXPART is compiled and linked to the NetCDF li-
The physical unit used for the output data in the files braries, output files in NetCDF format can be produced
grid_conc_date_nnn and grid_time_date_nnn by adding 8 to the IOUT parameter in the input file
depends on the settings of the switches ind_source COMMAND, e.g., IOUT=9 corresponds to IOUT=1 with the
and ind_receptor, following Table 12. It is notewor- standard binary output; see Table 11 and Sect. 5.1.1. In the
thy that the unit of mass mixing ratio can also be used in NetCDF module netcdf_output_mod.f90 a parame-
grid_conc_date_nnn. For forward runs, additional files ter write_releases determines at compile time if the
grid_pptv_date_nnn can be created (setting IOUT to information on the releases should also be written to the
values of 2 or 3), which contain data such as volume mix- NetCDF file. Only one NetCDF file is written that contains
ing ratios (requires molar weight in SPECIES_nnn file). all species and all time steps. Both mother and nested output
Source–receptor relationships (i.e., emission sensitivities) (if present) are contained in that file. Since the NetCDF out-
in backward mode for atmospheric receptors are written put is specified in the climate and forecast (CF) format, any
out in grid_time_date_nnn files; those for deposited standard software can be used for displaying and processing
mass are recorded in files grid_wetdep_date_nnn and the output (e.g., panoply, ncview). NetCDF output data files
grid_drydep_date_nnn (see Seibert and Frank (2004), are compressed.
Eckhardt et al. (2017), Sect. 2.5, and Table 12 for output The NetCDF output file contains information on the run
units). Notice that the user can also provide different input settings and the simulation grid from the COMMAND and
units. For instance, if emissions in a forward run are spec- OUTGRID* files. It also contains additional information in
ified in Becquerels (Bq), the output would be in nanobec- the header on the producing center, as listed in Table 13.
querels per cubic meter (nBq m−3 ) with ind_source=1 The content of these attributes can be adapted in the file
and ind_receptor=1. Notice further that all gridded out- netcdf_output_mod.f90 before compilation.
put quantities in FLEXPART are grid cell averages, not point
values. 6.4 Post-processing routines

6.2 Sparse matrix output For the NetCDF output of FLEXPART, standard visualiza-
tion tools, for example Panoply, can be used. For the sparse
Depending on the type of model run, the gridded output matrix binary output, several post-processing routines (MAT-
can contain many grid cells with zero values (e.g., disper- LAB, Fortran, R, Python and IDL) have been developed in
sion from a point source, backward run from a single re- order to assist in the usage and analysis of these data. A
ceptor). The output is therefore written in a sparse matrix number of post-processing tools are available online (https:
format, which is specific to FLEXPART. The array contain- //flexpart.eu/wiki/FpOutput, last access: 16 August 2018).
ing the data to be written out is scanned for sequences of Note that some of these tools require reading a text string
nonzero values. The number of sequences found is stored in containing the model version. Since the length of this string
an integer variable sp_count_i, and the field positions at changed in FLEXPART v9.2, the post-processing routines
which each sequence begins are stored in a 1-D integer ar- now require the allocation of a longer string.

Geosci. Model Dev., 12, 4955–4997, 2019 www.geosci-model-dev.net/12/4955/2019/

I. Pisso et al.: FLEXPART version 10.4 4983

Table 12. Physical units of the input (in file RELEASES) and output data for forward (files grid_conc_date_nnn) and backward (files
grid_time_date_nnn) runs for the various settings of the unit switches ind_source and ind_receptor (for both switches, 1
refers to mass units, 2 to mass mixing ratio units). IOUT is 1 (or 9 for NetCDF output) except where indicated; “(dep.)” in lines 5 and 6 of
the table refer to the deposition output provided in addition to the atmospheric output in files grid_conc_date_nnn.

Direction File name ind_source ind_receptor Input unit Output unit

grid_conc* 1 1 kg ng m−3
grid_conc* 1 2 kg ppt by mass
Forward

grid_conc* 2 1 1 ng m−3
grid_conc* 2 2 1 ppt by mass
grid_conc* 1 1 or 2 (dep.) kg ng m−2
grid_conc* 2 1 or 2 (dep.) 1 ng m−2
grid_pptv* (IOUT=2, 3) 1 1 1 ppt by volume
grid_time* 1 1 1 s
grid_time* 1 2 1 s m3 kg−1
grid_time* 2 1 1 s kg m−3
grid_time* 2 2 1 s
Backward

grid_wetdep* 1 3 (wet dep.) 1 m

grid_drydep* 1 4 (dry dep.) 1 m
grid_wetdep* 2 3 (wet dep.) 1 kg m−2
grid_drydep* 2 4 (dry dep.) 1 kg m−2
grid_initial* 1 1 1 1
grid_initial* 1 2 1 m3 kg−1
grid_initial* 2 1 1 kg m−3
grid_initial* 2 2 1 1

Table 13. Additional information in the NetCDF output file as attributes.

Conventions CF-1.6 (NetCDF CF convention identifier)

Title FLEXPART model output (content title)
Institution producer string “institution” set in netcdf_output_mod.f90
Source creation string “flexversion” model output set in FLEXPART.f90
History date string with login and host name
References Stohl et al. (2005)

Fortran routines are available for download on the FLEX- Several Python tools are available for reading FLEXPART
PART website with the subroutines readheader.f for data from release 8.0 and above. The module reflexible,
reading the header and readgrid.f for reading the grid- available from the FLEXPART website and also at https:
ded binary fields. Analysis or plotting programs written in //github.com/spectraphilic/reflexible (last access: 6 August
Fortran can call these subroutines. 2018), enables the user to easily read and access the header
There are also MATLAB tools working in a similar way and grid output data of the FLEXPART model runs. It pro-
as the Fortran routines, with flex_header.m for read- vides a simple tool that facilitates consistent reading of both
ing the header and flex_read.m for reading the data the original sparse matrix output files and the NetCDF out-
fields. If particle dumps were made, the MATLAB function put. Some basic plotting functionality is provided to quickly
readpart.m reads the corresponding data files (a similar assess and validate runs or to look at the input parameters.
Fortran code is also available). An alternative Python tool is Quicklook that can be also
The R programs available for post-processing FLEXPART downloaded from the https://flexpart.eu website.
output include routines to read the binary output in the
grid_conc (or grid_pptv) and grid_time files and
to plot maps. Routines are also available to plot trajectories 7 Application examples
on a map from the file trajectories.txt and to plot
time series of concentrations (or mixing ratios) from the file In this section we provide 38 examples of the FLEXPART
receptor_conc (or receptor_pptv). model that serve three purposes: (1) verification of a new
FLEXPART installation; (2) demonstration of the model ca-

www.geosci-model-dev.net/12/4955/2019/ Geosci. Model Dev., 12, 4955–4997, 2019

4984 I. Pisso et al.: FLEXPART version 10.4

pabilities for new users; and (3) confirmation of consistency

in the model output when code changes are made that should
not change the results. These examples do not represent an
exhaustive set of all possible model uses, but they are de-
signed to demonstrate and test different widely used func-
tionalities of the model.
All examples are variations of a default example case,
which uses the settings in the user input files as distributed
with the FLEXPART v10.4 code package. These default in-
put files are located in the directory options (Sect. 5) and
are consistent with the default meteorological data retrieved
from the ECMWF by the flex_extract package (Ap-
pendices A5 and B1). An AVAILABLE file fitting with these
input data is also distributed with FLEXPART. These default
settings are described in detail in Appendix B2.
Using the default example as a basis, the different func-
tionalities of the model can be activated by adequately chang-
ing certain parameters in the user input files, thereby generat-
ing 36 other example runs. We have categorized these exam- Figure 9. Hypothetical Grímsvötn eruption on 1 April 2015 at
00:00 UTC (instantaneous release). Total column concentrations are
ples into 10 different groups; each group explores different
shown (µg m−2 ) 18 h after the eruption.
capabilities of the model. Table 14 lists all examples and the
parameter changes needed to produce them. The first group
includes the default example and explores the different op- scripts, described in Appendix B3, generate the input files by
tions for producing gridded model output (e.g., output units, modifying the namelists in the default options directory
output formats) for a simple forward model run with a single provided with the distribution. This is done by the bash script
starting point over the North Atlantic. The second group of gen_options_all.sh. For instance, the example “bwd”
examples introduces FLEXPART’s backward simulation ca- is generated by changing the line containing the parameter
pability. The third group demonstrates different usages of the LDIRECT to −1 in the file COMMAND.
particle dump output. The fourth group gives examples for After the input data files are generated, all examples can
the use of mass vs. mass mixing ratio units at both the source be executed interactively from the command line. Alterna-
and the receptor and for both forward and backward simula- tively, the script gen_batch_jobs_cl.sh generates a
tions to establish source–receptor relationships as in Seibert batch script for each case (to be run from the command line
and Frank (2004). The fifth group produces output for differ- or using a workload manager such as SLURM). This proce-
ent chemical species and aerosols. The sixth group illustrates dure automates the sample output generation. Once the out-
the use of nested output fields. Group seven is constituted put files are created, they can be read using the tools in the
by a single domain-filling run, as used, for instance, in Stohl directory postprocess. They can be plotted and analyzed
and James (2004). Group eight contains settings for a back- with, e.g., the reading routines described in Sect. 6.4. In ad-
ward run providing 2-D sensitivities to gridded surface fluxes dition, some testing capabilities have been added. These are
and 3-D sensitivities to initial conditions, as they are typi- presented in Appendix B.
cally required for the inverse modeling of greenhouse gases
(e.g., Thompson et al., 2017). Group nine shows the use of
the new skewed turbulence parameterization (Cassiani et al., 8 Final remarks, outlook and future code development
2015). Group 10 shows the use of the new backward deposi-
tion (Eckhardt et al., 2017). Group 11 contains a forward 2 d In this paper, we have described the Lagrangian particle dis-
run simulating instantaneous emissions from a hypothetical persion model FLEXPART v10.4; 2 decades ago, the model
Grímsvötn eruption (Fig. 9). code was developed mainly by one person, with specific code
The list of examples may be extended in the future to allow input from a few other researchers. At that time, no specific
for the testing of even more model features and to provide a measures were needed to ensure code consistency, track code
reference archive to see how FLEXPART results may change changes or identify coding bugs. However, as the number of
as the code is being developed further. The user can get these FLEXPART users has grown substantially in recent years,
reference results from https://flexpart.eu. A quick reference more and more people have started to develop the code, con-
containing mean and maximum grid values for every exam- tributed code snippets, and reported or identified bugs. The
ple is also referenced in Appendix B5. resulting code changes range from the adaptation of more
The directory tests/examples/ contains scripts that modern coding standards, parallelization and efficiency en-
generate all the files necessary to run the examples. These hancements, the improvement of the model functionality, and

Geosci. Model Dev., 12, 4955–4997, 2019 www.geosci-model-dev.net/12/4955/2019/

I. Pisso et al.: FLEXPART version 10.4 4985

Table 14. List of the test cases for FLEXPART 10.4.

Group Test name Change to default options Description

1 default (IOUT=1) default option: forward run with mass concen-
(1) Gridded output

tration output; see Table 12

2 IOUT=2 mixing ratio
3 IOUT=3 concentration and mixing ratio
5 IOUT=5 concentration and trajectory cluster
9 IOUT=9 mass concentration in NetCDF output format
10 IOUT= 10 mixing ratio in NetCDF output format
11 IOUT= 11 concentration and mixing ratio in NetCDF out-
put format
(2) Backward

bwd LDIRECT= −1 SRR

bwd5 LDIRECT=−1, IOUT=5 backward trajectory cluster
bwd_nc LDIRECT=−1, IOUT=9 SRR in NetCDF output format
(3) Particles

part1 IPOUT= 1 particle dump

part2 IPOUT=2 particle dump at end of simulation
part_bwd1 IPOUT=1, LDIRECT=−1 backward trajectories

ind_1_2 IND_RECEPTOR=2 receptor (gridded) in mass mixing ratio units

ind_2_1 IND_SOURCE=2 source in mass mixing ratio units
(4) Units

ind_2_2 IND_SOURCE=2, IND_RECEPTOR= 2 source and receptor in mass mixing ratio units
bwd_ind_1_2 IND_RECEPTOR=2, LDIRECT=−1 receptor in mass mixing ratio units, backward
bwd_ind_2_1 IND_SOURCE=2, LDIRECT=−1 source in mass mixing ratio units, backward
bwd_ind_2_2 IND_SOURCE=2, IND_RECEPTOR=2, LDIRECT=−1 source and receptor in mixing ratio units, back-
ward
(5) Species

specNO SPECNUM_REL=3 nitric oxide species

specCO SPECNUM_REL=22 carbon monoxide species
specAERO-TRACE SPECNUM_REL=23 idealized aerosol simulation
specBC SPECNUM_REL=40 black carbon simulation
nested NESTED_OUTPUT= 1 nested output
nested_mr NESTED_OUTPUT=1, IOUT=2 volume mixing ratio nested output
(6) Nests

nested_bwd NESTED_OUTPUT=1, LDIRECT=−1 nested output backwards

nested_nc NESTED_OUTPUT=1, IOUT=9 nested output in NetCDF format
nested_mr_nc NESTED_OUTPUT=1, IOUT=10 volume mixing ratio nested output NetCDF
nested_bwd_nc NESTED_OUTPUT=1, LDIRECT=−1, IOUT=9 nested output backwards in NetCDF output for-
mat
(7)

DOMAINFILL MDOMAINFILL= 1 domain-filling run

(8) Init. cond.

init_cond LINIT_COND=1, LDIRECT=−1 sensitivity to initial conditions

init_cond_ind_1_2 LINIT_COND=1, LDIRECT=−1, IND_RECEPTOR=2 sensitivity to initial conditions in mixing ratio
init_cond_surf LINIT_COND=1, SURF_ONLY=1, LDIRECT=−1 sensitivity to initial conditions adapted to sur-
face fluxes
(10) B.d. (9) CBL

CBLFLAG CBLFLAG=1, CTL=40, IFINE=5 skewed turbulence

CBLFLAG_bwd CBLFLAG= 1, LDIRECT=−1, CTL=40, IFINE=5 skewed turbulence backwards

bwd_ind_1_3 IND_RECEPTOR=3, LDIRECT=−1 backward wet deposition

bwd_ind_1_4 IND_RECEPTOR=4, LDIRECT=−1 backward dry deposition
(11)

Volc Modified RELEASES file hypothetical volcanic eruption

www.geosci-model-dev.net/12/4955/2019/ Geosci. Model Dev., 12, 4955–4997, 2019

4986 I. Pisso et al.: FLEXPART version 10.4

the addition of output options, to revisions and extensions of

the model physics. All this has been documented in this pa-
per. Integration of all these changes into a single stable model
version represents a growing challenge in itself, and efforts
to address this challenge (e.g., model website and repository,
version control, testing environment) have also been docu-
mented here.
As FLEXPART is developed further, updates will con-
tinue to be made available on the FLEXPART website at
https://flexpart.eu. We encourage established and new users
to contribute to FLEXPART development by providing their
code changes, as well as a description of these changes, as
new feature branches of the latest commits in the FLEX-
PART git repository. New code should pass all test cases pro-
vided in the FLEXPART distribution and provide consistent
output, unless there are specific reasons why output should be
different, such as improvements in the model physics. This
will expedite the integration of important new model features
in the main development branch of the model.

Code and data availability. The code described

in this work is archived as flexpart10.4.tar at
https://doi.org/10.5281/zenodo.3542278 (Pisso et al., 2019).
FLEXPART downloads are available at https://www.flexpart.eu/
downloads (last access: 22 November 2019) and FLEXPART re-
leases are available at https://git.nilu.no/flexpart/flexpart/-/releases
(last access: 22 November 2019). The working git repos-
itory for this version 10.4 (branches master, dev and re-
lease) can be accessed at https://git.nilu.no/flexpart/flexpart
(last access: 22 November 2019) as well as the mirrors
https://www.flexpart.eu/gitmob/flexpart (last access: 22 Novem-
ber 2019), https://github.com/flexpart/flexpart (last access:
22 November 2019) and https://bitbucket.org/flexpart/flexpart (last
access: 22 November 2019).

Geosci. Model Dev., 12, 4955–4997, 2019 www.geosci-model-dev.net/12/4955/2019/

I. Pisso et al.: FLEXPART version 10.4 4987

Appendix A: Installing FLEXPART and curl https://www.ece.uvic.ca/~frodo/jasper/

flex_extract
software/jasper-1.900.1.zip
Here, we provide step-by-step instructions on how to install --output jasper-1.900.1.zip
FLEXPART on Linux from scratch. This has been tested on sudo apt install unzip
an Ubuntu 16.4 distribution running on a dedicated instance unzip jasper-1.900.1.zip
in the Amazon cloud. Notice that in most environments, cd jasper-1.900.1
some or all of the required libraries (e.g., a Fortran compiler) ./configure
are already installed and an installation totally from scratch make
would thus not be needed. In such cases, we strongly recom- make check
mend that these libraries are used instead of installing every- sudo make install
thing from scratch. However, sometimes it may be necessary
to install them from source (e.g., to avoid incompatibilities Download the grib_api library from the ECMWF website
between different compilers or different versions of the same (https://software.ecmwf.int/wiki/display/GRIB/Home, last
compiler). In the following, we assume that the user has root access: 30 October 2019) and install it.
privileges in the system, but it is also possible for normal
gunzip grib_api-X.X.X.tar.gz
users to install the libraries in nonstandard locations. It is
tar -xf grib_api-X.X.X.tar
possible to ask for help by writing to the FLEXPART user
./configure [--prefix=grib_api_dir]
email list (registration needed) or by creating a ticket on the
[--with_jasper=<jasper installation
community website at https://flexpart.eu.
path>]
A1 System requirements make
make check
To begin, ensure that the latest packages are being used. This make install
section is for completeness only, and most users (if not start-
If you have no root privileges in your system, give the full
ing from a new system installation) can skip it and jump to
path of grib_api_dir to the prefix option. If jasper is in
Sect. A2.
a nonstandard location, is has to be passed to the grib_api
sudo apt-get update configuration script. Please note that GRIB-API is no longer
maintained. The primary GRIB encoding–decoding package
FLEXPART is developed using gfortran.
used at the ECMWF is currently ecCodes. Any new features
sudo apt-get install g++ gfortran for the handling of GRIB files will only be developed in ec-
Codes. However, for FLEXPART v10.4 grib_api is suf-
Some libraries (e.g., grib_api, jasper-1.900.1) ficient. We keep the grib_api instructions for backward
require the GNU autotools suite in order to configure, build consistency.
and install. For future versions, ensure that the path
sudo apt-get install autoconf libtool /usr/local/lib/ is in the environment variable
automake flex bison $PATH; otherwise, ecCodes may not find it. Obtain and
unpack ecCodes.
Newer packages (e.g., ecCodes) use CMake instead.
curl https://software.ecmwf.int/wiki/
sudo apt-get install cmake
download/attachments/45757960/
Python is not required for FLEXPART itself but is nec- eccodes-2.7.3-Source.tar.gz \
essary for some preprocessing and post-processing tools, --output eccodes-2.7.3-Source.tar.gz
in particular flex_extract for retrieving ECMWF wind tar -xvf eccodes-2.7.3-Source.tar.gz
fields. Git is recommended to access the code repositories.
The ecCodes requires CMake. The installation procedure
An editor (e.g., vim) is usually also necessary.
is described on the ECMWF ecCodes web page.
sudo apt-get install python-dev
python-pip git-core vim A3 Installing NetCDF libraries

A2 Installing GRIB libraries NetCDF output is optional. In order to enable NetCDF out-
put, the NetCDF library has to be available in the system.
If JPG compression is needed to decode the input meteoro- For building the NetCDF library it is recommended to first
logical winds, download the jasper library from the jasper build HDF5 with support for compression (zlib). For this,
project page (http://www.ece.uvic.ca/~mdadams/jasper/, last download zlib (version 1.2.8) from the zlib website (https:
access: 30 October 2019) and install it. //www.zlib.net/, last access: 30 October 2019) and install it.

www.geosci-model-dev.net/12/4955/2019/ Geosci. Model Dev., 12, 4955–4997, 2019

4988 I. Pisso et al.: FLEXPART version 10.4

tar -xzvf zlib-1.2.8.tar.gz git clone https://www.flexpart.eu/gitmob/

cd zlib-1.2.8/
./configure [--prefix=<installation path>] flexpart

make This mirrors https://git.nilu.no/flexpart/flexpart. Addi-

make install tional mirrors exist, e.g., at Bitbucket (https://bitbucket.org/
flexpart/flexpart, last access 30 October 2019) and GitHub
Download HDF5 from the HDF Group website (https: (https://github.com/flexpart/flexpart, last access 30 Oc-
//www.hdfgroup.org/HDF5/release/obtainsrc.html, last ac- tober 2019). Edit the library path variable in the
cess: 30 October 2019) and install it. makefile according to the position of libeccodes
(or libgrib_api) and libjasper. Optionally, edit
tar -xzvf hdf5-1.8.17.tar.gz the file par_mod.f90 to set parameters for the me-
cd hdf5-1.8.17/ teorological data, grid dimension and maximum parti-
./configure --with-zlib=<path to zlib> cle number (maxpart, maxspec, nxmax, nymax,
[--prefix=<installation path>] nuvzmax, nwzmax, nzmax, nxshift). The default
make values are set to work with the test cases in Sect. 7 but may be
make check too small for large simulations or too large for the available
make install system resources. Then type

Download the latest stable version of NetCDF–C from the make

Unidata website (https://www.unidata.ucar.edu/downloads/
in order to create the executable. Invoking the executable
netcdf/, last access: 30 October 2019) and install it.
FLEXPART should now print in the standard output.
tar -xzvf netcdf-4.4.1.tar.gz
Welcome to FLEXPART Version 10.4
cd netcdf-4.4.1/
FLEXPART is free software released under
./configure --enable-netcdf-4
the GNU General Public License.
[--prefix=<installation path>]
make However, without access to valid input data, the program
make check will issue an error. Appendix C explains how to generate
make install valid output with the standard meteorological fields from the
ECMWF that can be obtained following the procedure de-
Download the latest stable version of NetCDF–Fortran scribed in Sect. A5. The makefile also allows the following
from the Unidata website (https://www.unidata.ucar.edu/ command.
downloads/netcdf/, last access: 30 October 2019) and install
it. make clean

tar -xzvf netcdf-fortran-4.4.4.tar.gz This can be used to safely remove all object and module
cd netcdf-fortran-4.4.4/ files, e.g., if one wants to recompile after compiler option
./configure [--prefix=<installation path>] changes.

make A5 Installing flex_extract

make check
A short description of the installation steps for this software
make install
is given for the public user mode (other modes are described
A4 Installing FLEXPART in the flex_extract documentation). For this mode, the
user does not need to be a member state user (https://www.
Download the latest release of the FLEXPART source from ecmwf.int/en/about/who-we-are/member-states, last access:
the FLEXPART community website (https://flexpart.eu/wiki/ 13 October 2018) but can simply register at the ECMWF
FpDownloads, last access: 14 November 2019) or from https: website. For the other operating modes and a more detailed
//transport.nilu.no (last access: 14 November 2019) and untar explanation, see the README.md file of the python direc-
it. tory in the flex_extract distribution or the documenta-
tion files SIP.pdf and SUT_ondemand.pdf.
tar -xvf flexpart10.4.tar.gz First of all, the user should register at the ECMWF website
(https://www.ecmwf.int/en/forecasts/accessing-forecasts/
Alternatively, clone the FLEXPART repository directly order-historical-datasets, last access: 13 October 2018).
from the FLEXPART community site git. To access public datasets each dataset license has to be

Geosci. Model Dev., 12, 4955–4997, 2019 www.geosci-model-dev.net/12/4955/2019/

I. Pisso et al.: FLEXPART version 10.4 4989

accepted separately before the account can be used for 9. Check the availability of Python packages (e.g., check
retrieval of these data. This can be done at the following the Python console for the following commands:
website: https://software.ecmwf.int/wiki/display/WEBAPI/ import eccodes, import grib_api, import
Available+ECMWF+Public+Datasets (last access: 13 Octo- ecmwfapi)
ber 2018).
10. Start a simple test retrieval (following the instructions
A5.1 System preparation for flex_extract on the ECMWF WebAPI website).

flex_extract requires a Python environment and a For- 11. Install flex_extract (see the next section).
tran compiler. See Sect. A1 for installation instructions. To
It is important to use the same compiler and compiler ver-
prepare the environment for the flex_extract installa-
sion for all libraries and the Fortran program CONVERT2.
tion, it is advisable to consider the official documentation and
information from the ECMWF websites. We recommend the
A5.2 Building flex_extract
following steps.
To install flex_extract a script install.py was pre-
1. For important information read the Emoslib (https:
pared. The user can find it in the python directory of the
//software.ecmwf.int/wiki/display/EMOS/Emoslib, last
flex_extract distribution.
access: 13 October 2018) installation instructions first.
The public user mode requires a local installation of
2. Read the ECMWF blog about gfortran (https: flex_extract. Hence, we recommend adapting the paths
//software.ecmwf.int/wiki/display/SUP/2015/05/ to ecCodes, Emoslib or grib_api in one of the pre-
11/Building+ECMWF+software+with+gfortran, last pared makefiles, such as Makefile.local.gfortran,
access: 13 October 2018) for details on the installation which can be found in the src directory. If a different com-
process of the libraries. piler is used, this must also be adapted in the makefile. Then
the installation script can be called as follows.
3. Install FFTW (http://www.fftw.org, last access: 13 Octo-
./install.py --target=local
ber 2018) for Fortran, which is a library for computing
--makefile=Makefile.local.gfortran
the discrete Fourier transformation. This library is nec-
essary for Emoslib. (Note: apply make twice! Once With this setting flex_extract is installed within the
without any options and once with single precision op- current flex_extract directory. To install it in a different
tion; see the information on the Emoslib website). place, e. g. within a FLEXPART distribution, the user can set
the path with the parameter flexpart_root_scripts.
4. Install the interpolation library Emoslib for Fortran. The installation was successful if the compilation of the For-
tran program (CONVERT2) did not fail and is displayed at
5. Install ecCodes (https://software.ecmwf.int/wiki/ the end in the terminal.
display/ECC, last access: 13 October 2018) or
grib_api (https://software.ecmwf.int/wiki/display/ A5.3 Running flex_extract
GRIB/Home, last access: 13 October 2018) (for Python
and Fortran). The grib_api support will be discon- flex_extract is controlled by providing CONTROL files
tinued at the end of 2018 but ecCodes is downward that contain a list of parameter settings. These parameters are
compatible with grib_api. described in detail in the Software User Tutorial (SUT.pdf)
in the docs directory. The CONTROL files specify which
6. Install the ECMWF WebAPI (https://confluence. ECMWF dataset is to be retrieved, the time and spatial res-
ecmwf.int//display/WEBAPI/Access+MARS, last ac- olution, the format of the GRIB file, and other options. In
cess: 13 October 2018) client by following the instruc- the Python directory are some example CONTROL files for
tions on the website. It is a Python library to provide the different datasets and access modes. They can be used as
external access to the ECMWF servers. templates. CONTROL files with a .public ending are us-
able for the public access mode. The main difference is the
7. Check whether LD_LIBRARY_PATH and PATH envi- parameter dataset, which explicitly specifies the public
ronment variables contain all paths to the previously in- datasets. Note that not all meteorological fields, times and
stalled libraries. The user should modify the .bashrc parameters were archived in the public datasets. This is al-
or .tcshrc file to guarantee that the variables contain ready considered in the public CONTROL files.
the paths every time a new console is used. To run flex_extract, the main program submit.py
must be called. It retrieves the ECMWF data and generates
8. Install the python package numpy via pip (https://scipy. the FLEXPART input files. To show all possible parameter
org/install.html, last access: 13 October 2018). options one can use the -h option. The script must be called

www.geosci-model-dev.net/12/4955/2019/ Geosci. Model Dev., 12, 4955–4997, 2019

4990 I. Pisso et al.: FLEXPART version 10.4

from the python directory of the flex_extract distri- export PYTHONPATH=path/to/ecmwfapi:path/to/

bution. From the -h output it is clear that most parameters grib_api
have default values or were already set via a CONTROL file $flex_extracthome/Python>./submit.py
parameter, except for the date. To retrieve just one day, one --start_date 20170102 --controlfile
only needs to provide the start date. The rest will be done by CONTROL_EA5
flex_extract. This leads to the following script call for
an arbitrary date. This should generate the files EA170102?? in the fol-
lowing directory.
./submit.py --controlfile=CONTROL_EI.public \
$flex_extracthome/work/
--start_date=20120101 \
--public=1 An AVAILABLE consistent with these wind fields is
shipped together with the FLEXPART distribution.
The program now displays each MARS request and some
messages for the preparation of the FLEXPART input files. $flexhome/AVAILABLE
Eventually, the program will finish with a Done! message
if there was no error. Output will be stored in the default B2 Running the default example: installation
directory work, which is a subdirectory of the distribution verification
directory (flex_extract_v7.0.4). The produced files
can serve as input to FLEXPART. With the input files, which are included in the FLEXPART
distribution and described in Sect. 5, a first test case to
verify that FLEXPART was installed correctly can be run.
Appendix B: Running and testing FLEXPART To start the model run, the meteorological data have to be
in $flex_extracthome/work/ (see Sect. B1), the
After a working FLEXPART executable is built (Ap- file pathnames in $flexhome, and the executable in
pendix A), the next step is running the model and gen- $flexhome/src/ in the $flexhome directory type.
erating valid output. This requires consistent meteorologi-
cal input data and user input files. In this section we de- $flexhome>./src/FLEXPART
scribe the following: how to obtain the necessary wind
The results created by this run are stored, e.g., in a direc-
fields (1), how to test run the executable with a de-
tory $flexhome/output (as defined in pathnames).
fault example (2), how to generate other examples (3),
The run should end with the following message.
and how to run these examples and compare them with
a reference output (4). In the following, $flexhome in- CONGRATULATIONS: YOU HAVE SUCCESSFULLY
dicates the path to the root FLEXPART directory (e.g., COMPLETED A FLEXPART MODEL RUN!
$HOME/flexpart/) and $flex_extracthome indi-
cates the path to the flex_extract root directory (e.g., If this message is received, the model has completed the
$flexhome/preprocess/flex_extract/). simulation, which confirms that FLEXPART and all required
libraries are installed correctly. However, it does not guaran-
B1 Meteorological input for the examples tee valid output. To verify that the results obtained are valid,
see Sect. B5.
Appendix A describes how to build the flex_extract
version included in the source code. Here, we de- B3 Generating variations of the default example
scribe the settings to produce the meteorological in-
put data required for running the default (Sect. B2) To demonstrate more functionalities, a set of shell
and derived (Sect. B3) cases. The instructions are for scripts generating different FLEXPART setups are pro-
ECMWF ERA5 reanalysis, which is a publicly avail- vided in $flexhome/tests/examples. The script
able dataset (https://confluence.ecmwf.int/display/WEBAPI/ set_default_example.sh takes the content of the
Access+ECMWF+Public+Datasets, last access: 30 Octo- options directory and pathnames file from Sect. B2
ber 2019). Therefore, the data can be obtained via as a basis, and then gen_options_all.sh creates new
ecmwfapi and no special access rights to the ECMWF are options_suffix directories for all of the cases de-
needed. However, in order to retrieve the data the user needs scribed in Table 14. Here, the suffix corresponds to the
to register, obtain a personal Secure Shell (SSH) key and example name as given in column 2 in Table 14. Fi-
properly configure the file .ecmwfapirc. The execution of nally, the script gen_pathnames.sh generates corre-
the retrieval requires the Python packages ecmwfapi (for sponding pathnames_suffix files pointing to all the
access) and grib_api or eccodes (for processing). To options_suffix directories. With this, all example cases
retrieve the data, execute the following commands. in Table 14 are ready to run.

Geosci. Model Dev., 12, 4955–4997, 2019 www.geosci-model-dev.net/12/4955/2019/

I. Pisso et al.: FLEXPART version 10.4 4991

B4 Running the examples the meteorological output data generated by the CMIP5 ver-
sion of NorESM1-M (the Norwegian Earth System Model)
The examples can be run interactively one by one with 1.89◦ × 2.5◦ horizontal resolution and 26 vertical lev-
by invoking FLEXPART with the corresponding els. The standard time resolution of the NorESM/CAM
pathnames_suffix file. Alternatively, the script meteorological data is 3 h. FLEXPART–NorESM/CAM is
gen_batch_jobs_cl.sh generates a one-line script based on FLEXPART v9, and the atmospheric component
for each example case containing a call of FLEXPART of NorESM1-M is based on CAM4 (the Community Atmo-
and the appropriate pathnames_suffix file as a sphere Model). The adaptation of FLEXPART to NorESM
command-line parameter. All example scripts can then be required new routines to read meteorological fields, new
run sequentially with run_batch_cl.sh, which creates post-processing routines to obtain the vertical velocity in
output_suffix directories with the results, as well as the FLEXPART coordinate system and other changes, as de-
log files batch_job_pathnames_suffix.stdout tailed by Cassiani et al. (2016). The code can be downloaded
for each run. The examples described above can now be read from https://www.flexpart.eu/wiki/FpClimateNorESM (last
and plotted with the tools included in the distribution. These access: 30 October 2019).
steps are also automated in a makefile. All of the files and
directories created by executing the scripts from Sects. B2 to C2 FLEXPART–WRF
B4 can be removed again with the command make clean.
This FLEXPART version uses output from the Weather Re-
B5 Comparing the results search and Forecasting (WRF) mesoscale meteorological
model (Brioude et al., 2013). Originally it was developed
To verify that FLEXPART is producing valid output, it is at the PNNL (Pacific Northwest National Laboratory) and
useful to compare the output of a new installation with named PILT (PNNL Integrated Lagrangian Transport). Com-
existing model output. It is also useful to repeat such pared to PILT, the further developed FLEXPART–WRF can
a comparison after code changes to make sure the out- use both instantaneous and time-averaged meteorological
put is not affected, except for model simulations in which output of the WRF model. The latest version also includes
changes in the results are intended. While comprehensive the skewed turbulence scheme that was subsequently ported
comparisons of model results are possible, here we pro- to the standard FLEXPART version 10.4. FLEXPART–WRF
vide only a very simple way of checking the model re- output can either be in binary or Network Common Data
sults. The directory included in the FLEXPART distri- Form (NetCDF) format, both of which have efficient data
bution $flexhome/tests/examples_reference/ compression. FLEXPART–WRF also offers effective paral-
contains the output of the examples described in Table 14. lelization with OpenMP in shared memory and an MPI li-
The file read_examples_output.txt contains, for the brary in distributed memory. Released versions of the code
relevant examples that produce gridded output, the mean and can be downloaded from https://www.flexpart.eu/wiki/ (last
the maximum value that occurs in the gridded output files. access: 30 October 2019) or cloned from the open repository
This shall serve as a reference to which users can compare [email protected]:flexpart/flexpart-wrf.git.
their results and thus verify that the model produces output as
expected. In addition, the directory compare_examples
C3 FLEXPART–COSMO
contains the script compare_grids.sh that allows
for the partial automation of this task (output in In Europe several national weather services and research
compare_examples_output.txt). groups cooperate to develop and operate the non-hydrostatic
limited-area atmospheric model COSMO (Consortium for
Appendix C: FLEXPART model versions Small-scale Modeling). At MeteoSwiss COSMO is opera-
tionally run with data assimilation on two grids with approx-
In addition to the reference version of FLEXPART described imately 7 × 7 km2 and 2 × 2 km2 horizontal resolution cen-
in this paper, there are many different model branches that tered over Switzerland. This enables the study of atmospheric
were developed either for special purposes or to ingest other transport over complex terrain on a long-term basis. To this
meteorological input data. This Appendix provides an in- end, we have developed a new version of FLEXPART that
complete list and a short description of some of these other is offline coupled to COSMO output (FLEXPART–COSMO
versions. Further contributions are welcome in order to keep hereafter) and supports output from multiple COSMO nests.
this list up to date. Particles are internally referenced against the native vertical
coordinate system used in COSMO and not, as in standard
C1 FLEXPART–NorESM/CAM FLEXPART, in a terrain-following z system. This eliminates
the need for an additional interpolation step. A new flux de-
Recently, the FLEXPART model version FLEXPART– accumulation scheme was introduced that removes the need
NorESM/CAM was developed, which is tailored to run with for additional preprocessing of the input files. In addition to

www.geosci-model-dev.net/12/4955/2019/ Geosci. Model Dev., 12, 4955–4997, 2019

4992 I. Pisso et al.: FLEXPART version 10.4

the existing Emanuel-based convection parameterization, a C5 TRACZILLA

convection parameterization based on the Tiedtke scheme,
which is identical to the one implemented in COSMO it- This branch-off from FLEXPART version 5 was originally
self, was introduced. A possibility for offline nesting of a developed for studies of transport and mixing in the up-
FLEXPART–COSMO run into a FLEXPART–ECMWF run per troposphere–lower stratosphere region (e.g., Legras et
for backward simulations was developed that only requires al., 2003; Pisso and Legras, 2008). The modifications from
minor modifications of the FLEXPART–ECMWF version the FLEXPART advection scheme consist mainly of discard-
and allows particles to leave the limited COSMO domain. ing the intermediate terrain-following coordinate system and
The OpenMP shared memory parallelization to the model performing a direct vertical interpolation of winds, linear in
allows for asynchronous reading of input data. The code log pressure, from hybrid levels. The vertical velocities are
is available on request from [email protected] and computed by the FLEXPART preprocessor using a mass-
[email protected]. conserving scheme in the hybrid ECMWF coordinates. Al-
ternatively, the vertical velocities can be computed from the
C4 FLEXPART–AROME rates of diabatic heating from ECMWF winds. In addition to
the reanalyses from the ECMWF, the current version can use
The Applications of Research to Operations at Mesoscale MERRA (Modern-Era Retrospective analysis for Research
(AROME) numerical weather prediction model is run opera- and Applications) from NASA and JRA-55 (the Japanese 55-
tionally by Météo-France at the mesoscale. AROME fore- year Reanalysis) from the Japanese Meteorological Agency
casts for Europe exist at a resolution ranging from 0.5 to (JMA). The parallelization uses the OMP version of PGI. All
2.5 km. The standard time resolution of the AROME meteo- arrays are allocated dynamically. The code can be obtained
rological data is 1 h. Based on FLEXPART–WRF, a coupling from https://github.com/bernard-legras/traczilla (last access:
between FLEXPART and AROME was developed at Labo- 30 October 2019).
ratoire de l’Atmosphère et des Cyclones (LACy, a joint insti-
tute between CNRS, Météo-France and the University of Re-
union Island) using AROME high-resolution (2.5 × 2.5 km2 )
forecasts over the southwest Indian Ocean. The FLEXPART–
AROME branch (Verreyken et al., 2019b) simulates turbu-
lent transport using the Thomson turbulent scheme (Thom-
son, 1987) already implemented by Lin et al. (2003) in
the Stochastic Time-Inverted Lagrangian Transport (STILT)
model. This method constrains mass transport between dif-
ferent turbulent regions to conserve mass locally for a passive
well-mixed tracer. Turbulent kinetic energy profiles are taken
directly from AROME model outputs. Such treatment of tur-
bulent motion ensures consistency between the turbulence
in the meteorological fields calculated by the NWP model
and turbulence computed in the offline Lagrangian transport
model. It has been noticed that the use of a dedicated ABL
scheme such as Hanna in the FLEXPART model may gen-
erate inconsistency between the ABL turbulent domain and
the resolved wind fields used to drive FLEXPART. Simula-
tions using the Thomson scheme show a better representation
of the turbulent mixing between boundary layer air and free
tropospheric air.

Geosci. Model Dev., 12, 4955–4997, 2019 www.geosci-model-dev.net/12/4955/2019/

I. Pisso et al.: FLEXPART version 10.4 4993

Author contributions. IP coordinated the contributions to the pa- Financial support. This research has been supported by NordForsk
per and the code development since version 9, including I/O, up- (the Nordic Center of Excellence eSTICC, grant no. 57001), the Eu-
dates to turbulent mixing, the implementation of the tests and the ropean Research Council (project COMTESSA, grant no. 670462),
distributed version control. ES developed and wrote the descrip- and the CTBTO (Enhancements of the FLEXPART software on a
tion of the parallelized version of FLEXPART and led the assem- call-off basis).
bling of the new code developments into the main model version
10.4. HG developed and tested the new wet deposition scheme for
aerosols. NIK contributed to the new wet deposition scheme for Review statement. This paper was edited by Slimane Bekki and re-
aerosols by testing the new model version and coordinated the con- viewed by two anonymous referees.
tributions to the first version of the paper. MC developed the op-
tional new turbulence scheme and the NorESM version and con-
tributed to the WRF version. SE developed and wrote the descrip-
tion of the backward deposition, performed the benchmark test case
together with IP, and worked on ECMWF data retrieval and test- References
ing. DA and DM contributed to the CTBTO developments including
the unified executable, the Vtables approach and testing environ- Arnalds, O., Dagsson-Waldhauserova, P., and Olafsson, H.:
ment. RLT developed the temporal variation and temperature de- The Icelandic volcanic aeolian environment: Processes
pendence of the OH reaction. CDGZ developed the dust mobiliza- and impacts – A review, Aeolian Res., 20, 176–195,
tion scheme around FLEXPART and performed testing of the new https://doi.org/10.1016/j.aeolia.2016.01.004, 2016.
model version. NE tested the new model version for black carbon Arnold, D., Maurer, C., Wotawa, G., Draxler, R., Saito,
and radionuclide applications. HS implemented the namelist input K., and Seibert, P.: Influence of the meteorological in-
file format and contributed to the implementation of the NetCDF put on the atmospheric transport modelling with FLEX-
output and GRIB input routines. LH developed versions 2.0–7.02 PART of radionuclides from the Fukushima Daiichi nu-
of the flex_extract retrieval software, in particular the codes clear accident, J. Environ. Radioactiv., 139, 212–225,
for calculating the hybrid coordinate vertical velocity. He also su- https://doi.org/10.1016/j.jenvrad.2014.02.013, 2015.
pervised the most recent developments and wrote the description Asman, W. A. H.: Parametrisation of below-cloud scavenging of
together with AP. SH and DB contributed to the implementation highly soluble gases under convective conditions, Atmos. Envi-
of the NetCDF output module. JB coordinated the development up ron., 29, 1359–1368, 1995.
to FLEXPART version 8.3 and contributed to post-processing the Atkinson, R.: Gas-phase tropospheric chemistry of volatile organic
Python module. AF developed the new Phyton-based ECMWF data compounds: 1. Alkanes and alkenes, J. Phys. Chem. Ref. Data,
retrieval software. JB led the development of the WRF and AROME 26, 215–290, 1997.
versions and contributed to the turbulence scheme. AP developed, Balluch, M., and Haynes, P.: Quantification of lower stratospheric
maintained and wrote the description of the flex_extract re- mixing processes using aircraft data, J. Geophys. Res., 102,
trieval routines and contributed to the testing environment. PS de- 23487–23504, 1997.
vised the community website http://flexpart.eu, takes care of tickets Bey, I., Jacob, D. J., Logan, J. A., and Yantosca, R. M.:
and the wiki, contributed to various parts of the code development, Asian chemical outflow to the Pacific in spring: Origins,
and contributed to editing the paper. AS developed the first ver- pathways, and budgets, J. Geophys. Res., 106, 23073–23095,
sion of the code in 1998 and supervised all recent developments, https://doi.org/10.1029/2001jd000806, 2001.
including the new settling parameterization for aerosols and time- Brioude, J., Arnold, D., Stohl, A., Cassiani, M., Morton, D., Seib-
averaged particle output, and worked on the writing and structuring ert, P., Angevine, W., Evan, S., Dingwell, A., Fast, J. D., Easter,
of the paper. R. C., Pisso, I., Burkhart, J., and Wotawa, G.: The Lagrangian
particle dispersion model FLEXPART-WRF version 3.1, Geosci.
Model Dev., 6, 1889–1904, https://doi.org/10.5194/gmd-6-1889-
Competing interests. The authors declare that they have no conflict 2013, 2013.
of interest. Bullard, J. E., Baddock, M., Bradwell, T., Crusius, J., Darlington,
E., Gaiero, D., Gasso, S., Gisladottir, G., Hodgkins, R., Mc-
Culloch, R., Neuman, C. M., Mockford, T., Stewart, H., and
Thorsteinsson, T.: High latitude dust in the earth system, Rev.
Acknowledgements. The work was performed at the Nordic Cen-
Geophys., 54, 447–485, https://doi.org/10.1002/2016RG000518,
ter of Excellence eSTICC. Pirmin Kaufmann and Martin Schraner
2016.
(MeteoSwiss) are acknowledged for code reformatting from fixed
Cassiani, M., Stohl, A., and Brioude, J.: Lagrangian stochastic mod-
Fortran 77 to free Fortran 90 format. We thank Mariëlle Mulder
elling of dispersion in the convective boundary layer with skewed
for comments on an early version of this paper. The resources for
turbulence conditions and a vertical density gradient: Formu-
the numerical simulations were provided by UNINETT Sigma2 (the
lation and implementation in the FLEXPART Model, Bound.-
National Infrastructure for High Performance Computing and Data
Lay. Meteorol., 154, 367–390, https://doi.org/10.1007/s10546-
Storage in Norway) under projects NN9419K and NS9419K. Input
014-9976-5, 2015.
wind fields were provided by the ECMWF.
Cassiani, M., Stohl, A., Olivié, D., Seland, Ø., Bethke, I., Pisso,
I., and Iversen, T.: The offline Lagrangian particle model
FLEXPART–NorESM/CAM (v1): model description and com-
parisons with the online NorESM transport scheme and with

www.geosci-model-dev.net/12/4955/2019/ Geosci. Model Dev., 12, 4955–4997, 2019

4994 I. Pisso et al.: FLEXPART version 10.4

the reference FLEXPART model, Geosci. Model Dev., 9, 4029– Groot Zwaaftink, C. D., Arnalds, Ó., Dagsson-Waldhauserova,
4048, https://doi.org/10.5194/gmd-9-4029-2016, 2016. P., Eckhardt, S., Prospero, J. M., and Stohl, A.: Temporal
Cozic, J., Verheggen, B., Mertes, S., Connolly, P., Bower, K., and spatial variability of Icelandic dust emissions and at-
Petzold, A., Baltensperger, U., and Weingartner, E.: Scav- mospheric transport, Atmos. Chem. Phys., 17, 10865–10878,
enging of black carbon in mixed phase clouds at the high https://doi.org/10.5194/acp-17-10865-2017, 2017.
alpine site Jungfraujoch, Atmos. Chem. Phys., 7, 1797–1807, Groot Zwaaftink, C. D., Henne, S., Thompson, R. L., Dlugo-
https://doi.org/10.5194/acp-7-1797-2007, 2007. kencky, E. J., Machida, T., Paris, J.-D., Sasakawa, M., Segers,
Dee, D. P., Uppala, S. M., Simmons, A. J., Berrisford, P., Poli, A., Sweeney, C., and Stohl, A.: Three-dimensional methane
P., Kobayashi, S., Andrae, U., Balmaseda, M. A., Balsamo, G., distribution simulated with FLEXPART 8-CTM-1.1 constrained
Bauer, P., Bechtold, P., Beljaars, A. C. M., van de Berg, L., Bid- with observation data, Geosci. Model Dev., 11, 4469–4487,
lot, J., Bormann, N., Delsol, C., Dragani, R., Fuentes, M., Geer, https://doi.org/10.5194/gmd-11-4469-2018, 2018.
A. J., Haimberger, L., Healy, S. B., Hersbach, H., Hólm, E. V., Grythe, H., Kristiansen, N. I., Groot Zwaaftink, C. D., Eck-
Isaksen, L., Kallberg, P., Köhler, M., Matricardi, M., McNally, hardt, S., Ström, J., Tunved, P., Krejci, R., and Stohl, A.: A
A. P., Monge-Sanz, B. M., Morcrette, J.-J., Park, B.-K., Peubey, new aerosol wet removal scheme for the Lagrangian particle
C., de Rosnay, P., and Tavolato, C., Thépaut, J.-N., and Vitart, F.: model FLEXPART v10, Geosci. Model Dev., 10, 1447–1466,
The ERA-Interim reanalysis: configuration and performance of https://doi.org/10.5194/gmd-10-1447-2017, 2017.
the data assimilation system. Q. J. Roy. Meteor. Soc., 137, 553– Haynes, P. and Anglade, J.: The Vertical-Scale Cascade in Atmo-
597, https://doi.org/10.1002/qj.828, 2011. spheric Tracers due to Large-Scale Differential Advection, J. At-
Eckhardt, S., Prata, A. J., Seibert, P., Stebel, K., and Stohl, A.: Esti- mos. Sci., 54, 1121–1136, 1997.
mation of the vertical profile of sulfur dioxide injection into the Heinz, S.: Statistical Mechanics of Turbulent Flows, Springer,
atmosphere by a volcanic eruption using satellite column mea- Berlin, Heidelberg, Germany, 214 pp., 2003.
surements and inverse transport modeling, Atmos. Chem. Phys., Henne, S., Brunner, D., Oney, B., Leuenberger, M., Eugster, W.,
8, 3881–3897, https://doi.org/10.5194/acp-8-3881-2008, 2008. Bamberger, I., Meinhardt, F., Steinbacher, M., and Emmeneg-
Eckhardt, S., Cassiani, M., Evangeliou, N., Sollum, E., Pisso, I., ger, L.: Validation of the Swiss methane emission inventory
and Stohl, A.: Source–receptor matrix calculation for deposited by atmospheric observations and inverse modelling, Atmos.
mass with the Lagrangian particle dispersion model FLEXPART Chem. Phys., 16, 3683–3710, https://doi.org/10.5194/acp-16-
v10.2 in backward mode, Geosci. Model Dev., 10, 4605–4618, 3683-2016, 2016.
https://doi.org/10.5194/gmd-10-4605-2017, 2017. Henning, S., Bojinski, S., Diehl, K., Ghan, S., Nyeki, S., Weingart-
ECMWF: User Guide to ECMWF Products 2.1. Meteorological ner, E., Wurzler, S., and Baltensperger, U.: Aerosol partitioning
Bulletin M3.2. Reading, UK, 1995. in natural mixed-phase clouds, Geophys. Res. Lett., 31, L06101,
Emanuel, K. A. and Živković-Rothman, M.: Development and eval- https://doi.org/10.1029/2003GL019025, 2004.
uation of a convection scheme for use in climate models. J. At- Hertel, O., Christensen, J. Runge, E. H., Asman, W. A. H., Berkow-
mos. Sci., 56, 1766–1782, 1999. icz, R., Hovmand, M. F., and Hov, O.: Development and testing
Fang, X., Shao, M., Stohl, A., Zhang, Q., Zheng, J., Guo, H., Wang, of a new variable scale air pollution model – ACDEP, Atmos.
C., Wang, M., Ou, J., Thompson, R. L., and Prinn, R. G.: Top- Environ., 29, 1267–1290, 1995.
down estimates of benzene and toluene emissions in the Pearl Hittmeir, S., Philipp, A., and Seibert, P.: A conservative recon-
River Delta and Hong Kong, China, Atmos. Chem. Phys., 16, struction scheme for the interpolation of extensive quantities in
3369–3382, https://doi.org/10.5194/acp-16-3369-2016, 2016. the Lagrangian particle dispersion model FLEXPART, Geosci.
Flesch, T. K., Wilson, J. D., and Lee, E.: Backward-time Lagrangian Model Dev., 11, 2503–2523, https://doi.org/10.5194/gmd-11-
stochastic dispersion models and their application to estimate 2503-2018, 2018.
gaseous emissions, J. Appl. Meteorol., 34, 1320–1333, 1995. Hoinka, K: The tropopause: discovery, definition and demarcation,
Forster, C., Wandinger, U., Wotawa, G., James, P., Mattis, I., Al- Meteorol. Z., 6, 281–303, 1997.
thausen, D., Simmonds, P., O’Doherty, S., Kleefeld, C., Jennings, Hoyer, S. and Hamman, J.: xarray: N-D labeled Arrays and
S. G., Schneider, J., Trickl, T., Kreipl, S., Jäger, H., and Stohl, A.: Datasets in Python, Journal of Open Research Software, 5, p. 10,
Transport of boreal forest fire emissions from Canada to Europe, https://doi.org/http://doi.org/10.5334/jors.148, 2017.
J. Geophys. Res., 106, 22887–22906, 2001. Jones, A., Thomson, D., Hort, M., and Devenish, B.: The UK Met
Forster, C., Stohl, A., and Seibert, P.,: Parameterization of Con- Office’s next generation atmospheric dispersion model, NAME
vective Transport in a Lagrangian Particle Dispersion Model III, in Air Pollution Modelling and its Application XVII, edited
and Its Evaluation, J. Appl. Meteorol. Clim., 46, 403–422, by: Borrego, C. and Norman, A.-L., Springer, New York, 580–
https://doi.org/10.1175/JAM2470.1, 2007. 589, 2007.
Global Soil Data Task: Global soil data products CD- Kok, J. F.: A scaling theory for the size distribution of emitted dust
ROM contents (IGBP-DIS), Data Set, Oak Ridge Natl. aerosols suggests climate models underestimate the size of the
Lab. Distrib. Active Arch. Cent., Oak Ridge, Tenn., global dust cycle, P. Natl. Acad. Sci. USA, 108, 1016–1021,
https://doi.org/10.3334/ORNLDAAC/565, 2014. https://doi.org/10.1073/pnas.1014798108, 2011.
Groot Zwaaftink, C. D., Grythe, H., Skov, H., and Stohl, A.: Sub- Kristiansen, N. I., Stohl, A., Prata, A. J., Richter, A., Eckhardt, S.,
stantial contribution of northern high-latitude sources to mineral Seibert, P., Hoffmann, A., Ritter, C., Bitar, L., Duck, T. J., and
dust in the Arctic, J. Geophys. Res.-Atmos., 121, 13678–13697, Stebel, K.: Remote sensing and inverse transport modelling of
https://doi.org/10.1002/2016JD025482, 2016. the Kasatochi eruption sulphur dioxide cloud, J. Geophys. Res.,
115, D00L16, https://doi.org/10.1029/2009JD013286, 2010.

Geosci. Model Dev., 12, 4955–4997, 2019 www.geosci-model-dev.net/12/4955/2019/

I. Pisso et al.: FLEXPART version 10.4 4995

Kristiansen, N. I., Stohl, A., Prata, F., Bukowiecki, N., Dacre, H., expansion during antiquity, P. Natl. Acad. Sci. USA, 115, 5726–
Eckhardt, S., Henne, S., Hort, M., Johnson, B., Marenco, F., 5731, https://doi.org/10.1073/pnas.1721818115, 2018.
Neininger, B., Reitebuch, O., Seibert, P., Thomson, D., Webster, McMahon, T. A. and Denison, P. J.: Empirical atmospheric deposi-
H., and Weinzierl, B.: Performance assessment of a volcanic ash tion parameters – a survey, Atmos. Environ., 13, 571–585, 1979.
transport model mini-ensemble used for inverse modelling of the Moxnes, E., Kristiansen, N.I., Stohl, A., Clarisse, L., Durant, A.,
2010 Eyjafjallajökull eruption, J. Geophys. Res., 117, D00U11, Weber, K., and Vogel, A.: Separation of ash and sulfur dioxide
https://doi.org/10.1029/2011JD016844, 2012. during the 2011 Grímsvötn eruption, J. Geophys. Res.-Atmos.,
Kristiansen, N. I., Prata, A. J., Stohl, A., and Carn, S. A. 119, 7477–7501, https://doi.org/10.1002/2013JD021129, 2014.
: Stratospheric volcanic ash emissions from the 13 Febru- Message Passing Interface Forum: available at: https://www.
ary 2014 Kelut eruption, Geophys. Res. Lett., 42, 588–596, mpi-forum.org/ (last access: 30 October 2019), 2015.
https://doi.org/10.1002/2014GL062307, 2015. Naeslund, E. and Thaning, L.: On the settling velocity in a nonsta-
Kristiansen, N. I., Stohl, A., Olivié, D. J. L., Croft, B., Søvde, tionary atmosphere. Aerosol Sci. Tech., 14, 247–256, 1991.
O. A., Klein, H., Christoudias, T., Kunkel, D., Leadbetter, S. Oney, B., Henne, S., Gruber, N., Leuenberger, M., Bam-
J., Lee, Y. H., Zhang, K., Tsigaridis, K., Bergman, T., Evange- berger, I., Eugster, W., and Brunner, D.: The CarboCount
liou, N., Wang, H., Ma, P.-L., Easter, R. C., Rasch, P. J., Liu, CH sites: characterization of a dense greenhouse gas ob-
X., Pitari, G., Di Genova, G., Zhao, S. Y., Balkanski, Y., Bauer, servation network, Atmos. Chem. Phys., 15, 11147–11164,
S. E., Faluvegi, G. S., Kokkola, H., Martin, R. V., Pierce, J. R., https://doi.org/10.5194/acp-15-11147-2015, 2015.
Schulz, M., Shindell, D., Tost, H., and Zhang, H.: Evaluation of Ottino, J.: The Kinematics of Mixing: Stretching, Chaos and Trans-
observed and modelled aerosol lifetimes using radioactive trac- port, Cambridge Univ. Press, New York, 1989.
ers of opportunity and an ensemble of 19 global models, At- Pisso, I. and Legras, B.: Turbulent vertical diffusivity in the
mos. Chem. Phys., 16, 3525–3561, https://doi.org/10.5194/acp- sub-tropical stratosphere, Atmos. Chem. Phys., 8, 697–707,
16-3525-2016, 2016. https://doi.org/10.5194/acp-8-697-2008, 2008.
Kyrö, E.-M., Grönholm, T., Vuollekoski, H., Virkkula, A., Kulmala, Pisso, I., Real, E., Law, K. S., Legras, B., Bousserez, N., At-
M., and Laakso, L. : Snow scavenging of ultrafine particles: field tié, J. L., and Schlager, H.: Estimation of mixing in the tropo-
measurements and parameterization. Boreal Environ. Res., 14, sphere from Lagrangian trace gas reconstructions during long-
527–538, 2009. range pollution plume transport, J. Geophys. Res., 114, D19301,
Laakso, L., Grönholm, T., Rannika, Ü., Kosmalea, M., Fiedlera, https://doi.org/10.1029/2008JD011289, 2009.
V., Vehkamäki, H., and Kulmala, M.: Ultrafine particle scav- Pisso, I., Sollum, E., Grythe, H., Kristiansen, N.I., Cassiani,
enging coefficients calculated from 6 years field measurements, M., Eckhardt, S., Arnold, D., Morton, D., Thompson, R.L.,
Atmos. Environ. 37, 3605–3613, https://doi.org/10.1016/S1352- Groot Zwaaftink, C.D., Evangeliou, N., Sodemann, H., Haim-
2310(03)00326-1, 2003. berger, L., Henne, S., Brunner, D., Burkhart, J.F., Fouilloux,
Laloyaux, P., de Boisseson, E., Balmaseda, M., Bidlot, J.R. Broen- A., Brioude, J., Philipp, A., Seibert, P., and Stohl, A.: FLEX-
nimann, S., Buizza, R., Dalhgren, P., Dee, D., Haimberger, L., PART 10.4 (Version 10.4), Geosci. Model Dev. Discuss. Zenodo,
Hersbach, H., Kosaka, Y., Martin, M., Poli, P., Rayner, N., Ruste- https://doi.org/10.5281/zenodo.3542278, 2019.
meier, E., and Schepers, D.: CERA20C: A coupled reanalysis of Ramli, Huda Mohd. and Esler, J. G.: Quantitative evalua-
the twentieth century, J. Adv. Model Earth Sy., 10, 1172–1195, tion of numerical integration schemes for Lagrangian parti-
https://doi.org/10.1029/2018MS001273, 2018. cle dispersion models, Geosci. Model Dev., 9, 2441–2457,
Legras, B., Joseph, B., and Lefevre, F.: Vertical diffusivity in https://doi.org/10.5194/gmd-9-2441-2016, 2016.
the lower stratosphere from Lagrangian back-trajectory recon- Rastigejev, Y., Park, R., Brenner, M., and Jacob, D.: Re-
structions of ozone profiles, J. Geophys. Res., 108, 4562, solving intercontinental pollution plumes in global models
https://doi.org/10.1029/2002JD003045, 2003. of atmospheric transport, J. Geophys. Res., 115, D02302,
Lin, J. C., Gerbig, C., Wofsy, S. C., Andrews, A. E., https://doi.org/10.1029/2009JD012568, 2010.
Daube, B. C., Davis, K. J., and Grainger, C. A.: A near- Reithmeier, C. and Sausen, R.: ATTILA: Atmospheric Tracer
field tool for simulating the upstream influence of atmo- Transport in a Lagrangian Model, Tellus B, 54, 278–299, 2002.
spheric observations: The Stochastic Time-Inverted Lagrangian Rodean, H.: Stochastic Lagrangian models of turbulent diffusion,
Transport (STILT) model, J. Geophys. Res., 108, 4493, Meteorological Monographs, 26, American Meteorological So-
https://doi.org/10.1029/2002JD003161, 2003. ciety, Boston, USA, 1996.
Luhar, A. K. and Britter, R. E.: Random walk model for dispersion Seibert, P. and Frank, A.: Source-receptor matrix calculation with
in inhomogeneous turbulence in a convective boundary layer, At- a Lagrangian particle dispersion model in backward mode, At-
mos. Environ., 23, 1911–1924, 1989. mos. Chem. Phys., 4, 51–63, https://doi.org/10.5194/acp-4-51-
Marticorena, B. and Bergametti, G.: Modeling the atmo- 2004, 2004.
spheric dust cycle: 1. Design of a soil-derived dust Seibert, P., Krüger, B., and Frank, A.: parametrisation of convec-
emission scheme, J. Geophys. Res., 100, 16415–16430, tive mixing in a Lagrangian particle dispersion model, Proceed-
https://doi.org/10.1029/95JD00690, 1995. ings of the 5th GLOREAM Workshop, Wengen, Switzerland,
McConnell, J. R., Wilson, A. I., Stohl, A., Arienzo, M. M., Chell- 24–26 September 2001.
man, N. J., Eckhardt, S., Thompson, E. M., Pollard, A. M., and Shao, Y. and Lu, H.: A simple expression for wind erosion
Steffensen, J. P.: Lead pollution recorded in Greenland ice in- threshold friction velocity, J. Geophys. Res., 105, 22437–22443,
dicates European emissions tracked plagues, wars, and imperial https://doi.org/10.1029/2000JD900304, 2000.

www.geosci-model-dev.net/12/4955/2019/ Geosci. Model Dev., 12, 4955–4997, 2019

4996 I. Pisso et al.: FLEXPART version 10.4

Sodemann, H., Lai, T. M., Marenco, F., Ryder, C. L., Flamant, C., monds, P. G., Vollmer, M. K., Weiss, R. F., and Yokouchi, Y.:
Knippertz, P., Rosenberg, P., Bart, M., and McQuaid, J. B.: La- An analytical inversion method for determining regional and
grangian dust model simulations for a case of moist convective global emissions of greenhouse gases: Sensitivity studies and
dust emission and transport in the western Sahara region during application to halocarbons, Atmos. Chem. Phys., 9, 1597–1620,
Fennec/LADUNEX, J. Geophys. Res.-Atmos., 120, 6117–6144, https://doi.org/10.5194/acp-9-1597-2009, 2009.
https://doi.org/10.1002/2015JD023283, 2015. Stohl, A., Prata, A. J., Eckhardt, S., Clarisse, L., Durant, A.,
Spichtinger, N., Wenig, M., James, P., Wagner, T., Platt, U., and Henne, S., Kristiansen, N. I., Minikin, A., Schumann, U., Seib-
Stohl, A.: Satellite detection of a continental-scale plume of ni- ert, P., Stebel, K., Thomas, H. E., Thorsteinsson, T., Tørseth, K.,
trogen oxides from boreal forest fires, Geophys. Res. Lett., 28, and Weinzierl, B.: Determination of time- and height-resolved
4579–4582, 2001. volcanic ash emissions and their use for quantitative ash dis-
Stein, A. F., Draxler, R. R, Rolph, G. D., Stunder, B. J. B., persion modeling: the 2010 Eyjafjallajökull eruption, Atmos.
Cohen, M. D., and Ngan, F.: NOAA’s HYSPLIT atmospheric Chem. Phys., 11, 4333–4351, https://doi.org/10.5194/acp-11-
transport and dispersion modeling system, B. Am. Meteo- 4333-2011, 2011.
rol. Soc., 96, 2059–2077, https://doi.org/10.1175/BAMS-D-14- Stohl, A., Seibert, P., Wotawa, G., Arnold, D., Burkhart, J. F., Eck-
00110.1, 2015. hardt, S., Tapia, C., Vargas, A., and Yasunari, T. J.: Xenon-
Stohl, A.: Computation, accuracy and applications of trajectories – 133 and caesium-137 releases into the atmosphere from the
a review and bibliography, Atmos. Environ., 32, 947–966, 1998. Fukushima Dai-ichi nuclear power plant: determination of the
Stohl, A., and James, P.: A Lagrangian analysis of the atmospheric source term, atmospheric dispersion, and deposition, Atmos.
branch of the global water cycle: 1. Method description, valida- Chem. Phys., 12, 2313–2343, https://doi.org/10.5194/acp-12-
tion, and demonstration for the August 2002 flooding in Central 2313-2012, 2012.
Europe, J. Hydrometeorol., 5, 656–678, 2004. Stohl, A., Klimont, Z., Eckhardt, S., Kupiainen, K., Shevchenko,
Stohl, A. and Thomson, D. J.: A density correction for Lagrangian V. P., Kopeikin, V. M., and Novigatsky, A. N.: Black carbon in
particle dispersion models, Bound.-Lay. Meteorol., 90, 155–167, the Arctic: the underestimated role of gas flaring and residen-
1999. tial combustion emissions, Atmos. Chem. Phys., 13, 8833–8855,
Stohl, A. and Trickl, T.: A textbook example of long-range trans- https://doi.org/10.5194/acp-13-8833-2013, 2013.
port: Simultaneous observation of ozone maxima of stratospheric Stull, R. B.: An Introduction to Boundary Layer Meteorology,
and North American origin in the free troposphere over Europe, Kluwer Academic Publishers, Dordrecht, 1988.
J. Geophys. Res., 104, 30445–30462, 1999. Sutherland, W.: The viscosity of gases and molecular force, Philos.
Stohl, A., Wotawa, G., Seibert, P., and Kromp-Kolb, H.: Interpola- Mag., 36, 507–531, 1893.
tion errors in wind fields as a function of spatial and temporal Tateishi, R., Hoan, N. T., Kobayashi, T., Alsaaideh, B.,
resolution and their impact on different types of kinematic tra- Tana, G., and Phong, D. X.: Production of Global Land
jectories. J. Appl. Meteorol., 34, 2149–2165, 1995. Cover Data-GLCNMO2008, J. Geogr. Geol., 6, 99–122,
Stohl, A., Hittenberger, M., and Wotawa, G.: Validation of the https://doi.org/10.5539/jgg.v6n3p99, 2014.
Lagrangian particle dispersion model FLEXPART against large Tegen, I. and Fung, I.: Modeling of mineral dust in the atmosphere:
scale tracer experiment data, Atmos. Environ., 32, 4245–4264, Sources, transport, and optical thickness, J. Geophys. Res., 99,
1998. 22897–22914, https://doi.org/10.1029/94JD01928, 1994.
Stohl, A., Haimberger, L., Scheele, M. P., and Wernli, H.: An in- Thompson, R. L. and Stohl, A.: FLEXINVERT: an atmospheric
tercomparison of results from three trajectory models, Meteorol. Bayesian inversion framework for determining surface fluxes of
Appl., 8, 127–135, 2001. trace species using an optimized grid, Geosci. Model Dev., 7,
Stohl, A., Eckhardt, S., Forster, C., James, P., Spichtinger, N., and 2223–2242, https://doi.org/10.5194/gmd-7-2223-2014, 2014.
Seibert, P.: A replacement for simple back trajectory calcula- Thompson, R. L., Stohl, A., Zhou, L. X., Dlugokencky, E.,
tions in the interpretation of atmospheric trace substance mea- Fukuyama, Y., Tohjima, Y., Kim, S.-Y., Lee, H., Nisbet, E. G.,
surements, Atmos. Environ., 36, 4635–4648, 2002. Fisher, R. E., Lowry, D., Zhao, G., Weiss, R. F., Prinn, R. G.,
Stohl, A., Forster, C., Eckhardt, S., Spichtinger, N., Huntrieser, H., O’Doherty, S., Fraser, P., and White, J. W. C.: Methane emis-
Heland, J., Schlager, H., Wilhelm, S., Arnold, F., and Cooper, O.: sions in East Asia for 2000–2011 estimated using an atmo-
A backward modeling study of intercontinental pollution trans- spheric Bayesian inversion, J. Geophys. Res., 120, 4352–4369,
port using aircraft measurements, J. Geophys. Res., 108, 4370, https://doi.org/10.1002/2014JD022394, 2015.
https://doi.org/10.1029/2002JD002862, 2003. Thompson, R. L., Sasakawa, M., Machida, T., Aalto, T., Worthy,
Stohl, A., Cooper, O. R., Damoah, R., Fehsenfeld, F. C., Forster, C., D., Lavric, J. V., Lund Myhre, C., and Stohl, A.: Methane fluxes
Hsie, E.-Y., Hübler, G., Parrish, D. D., and Trainer, M.: Forecast- in the high northern latitudes for 2005–2013 estimated using a
ing for a Lagrangian aircraft campaign, Atmos. Chem. Phys., 4, Bayesian atmospheric inversion, Atmos. Chem. Phys., 17, 3553–
1113–1124, https://doi.org/10.5194/acp-4-1113-2004, 2004. 3572, https://doi.org/10.5194/acp-17-3553-2017, 2017.
Stohl, A., Forster, C., Frank, A., Seibert, P., and Wotawa, G.: Thomson, D. J.: Criteria for the selection of stochastic models of
Technical note: The Lagrangian particle dispersion model particle trajectories in turbulent flows, J. Fluid Mech., 180, 529–
FLEXPART version 6.2, Atmos. Chem. Phys., 5, 2461–2474, 556, 1987.
https://doi.org/10.5194/acp-5-2461-2005, 2005. Thomson, D. J.: A stochastic model for the motion of particle pairs
Stohl, A., Seibert, P., Arduini, J., Eckhardt, S., Fraser, P., Gre- in isotropic high-Reynolds-number turbulence, and its applica-
ally, B. R., Lunder, C., Maione, M., Mühle, J., O’Doherty, S., tion to the problem of concentration variance, J. Fluid Mech.,
Prinn, R. G., Reimann, S., Saito, T., Schmidbauer, N., Sim- 210, 113–153, 1990.

Geosci. Model Dev., 12, 4955–4997, 2019 www.geosci-model-dev.net/12/4955/2019/

I. Pisso et al.: FLEXPART version 10.4 4997

Thomson, D. J. and Wilson, J. D.: History of Lagrangian stochastic Weil, J. C.: Updating Applied Diffusion Models, Lectures on Air
models for turbulent dispersion, in: Lagrangian modelling of the Pollution Modeling, edited by: Venkatram, A. and Wyngaard, J.
atmosphere, edited by: Lin, J., Brunner, D., Gerbig, C., Stohl, C., J. Appl. Meteorol. Clim., 24, 1111–1130, 1985.
A., Luhar, A., and Webleyv, P., American Geophysical Union, Wilson, J. D. and Sawford, B. L.: Review of Lagrangian stochastic
Washington, DC, 2013. models for trajectories in the turbulent atmosphere, Bound.-Lay.
Tiedtke, M.: Representation of clouds in large-scale models, Mon. Meteorol., 78, 191–210, 1996.
Weather Rev., 121, 3040–3061, 1993. Wilson, J. D., Thurtell, G. W., and Kidd, G. E.: Numerical simu-
Uliasz, M.: Lagrangian particle dispersion modeling in mesoscale lation of particle trajectories in inhomogeneous turbulence part
applications, in: Environmental Modeling, Vol. II, edited by: II: Systems with variable turbulence velocity scale, Bound.-Lay.
Zannetti, P., Computational Mechanics Publications, Southamp- Meteorol., 21, 423–441, 1981.
ton, UK, 1994. Wotawa, G., DeGeer, L.-E., Denier, P., Kalinowski, M., Toivonen,
Venkatram, A. and Wyngaard, J. C. (Eds.): Lectures on Air H., D’Amours, R., Desiato, F., Issartel, J.-P., Langer, M., Seibert,
Pollution Modeling, American Meteorological Society, ISBN P., Frank, A., Sloan, C., and Yamazawa, H.:, Atmospheric trans-
9780933876675, 390 pp., Boston, 1988. port modelling in support of CTBT verification – overview and
Verreyken, B., Brioude, J., and Evan, S.: Development of turbu- basic concepts, Atmos. Environ., 37, 2529–2537, 2003.
lent scheme in the FLEXPART-AROME v1.2.1 Lagrangian par- Zannetti, P.: Particle Modeling and Its Application for Simulating
ticle dispersion model, Geosci. Model Dev., 12, 4245–4259, Air Pollution Phenomena, in: Environmental Modelling, edited
https://doi.org/10.5194/gmd-12-4245-2019, 2019a. by: Melli, P. and Zannetti, P., Computational Mechanics Publica-
Verreyken, B., Brioude, J., and Evan, S.: Development of tur- tions, Southampton, UK, 1992.
bulent scheme in the FLEXPART-AROME v1.2.1 Lagrangian
particle dispersion model, Geosci. Model Dev. Discuss.,
https://doi.org/10.5194/gmd-2019-89, in review, 2019b.

www.geosci-model-dev.net/12/4955/2019/ Geosci. Model Dev., 12, 4955–4997, 2019