Review

Stochastic formulation of ecological

models and their applications
Andrew J. Black1 and Alan J. McKane2
1
School of Mathematical Sciences, The University of Adelaide, Adelaide, SA 5005, Australia
2
Theoretical Physics Division, School of Physics and Astronomy, University of Manchester, Manchester, M13 9PL, UK

The increasing use of computer simulation by theoreti-

Glossary
cal ecologists started a move away from models formu-
lated at the population level towards individual-based Discretise: the procedure whereby one replaces a continuous variable, such as
the fraction of the population that consists of a particular type in a PLM, by a
models. However, many of the models studied at the discrete variable, such as the number of individuals of that type.
individual level are not analysed mathematically and Individual-based model (IBM) or individual-level model (ILM): a model in which
remain defined in terms of a computer algorithm. This is the basic entities are individuals. This description in terms of discrete variables
is in contrast to PLMs, where the description is in terms of continuous
not surprising, given that they are intrinsically stochas- population densities. The former are sometimes referred to as a microscopic
tic and require tools and techniques for their study that description and the latter as a macroscopic description.
Limit cycle: a periodic cycling in the macroscopic dynamics. Limit cycles are
may be unfamiliar to ecologists. Here, we argue that
deterministic in that they remain in phase, as contrasted with quasi-cycles,
the construction of ecological models at the individual which are phase forgetting.
level and their subsequent analysis is, in many cases, Macroscopic: description of a system on a large scale, so that the detailed
(microscopic) structure is not observable. In ecology, this will typically be a
straightforward and leads to important insights. We description at the population level involving population densities, rather than
discuss recent work that highlights the importance of individuals.
stochastic effects for parameter ranges and systems Markov process: a stochastic process where the probability of making a
transition to a new state only depends on the current state of the system, and
where it was previously thought that such effects would not on its past history.
be negligible. Master equation: the equation for a continuous-time Markov process that
specifies how the probability of the system being in a given state changes with
time. More specifically, it gives an expression for dPðn;tÞ in terms of transitions
Why should ecological models be individual based and dt
into the state n minus transitions out of this state.
stochastic? Microscopic: description of a system at a small scale in terms of the entities
Ecology was the first area of biology where quantitative making up the system. In ecology, this will typically be a description at the
individual level. The processes via which the population of individuals evolves
models were constructed, and it is still one of the most will then be probabilistic.
mathematically well developed of the biological sciences. Phenomenological: refers to the process of constructing a model. Such a
Early models were formulated at the population level, model does not have a microscopic basis, but is simply postulated in a
consistent manner using observed phenomena and general principles.
frequently as differential equations, because these were Power spectrum (or power spectral density): a real positive function of the
the only tools available at the time (Box 1). The advent of frequencies that appear in a time series. If all frequencies are equally
electronic computers led to a huge increase in the variety represented (white noise) then it is a constant (i.e. flat), whereas if the signal
is sinusoidal (i.e. only one frequency, v0, is present) then it is a spike at v0. For a
and complexity of models that can be studied, most natu- stochastic or noisy sinusoidal signal, where the typical frequency is v0, but
rally formulated at the individual rather than the popula- where there is a range of frequencies within a band v0  Dv, the power
tion level. This is clearly a better starting point for spectrum will have a peak at v0 and a width of the order of Dv.
Quasi-cycles: a stochastic phenomenon where demographic noise resulting
building ecological models as it explicitly recognises the from the individual nature of the population excites the system, giving rise to
discreteness of the population and the stochastic nature of noisy cycling in the dynamics. In general, the power spectrum of these will be
the dynamics. As such, these types of model can capture a peaked with a finite width, indicating a range of excited frequencies.
Resonance: the tendency of a system to oscillate at greater amplitude at some
larger range of phenomena than can basic population- frequencies rather than at others. If the system is subject to external forces at
level models (PLMs; see Glossary). The main drawback these frequencies, large amplitude oscillations will be produced.
Stability matrix: a matrix of partial derivatives (the Jacobian) evaluated at a
of such an approach is that the results are frequently
fixed point of a set of differential equations. If all its eigenvalues have a
numerical in nature, making any theoretical understand- negative real part, then the fixed point is stable.
ing difficult. State variables: for an IBM, these are the number of different types of
individual at a given time: ni individuals of type i, where i = 1,2,. . .. This is
In this review, we highlight the recent trend whereby frequently written as a vector n ¼ ðn1 ; n2 ; . . .Þ. The i could label species,
ecological models are formulated as stochastic Markov locations in space, age, or any other trait or attribute. For a PLM, the state
processes so they can be studied analytically as well as variables are continuous and denoted as xi, with the entire state written as
x ¼ ðx 1 ; x 2 ; . . .Þ.
numerically [1–5]. We refer to these as individual-based Stochastic process: a process where the rule for making a transition to a new
models (IBMs). This is to be contrasted with what we term state of the system at time t+1 (or t+dt, if time is continuous) from the current
‘agent-based models’, which although based on individuals, state at time t is a stochastic (i.e. random) variable. So, unlike a deterministic
process, only the probability of being in a given state n at time t can be
are often defined in terms of algorithms, thus precluding specified. This is denoted by P(n,t).
any analysis. These two approaches, along with PLMs, are Stochastic simulation algorithm (SSA) or the Gillespie algorithm: a Monte
summarised in Box 1. Carlo scheme for the simulation of a stochastic Markov chain. This scheme is
exact; the algorithm is derived from the same assumptions as the master
equation.
Corresponding author: McKane, A.J. ([email protected]).

0169-5347/$ – see front matter ß 2012 Elsevier Ltd. All rights reserved. doi:10.1016/j.tree.2012.01.014 Trends in Ecology and Evolution, June 2012, Vol. 27, No. 6 337
Review Trends in Ecology and Evolution June 2012, Vol. 27, No. 6

White noise: the term ‘noise’ refers to a stochastic (random) process and can be obtained from IBMs using these tools can eluci-
‘white’ to one where all frequencies are equally represented. It draws its name
date well-known, but poorly characterised, phenomena.
from the idea that colours of different frequencies are equally represented in
white light. There are many empirical studies that have attributed
their findings to stochastic effects [6–11], making it an
important long-term goal to develop a sound theoretical
We first review what constitutes an IBM; we look at the understanding of such effects. One of the major points to
construction of IBMs and discuss setting up the (stochastic) emerge from this line of research is the importance of
dynamics of the system. This is illustrated using several stochastic effects in large populations, not just small
different types of ecological model. The fundamental equa- ones, as has been traditionally assumed in the study of
tion governing the dynamics of the stochastic process is the extinction effects.
so-called ‘master equation’, which is simply a continuous
time version of a Markov chain. However, when the IBM is Creating stochastic models
viewed in this way, powerful techniques can be used to Beginning with IBMs, and deducing PLMs from them, is
analyse the master equation, leading to quantitative pre- the approach taken in many of the physical sciences. In
dictions for several interesting ecological problems. chemistry, for example, reaction kinetics at the molecular
We emphasise the importance of deriving PLMs from level yield rate equations at the macroscale and, in physics,
IBMs, and discuss a variety of examples drawn from the statistical mechanical models at the microscale lead to a
literature. We show how the concrete predictions that thermodynamical description at the macroscale. In ecologi-
cal IBMs, random events (e.g. birth, death and predation)
Box 1. Three approaches to ecological modelling at the level of individuals (the microscale) give rise to
macroscopic dynamics for large populations of individuals.
Population-level models
In the limit of an infinite population, these dynamics
PLMs are typically formulated as ordinary differential equations for
the fraction of the population that is of a certain kind. For example, become deterministic and can be described by a set of
Levins’ original metapopulation model is written as Equation I [78]: ordinary differential equations: a PLM. As we discuss
later, when the population is large but still finite, these
dQ
¼ cQð1  QÞ  eQ; [I] dynamics are often still strongly stochastic. Before we do
dt
this, we first discuss the construction of IBMs and some
where Q is the fraction of patches that are occupied and c and e are methods of analysis.
effective parameters. This is a phenomenological equation; that is,
Suppose that one wishes to describe an ecological sys-
written down consistently, but not derived from a more basic model.
The variables, such as Q, are continuous, implicitly assuming an tem that contains m different species, and the only way
infinite population size, and thus the results are deterministic (non- that individuals will be labelled is as belonging to a given
random). The advantage of this approach is that the well-developed species. If the population size is assumed to be finite, equal
mathematics of dynamical systems is applicable. This is still consid- to N, then the state of the system is given by a vector of
ered the dominant methodology, with many textbooks available
integers n ¼ ðn1 ; . . . ; nm Þ, where ni is the number of indi-
[40,47]. If one starts with this approach, then the question of how
to include stochastic effects arises. Some authors discretise the viduals of species i at that time. This could also be the
equations whereas others add stochastic (noise) terms on to the number of individuals of species i per unit area, or some
equations. Both of these approaches are ill defined. other measure.
Given that the basic idea of model specification can be
Agent-based models
Agent-based models have a finite number of individuals in the
illustrated on a system with just one species, we focus on a
system, each typically having many attributes or degrees of freedom system with n identical individuals of type A. Suppose also
[79,80]. These are frequently defined in terms of computer that the only processes are death and (asexual) birth. We
algorithms ready for simulation. The advantages of models of this represent these as Equations 1 and 2, respectively:
type are the potentially unlimited amounts of detail that can be
included and the ease of programming. The main disadvantage is d
A!E; [1]
that the algorithmic approach can only produce numerical results; in
general, they are too complex for mathematical analysis. b
A þ E!A þ A [2]
Individual-based models
IBMs are also composed of a finite number of individuals, but Equation 1 indicates that an individual of type A dies at
typically with a smaller number of attributes. The rules for the time a rate d to give a vacancy (E for ‘empty’); Equation 2 shows
evolution of the individuals within the system are formulated that if there is a vacancy, another individual of the same
probabilistically, usually as a Markov process. Therefore, state
variables are discrete instead of continuous and, by definition, the
type is born at a rate b. The rate at which the number of
model is stochastic. In an IBM version of the metapopulation model individuals reduces from n to (n–1) is then T(n  1|n) = dn
(Equation I), n would be the number of patches and the dynamics of and the rate at which it increases from n to (n+1) is shown
colonisation and extinction of patches are random events that in Equation 3:
depend on n (explicit details of this model are given in Box 2). 
The population is usually divided up into a small number of n
Tðn þ 1jnÞ ¼ bn 1  [3]
classes, reducing the degrees of freedom; thus, all members of the N
same class are indistinguishable. The probabilistic formulation of
IBMs means they can be simulated exactly with Monte Carlo This last result assumes a well-mixed population and is
techniques [12], but most importantly they can be analysed proportional to the probability of an individual and a
analytically via the master equation. The corresponding PLM to a vacancy existing in the system. These are the fundamental
given IBM can also be analytically derived.
ingredients that underpin the dynamics. We should

338
Review Trends in Ecology and Evolution June 2012, Vol. 27, No. 6

emphasise that these rates depend on N and thus so do the describes the effects of demographic stochasticity. This
dynamics of the model. This is not true of a PLM. Several approximation is well known in the literature, and goes
examples of IBMs of increasing complexity are given in under several names: the van Kampen system size expan-
Box 2. sion [15], the Gaussian or diffusion approximation, or
Kurtz’s theorem [25,26]. The method is discussed in more
Analysis of stochastic models detail in Box 3, but essentially consists of approximating
Once a model has been specified, the complete stochastic the probability density in the master equation with a
dynamics are encoded analytically by the master equa- Gaussian distribution. The power of this technique is that,
tion, or individual realisations can be obtained by Monte starting from the master equation, it provides a very
Carlo simulation, which outputs a stochastic time series. general method of deriving equations describing the mac-
These are the two main ways of investigating and solving roscopic stochastic dynamics.
these models (i.e. obtaining a complete description of the
dynamics). Population level descriptions follow from individual
The stochastic simulation algorithm (SSA) developed by level descriptions
Gillespie provides a method for the exact simulation of a In an individual-based formulation, it is always the case
master equation [12]. This type of simulation is the usual that a finite population of N individuals is dealt with,
way of investigating these models, mainly because of its which evolves via several random processes (e.g. birth,
simplicity to program, but an important point is that the death and predation). For small N, the dynamics will be
SSA and master equation are both derived from the same random. When the population becomes large, the dynamics
underlying Markovian assumptions, so that there is an will start to become deterministic (non-random). One way
exact correspondence between the two. Thus, any model of investigating this behaviour is just to simulate large
that can be simulated in this way can be written as a systems and average the results. Far more insight can be
master equation and vice versa. Of course, there are the found by analytically deriving the corresponding PLM (a
usual drawbacks to just using simulations to investigate a set of differential equations) from the IBM in the limit of
model, such as the costs in terms of computing power and infinite N (Boxes 1 and 3).
time, the numerical nature of the results and the difficulty This methodology clearly has several advantages. First,
in investigating rare events [13,14]. the predictions of the IBM and PLM can be compared
In general, simulations provide individual realisations exactly; thus, it is possible to see under what conditions
of a stochastic process. The master equation (also known as a PLM will be a valid description of a given system. It is
Kolmogorov’s forward equation) describes analytically the also helpful in writing down a PLM when the ecology of the
temporal evolution of the probability density of being in a system is complicated [27]. In general, it is easier to
particular state n, thus encoding the full stochastic dy- consider the events that define a system, write down an
namics [12,15]. If it could be solved (for the probability IBM and then derive the corresponding PLM. Often these
density), then one would have a complete description of the derived equations may differ from the naively postulated
properties and the dynamics of the stochastic system equations in a simple population-level approach [28–31].
[4,16]. One calculation that is straightforward and can This also has the advantage that the explicit microscopic
always be carried out is to determine the macroscopic basis for the PLM is known and so more than just a
behaviour from the master equation, which is the phenomenological analysis is carried out.
N ! 1 limit, as discussed in Box 3, and which yields Unfortunately, this procedure is often reversed: a well-
the PLM corresponding to the original IBM. studied PLM is discretised, interpreted as an IBM and
The master equation cannot be solved analytically for then simulated to investigate various stochastic effects.
most cases of interest, and so several approximation meth- This is exactly the wrong way round: in general, there will
ods have been developed [17]. These can be extended if the be many IBMs (sometimes infinitely many) giving a par-
system involves a network [18–21]. If the population and ticular PLM, and so the PLM should always be derived
the number of degrees of freedom are small, then it can be from the IBM [31,32]. The reason why many IBMs give the
solved numerically [22]. Epidemic models with household same PLM is that IBMs define a stochastic process and
structure are another interesting example where a master PLMs are equations for the averages of this process; how-
equation is used to describe the spread of infection within a ever, many stochastic processes have the same averages.
household, but situated within an infinite population of
such households, which allows the derivation of several Applications of IBMs described by master equations
population-level results [23,24]. Other ecological systems The modelling approach we have described has been ap-
have this kind of hierarchical structure (e.g. involving plied to several problems over the past few years, and there
collections of local populations) for which similar techni- is a clear trend towards the increasing use of this method-
ques might be applicable. ology. For example, in the case of the neutral model dis-
We cannot discuss all these methods in this review, so we cussed in Box 2, the quantity that has been studied most
concentrate on techniques that set up an expansion of the intensively is the species abundance distribution. This is a
master equation in increasing powers of p1ffiffiffi
N
, where N is, for plot of the number of species with 1,2,. . .,m individuals in a
example, the total population. If N is large, which is typical given local community. It can be calculated exactly in the
in ecological systems, then solutions of the master equation classic model starting from the master equation [33], and
can be approximated by just two terms, which correspond fits well with empirical data [34–38]. Rather than discuss
to a deterministic part plus a stochastic correction that more specific examples, we review three broad aspects.
339
Review Trends in Ecology and Evolution June 2012, Vol. 27, No. 6

Box 2. Examples of ecological IBMs

An advantage of IBMs is that specifying them is simple and they are Table I. Neutral model.
straightforward to motivate biologically, with additional features
Event Transition
being easily incorporated. Here, we give several examples, three of
which are illustrated in Figure I. Birth–death 1m
A þ B! A þ A,
1m
A þ B! B þ B;
Neutral models m m
Immigration A!B, B!A:
The ‘classic’ neutral theory, introduced by Hubbell [81], consists of a
local community with birth–death processes of the Moran type [82]
and immigration from a metacommunity, which acts as a well-mixed
source pool of potential immigrants [83,84]. Given that all individuals Table II. Metapopulation model.
are assumed to have the same birth and death rates, we may fix our
Event Transition
attention on one species, A, and denote all other species as B. Then
Colonisation c
the transitions are as given in Table I. A þ E!A þ A;
Here, the rates do not include the combinatoric factor that gives the Extinction e
A!E,
probability of choosing individuals of the desired species to interact. m
Migration E!A:
The quantity m is the probability that the replacement of an individual
is the result of an immigration event, rather than by a birth–death
event. The probability of a particular immigrant being chosen is given
by the relative abundance of that particular species in the metacom-
Table III. Predator-prey model.
munity, which can also be described as an IBM with birth–death and
speciation [83]. This model is only implicitly spatial, but it can be Event Transition
extended in many ways; for example, by making it explicitly spatial Linear growth b
P!P þ P;
[84], incorporating a chain of islands [85] or a network structure [86]. e
Allee effect P þ P!P þ P þ P;
Predation 1 p1
Metapopulation models P þ H!H;
Metapopulations are sets of fragmented local populations connected Predation 2 p2
by migration [87]. At the simplest level of description, the patches can P þ H!H þ H;
be thought of as being either occupied or unoccupied [2,16]. If A Competition H þ H!H:
d

labels the occupied patches and E the unoccupied ones, then the
possible events are as detailed in Table II.
This leads to a simple one-dimensional master equation, where the
state variable is the number of occupied patches. The same transition by Levin and Segal [89]. An IBM that gives this model in the
scheme applies to many models, such as logistic growth and the SIS deterministic limit is given in [5]; within a patch, there are five
model [17]. Again, this simple model has been elaborated on; for processes between the plankton, P, and the herbivores, H (Table III).
instance, by making the number of colonisable patches into a The model can simply be taken to describe the dynamics on a large
dynamic variable [2,88]. well-mixed patch, but it can be made spatial by allowing both the P
and H to jump between patches on a regular lattice. The introduction
Spatial predator–prey model of spatial variation introduces no new points of principle in the
An example of a reaction–diffusion equation showing Turing patterns modelling procedure, although the analysis typically becomes more
can be found in the description of plankton–herbivore dynamics given complex [34,90].

(a) (c) p
Competition Allee effect

Growth of P
Colonisation Predation 2
Extinction Predation
redation 1

Immigration
N patches

Reproduction
(b) Homogeneous patch

Immigration

Migration Migration
Lattice of patches
TRENDS in Ecology & Evolution

Figure I. Three examples of ecological individual-based models (IBMs). (a) A simple metapopulation model. (b) A neutral island chain [85]. (c) The Levin–Segal model
[5,89], which describes the explicitly spatial plankton–herbivore dynamics.

340
Review Trends in Ecology and Evolution June 2012, Vol. 27, No. 6

Box 3. The master equation and the emergence of macroscopic dynamics

The transition rates T(n + 1jn) and T(n - 1jn) define the model. From replace the spike with a Gaussian. The system would now be
these, the probability of finding n individuals in the system at time t, stochastic, being described by a probability distribution function,
P(n,t) can be calculated from Equation I: although of the simplest kind. The width of the Gaussian would be
expected to scale as p1ffiffiffi
N
which means effectively writing Nn ¼ x þ pjffiffiffi
N
,
dPðn; tÞ where x ¼ hNni and j is a stochastic variable. This is the content of the
¼ T ðnjn þ 1ÞPðn þ 1; tÞ þ T ðnjn  1ÞPðn  1; tÞ
dt van Kampen system-size expansion. It turns out to be an excellent
 ½T ðn  1jnÞ þ T ðn þ 1jnÞPðn; tÞ: [I] approximation (far better than might naively be expected) away from
boundaries. Furthermore, given that Gaussian probability distribu-
This is the equation that governs the dynamics of the system. It is the tions are derived from linear stochastic differential equations, the
equation for a Markov chain in continuous time, and is called the whole theory is linear and so can be treated exactly, even though the
master equation by physical scientists. Although we have only underlying stochastic processes may be highly nonlinear.
discussed the case of a single species, all the formalism naturally For most interesting applications, more than one degree of freedom
generalises. We can replace n by n everywhere in the master equation is needed, but the whole theory generalises in this case to a set of
where n is a vector with components that can represent any set of stochastic differential equations for ji : (Equation II)
species in a large collection of spatial patches and having any number
of other attributes. dji X
The master equation contains far more information than the usual ¼ ai j j j ðtÞ þ hi ðtÞ; [II]
dt j
differential equations written down phenomenologically in terms of
the population density. The latter can be found by simply taking the where hi(t) is a Gaussian white noise with zero mean, which is the only
average of Nn ; all the detail of the stochastic fluctuations is lost. Doing remnant of the demographic stochasticity of the system. It has a
this for the metapopulation model described in Box 2 gives the well- correlation function given by a matrix bij. So the whole theory is given
known logistic equation [40]. by two matrices aij and bij, which can be systematically calculated from
In the deterministic description, the probability distribution, P(n,t), the transition rates that define the model. Figure I illustrates this theory
is a spike at n ¼ hni. The simplest approximation beyond this is to using the metapopulation model described in Box 2.

(a) 100 (b)

0.20
80
0.15
Patches

60 P
0.10
40
0.05
20
0.00
0 100
0 5 10 15 20 5
Time Time 50 Patches
10
0

(c) (d) 0.06

1400
1200
0.04
1000
Patches

800 P
600 0.02
400
200 0.00
0
0 5 10 15 20 5 1500
Time Time 1000
500
10 0 Patches

TRENDS in Ecology & Evolution

Figure I. Emergence of macroscopic behaviour. The number of occupied patches for the metapopulation model for (a) N = 100, and (c) N = 1500 carrying capacity. The
broken red lines show the deterministic result and the blue lines are from stochastic simulations. (b) and (d) show the evolution of the probability density for N = 100 and
N = 1500, respectively. The peak of the distributions move with the deterministic result and the width is given by the solution of the stochastic differential equation
1
(Equation II). As the system gets larger, the amplitude of the fluctuations decreases as N 2 . In the limit N!1, the probability density becomes a spike.

Quasi-cycles matrix at this fixed point must have at least one pair of
A common feature of some IBMs (with two or more degrees complex eigenvalues. However, although the oscillations in
of freedom) is the ability of the demographic stochasticity the deterministic system are damped and so die away, in
to excite macroscopic-scale coherent oscillations, known as the full (stochastic) system the demographic stochasticity
quasi-cycles [1,39]. An example of these is shown in acts as a forcing term and sustains the oscillations.
Figure 1. A prerequisite for quasi-cycles to occur in a given From a mathematical point of view, this effect is de-
IBM is that there is a stable fixed point in the correspond- scribed by Equation II of Box 3. A physical analogy is a
ing PLM that is approached in an oscillatory manner. pendulum that is lightly damped, so eventually any oscil-
Expressed mathematically, this means that the stability lations will die away. However, if it is randomly bombarded
341
 Review Trends in Ecology and Evolution June 2012, Vol. 27, No. 6

(a) 2000 (b) 120

100

Infectives
1500
80

Power
1000 60
40
500
20
0 0
0 5 10 15 20 0.0 0.2 0.4 0.6 0.8 1.0
Time (years) Frequency (1/years)
TRENDS in Ecology & Evolution

Figure 1. An example of quasi-cycles in a stochastic model. The blue and purple lines in (a) show two realisations of a stochastic individual-based model (IBM) [the
susceptible–infectious–recovered (SIR) model [3]] that are started from the same initial condition. The broken red line shows the result using the corresponding population-
level model (PLM). Whereas the oscillations are damped, tending to a stable fixed point in the population-level version, they are maintained in the stochastic IBM. These
stochastic oscillations are different from limit cycles in that they are phase forgetting. This can be seen in the two realisations that, although started from the same initial
condition, soon go out of phase. This is reflected in the power spectrum of the time series, shown in (b), which has a finite width. All frequencies shown in the spectrum are
excited in the time-series, but the position of the peak indicates the dominant frequencies that are amplified the most.

by weak white noise, these oscillations will be sustained. In patterns are found in ecosystems [45,46]. This would be
addition, the amplitude of these oscillations will be much expected on purely theoretical grounds, given that spatial
bigger than might naively be expected, because in among ecosystem dynamics is frequently described by reaction–
all the frequencies contained in the white noise will be one diffusion equations [47]. However, just as demographic
that corresponds to the natural frequency of the pendulum. stochasticity leads to stochastic oscillations where a deter-
This resonance effect has been called ‘stochastic amplifica- ministic analysis finds only static behaviour, it also leads to
tion’. Quasi-cycles differ from limit cycles in that they are a
stochastic phenomenon and provide a simple robust mech- Box 4. The power spectrum
anism that generates cyclic behaviour without the need for
additional complexities, as are often assumed, to generate The stochastic cycles found in the IBMs discussed in this review do
not have a single period, but a distribution of periods centred about
cyclic behaviour in deterministic PLMs [40]. an average value. This, together with the fluctuations in the
Figure 1 shows two realisations of a stochastic model amplitude of the cycles, means that when the time series from a
and it is clearly seen that, after some time, these go out of large number of realisations, ji(t), are averaged over, they average
phase, unlike true limit cycles. One of the most powerful out to zero. Yet, a single realisation corresponds to what will be seen
tools for analysing such oscillations is the power spectrum in an experiment, and so some averaging mechanism is needed that
will not wipe out the cycles. The simplest way to achieve this is to
[10,41]. This shows how the different frequencies that take the Fourier transform of the time series [41] (Equation I):
make up the time-series are distributed. Quasi-cycles
Z þ1
are characterised by a strongly peaked power spectrum, j̃i ðvÞ ¼ ji ðtÞe ivt dt [I]
which has a finite width. The width is important as it 1

shows how coherent the cycles are: the smaller the width, The frequencies centred around that corresponding to the average
the longer cycles remain in phase. The definition and period will have the largest magnitude, and so taking theDmodulus  2 E
meaning of the power spectrum are discussed in Box 4. squared of the Fourier transform and then averaging: j̃i ðvÞ ;
An analytic approach using the master equation is should give a smooth function spread about the characteristic
particularly useful for the characterisation of these cycles. frequency of the system. This is the power spectrum of the
fluctuations. The spread of the power spectrum will show how
From the stochastic differential equations derived using coherent the fluctuations are. If they were deterministic and
the system-size expansion, one can easily derive an ana- sinusoidal, the power spectrum would be a spike.
lytic expression for the power spectrum, again described in Although some ecologists were well aware of the existence of
more mathematical detail in Box 4. This allows the easy sustained oscillations, the effect has not been studied within a well-
defined mathematical scheme; neither were the power spectra
investigation of this phenomenon throughout a wide pa-
analysed carefully to learn more about the nature of the fluctuations.
rameter range as well as a vital check against the results of A range of phenomena have been revealed through more detailed
simulations. Other related questions that have been recent studies. First, the height of the peaks of the power spectra
addressed are: what is the connection of quasi-cycles with (reflecting the amplitude of the oscillations) are much larger than
limit cycles, which are the conventional way of describing might be expected owing to a resonance effect: all frequencies are
present in the white noise h(t) and this resonates with the
oscillations in nonlinear dynamical systems [3,42]?; and
characteristic frequency of the system. Second, this characteristic
are there characteristic signatures of the two types of frequency is not the same as the frequency of the decay of
oscillation that can be used to tell them apart in ecological perturbations to the deterministic system [1]. The latter is given by
time series [43]? jImlj; where l is a complex eigenvalue of the Jacobian of
fluctuations about the stationary state. Instead, the peak of the
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
Spatial patterns power spectrum is approximately at ðIm lÞ2  ðRe lÞ2 , and the
In 1952, Turing predicted that systems that consisted of precise value also depends on the bij. Third, it was not clear how the
agents of at least two different species that reacted together stochastic cycles related to conventional (deterministic) limit cycles,
but this has now been understood [42]. Finally, the mathematical
and spatially diffused could, under certain circumstances,
systematisation has led to the effect being found in other areas; for
give rise to patterns, now known as Turing patterns [44]. example, stochastic Turing patterns [5,51].
Field observations are now providing evidence that Turing
342
 Review Trends in Ecology and Evolution June 2012, Vol. 27, No. 6

(a) (b)
88
700 700
80
600 600
72

500 500 64

Population
56
400 400
48
300 300
40
200 200 32

100 100 24

16
0 0
0 100 200 300 400 500 600 700 0 100 200 300 400 500 600 700
TRENDS in Ecology & Evolution

Figure 2. Quasi-pattern formation in the stochastic Levin–Segal model as described in Box 2. (a) shows a population map of the model in the quasi-pattern phase. (b) shows
the same data, but randomised. If the patchiness were a statistical artefact, then it would be preserved in the randomised version. Reproduced, with permission, from [5].

stochastic Turing patterns in spatial systems, where reac- range of dynamics observed in both time-series case
tion–diffusion equations would indicate that none should report data and simple models is rich and understanding
exist. This has been shown in a variety of systems [5,48–50]; the mechanistic basis for it is still an active research topic
moreover, the range of parameters for which stochastic [59].
Turing patterns exist is usually much larger than for con- There are several potential mechanisms that could give
ventional Turing patterns, which have restrictions such as rise to the observed dynamics. One of the most fascinating
the diffusion constants of the two species being at least an questions is whether these mechanisms are deterministic
order of magnitude different to each other. An example of in origin, for example where noise causes the system to
this phenomenon is shown in Figure 2. switch between deterministic states, which could be un-
The mechanism that gives rise to stochastic Turing derstood using just a deterministic model [60]. This is in
patterns is essentially the same as that which gives rise contrast to the situation where noise plays an active role
to the stochastic oscillations discussed earlier. Now the that cannot be explained with a deterministic model [61–
model is spatial, and so the noise is a function of time and 63]. As typical of the whole field, older studies have tended
space, or of frequency, v, and wavenumber k, in the Fourier to use an ad hoc collection of methods to try and ‘disentan-
representation. The noise is still white, so all frequencies gle’ stochastic and deterministic elements [9], with a reli-
and wavenumbers are excited. We can now repeat the ance on simulations, leading to much debate and confusion
argument given above for the origin of stochastic oscilla- over the supposed role of stochasticity [62]. Newer studies
tions, replacing v with k. If there is a peak in the power using an IBM approach have started to provide a more
spectrum at a non-zero k, say kmax, then there is a natural systematic account of these models and their dynamics,
wavenumber of the system that has been excited by the where the macroscopic dynamics, including deterministic
noise and is now sustained rather than decaying. In the elements, is instead viewed as emergent from the micro-
conventional Turing description, this decaying mode would scopic dynamics.
signify that the homogeneous state was stable to perturba- In a previous subsection, we discussed the phenomenon
tions at k = kmax. The noise produces sustained oscillations of quasi-cycles about a deterministic fixed point. In models
(in space), which gives rise to spatial patterns with a of recurrent epidemics, there is a strong forcing on the
characteristic wavenumber kmax. It is probable that at system, which in turn leads to a limit cycle in the deter-
least some of the Turing patterns observed in nature are ministic dynamics, possibly with multiple stable attractors
of this type [5,34,51]. [60,63]. The previous analysis can be carried over, allowing
one to derive the power spectrum of the forced model [3,42],
Recurrent epidemics which again affords a view of the stochastic dynamics
IBMs have a long history in epidemiological modelling, ideally suited for understanding the underlying processes.
beginning with Bartlett’s pioneering work [52]. Many In these models, the noise excites the transient dynamics
epidemic models are most intuitively formulated on the about the limit cycle, so the resulting macroscopic time
individual level and master equation methods have been series are a superposition of these and the deterministic
a popular tool to investigate static properties [53,54]. The limit cycle. The relative proportions of the two periodicities
dynamics of childhood diseases, such as measles, whoop- reflected in the power spectrum then depend on the stabil-
ing cough and rubella [55–57], provide particularly inter- ity of the limit cycle and the size of the system. Such an
esting case studies because they are subject to both analysis can encompass and explain the differing periodi-
external forcing [58], owing to the aggregation of children cities of whooping cough and measles before and after mass
in schools, and demographic stochasticity. The resulting vaccination [3,60,64,65].
343
Review Trends in Ecology and Evolution June 2012, Vol. 27, No. 6

Concluding remarks 9 Rohani, P. et al. (2002) The interplay between determinism and
stochasticity in childhood diseases. Am. Nat. 159, 469–481
Our aim in this article was to highlight the trend towards
10 Reuman, D.C. et al. (2006) Power spectra reveal the influence of
models in ecology being individual based. There are several stochasticity on nonlinear population dynamics. Proc. Nat. Acad.
advantages in using stochastic models of this kind: they are Sci. U.S.A. 103, 18860–18865
easier to construct than are PLMs; they are more intuitive; 11 Krkosek, M. et al. (2011) Cycles, stochasticity and density dependence
and they predict phenomena that deterministic models in pink salmon population dynamics. Proc. R. Soc. B 278, 2060–2068
12 Gillespie, D.T. (1992) Markov Processes: An Introduction for Physical
miss. The importance of stochastic effects has long been
Scientists, Academic Press
known, going back to the earliest Monte Carlo simulations 13 Assaf, M. and Meerson, B. (2010) Extinction of metastable stochastic
of epidemic spreading and fade-out [52], but computational populations. Phys. Rev. E 81, 021116
investigations have outpaced the theoretical understand- 14 Ovaskainen, O. and Meerson, B. (2010) Stochastic models of population
ing of such models. Approaches based on the master equa- extinction. Trends Ecol. Evol. 25, 643–652
15 van Kampen, N.G. (1992) Stochastic Processes in Physics and
tion are starting to remedy this and we have given here an Chemistry, Elsevier
overview of the philosophy and methodology of these. We 16 Alonso, D. and McKane, A.J. (2002) Extinction dynamics in mainland-
expect that future research will continue to develop tech- island metapopulations: an N-patch stochastic model. Bull. Math. Biol.
niques based on this starting point. Several other areas are 64, 913–958
already using these techniques successfully; for instance, 17 Newman, T.J. et al. (2004) Extinction times and moment closure in the
stochastic logistic process. Theor. Popul. Biol. 65, 115–126
coevolutionary dynamics [66–68], cellular biology and ge- 18 Simoes, M. et al. (2008) Stochastic fluctuations in epidemics on
netics [69–71]. The construction of biological models is now networks. J. R. Soc. Interface 5, 555–566
driving the use of new techniques, such as incorporating 19 Rozhnova, G. and Nunes, A. (2009) Cluster approximations for
non-Markov processes into the formalism [64,72] and pa- infection dynamics on random networks. Phys. Rev. E 80, 051915
20 Sharkey, K.J. (2008) Deterministic epidemiological models at the
rameter estimation, which follows elegantly on from the
individual level. J. Math. Biol. 57, 311–331
master equation and Gaussian approximation [73]. 21 Dangerfield, C.E. et al. (2009) Integrating stochasticity and network
We expect that the future trend will be towards whole structure into an epidemic model. J. R. Soc. Interface 6, 761–774
classes of model that interpolate between these different 22 Keeling, M.J. and Ross, J.V. (2008) On methods for studying stochastic
traditions of model building. Thus, agent-based models disease dynamics. J. R. Soc. Interface 5, 171–181
23 Goshal, G. et al. (2004) SIS epidemics with household structure: the
would be simplified by construction of a chain of related
self-consistent field method. Math. Biosci. 190, 71–85
models that would terminate with a relatively simple IBM, 24 Ross, J.V. et al. (2010) Calculation of disease dynamics in a population
which in turn could be studied analytically and its deter- of households. PLoS ONE 5, e9666
ministic limit compared with the differential equations 25 Kurtz, T. (1978) Strong approximation theorems for density dependent
that are generally postulated phenomenologically. The Markov chains. Stoch. Process Appl. 6, 223–240
26 Baxendale, P.H. and Greenwood, P.E. (2010) Sustained oscillations for
range of models continues to grow and diversify. Some density dependent Markov processes. J. Math. Biol. 63, 433–457
are now becoming significantly more complicated and are 27 Wang, Z. and Goldenfeld, N. (2010) Fixed points and limit cycles in the
beginning to bridge the gap with agent-based models [74– population dynamics of lysogenic viruses and their hosts. Phys. Rev. E
77]. This would pave the way to a more unified and coher- 82, 011918
ent approach to model building in ecology, leading to 28 Goutsias, J. (2007) Classical versus stochastic kinetics modeling of
biochemical reaction systems. Biophys. J. 92, 2350–2365
greater predictive power. 29 Grima, R. (2009) Noise-induced breakdown of the Michaelis-Menten
equation in steady-state conditions. Phys. Rev. Lett. 102, 218103
Acknowledgements 30 Campillo, F. et al. (2011) Stochastic modeling of the chemostat. Ecol.
A.J.B. acknowledges support from the EPSRC and the Australian Model. 222, 2676–2689
Research Council’s Discovery Projects funding scheme (project number 31 McKane, A.J. and Newman, T.J. (2004) Stochastic models in
DP110102893). population biology and their deterministic analogs. Phys. Rev. E 70,
041902
References 32 Durrett, R. and Levin, S. (1994) The importance of being discrete (and
1 McKane, A.J. and Newman, T.J. (2005) Predator–prey cycles from spatial). Theor. Popul. Biol. 46, 363–394
resonant amplification of demographic stochasticity. Phys. Rev. Lett. 33 McKane, A.J. et al. (2004) Analytic solution of Hubbell’s model of local
94, 218102 community dynamics. Theor. Popul. Biol. 65, 67–73
2 Ross, J.V. (2006) Stochastic models for mainland–island 34 Antonovics, J. and Edwards, M. (2011) Spatio-temporal dynamics of
metapopulations in static and dynamic landscapes. Bull. Math. Biol. bumblebee nest parasites (Bombus subgenus Psythirus ssp.) and their
68, 417–449 hosts (Bombus spp.). J. Anim. Ecol. 80, 999–1011
3 Black, A.J. and McKane, A.J. (2010) Stochastic amplification in an 35 Volkov, I. et al. (2005) Density dependence explains tree species
epidemic model with seasonal forcing. J. Theor. Biol. 267, 85–94 abundance and diversity in tropical forests. Nature 438, 658–661
4 Shahrezaei, V. and Swain, P.S. (2008) Analytical distributions 36 Muneepeerakul, R. et al. (2008) Neutral metacommunity models
for stochastic gene expression. Proc. Natl. Acad. Sci. U.S.A. 105, predict fish diversity patterns in Mississippi–Missouri basin. Nature
17256–17261 453, 220–222
5 Butler, T. and Goldenfeld, N. (2011) Fluctuation-driven Turing 37 Rosindell, J. et al. (2010) Protracted speciation revitalizes the neutral
patterns. Phys. Rev. E 84, 011112 theory of biodiversity. Ecol. Lett. 13, 716–727
6 Higgins, K. et al. (1997) Stochastic dynamics and deterministic 38 Vanpeteghem, D. and Haegeman, B. (2010) An analytic approach to
skeletons: population behavior of Dungeness crab. Science 276, spatio-temporal dynamics of neutral community models. J. Math. Biol.
1431–1435 61, 323–357
7 Stenseth, N.C. et al. (1998) Phase- and density-dependent population 39 Kuske, R. et al. (2007) Sustained oscillations via coherence resonance
dynamics in Norwegian lemmings: interaction between deterministic in SIR. J. Theor. Biol. 245, 459–469
and stochastic processes. Proc. R. Soc. B 265, 1957–1968 40 Brauer, F. and Castillo-Chavez, C. (2001) Mathematical Models in
8 Bjornstad, O.N. and Grenfell, B.T. (2001) Noisy clockwork: time Population Biology and Epidemiology, Springer
series analysis of population fluctuations in animals. Science 293, 41 Priestley, M.B. (1982) Spectral Analysis and Time Series, Academic
638–643 Press

344
Review Trends in Ecology and Evolution June 2012, Vol. 27, No. 6

42 Boland, R.P. et al. (2009) Limit cycles, complex Floquet multipliers and 67 Reichenbach, T. et al. (2007) Mobility promotes and jeopardizes
intrinsic noise. Phys. Rev. E 79, 051131 biodiversity in rock–paper–scissors games. Nature 448, 1046–1049
43 Pineda-Krch, M. et al. (2007) A tale of two cycles – distinguishing quasi- 68 Altrock, P.M. et al. (2010) Stochastic slowdown in evolutionary
cycles and limit cycles in finite predator–prey populations. Oikos 116, processes. Phys. Rev. E 82, 011925
53–64 69 Thomas, P. et al. (2010) Stochastic theory of large-scale enzyme-
44 Turing, A.M. (1952) The chemical basis of morphogenesis. Philos. reaction networks: finite copy number corrections to rate equation
Trans. R. Soc. B 237, 37–72 models. J. Chem. Phys. 133, 195101
45 Maron, J.L. and Harrison, S. (1997) Spatial pattern formation in an 70 Assaf, M. et al. (2011) Determining the stability of genetic switches:
insect host-parasitoid system. Science 278, 1619–1621 explicitly accounting for mRNA noise. Phys. Rev. Lett. 106, 248102
46 Reitkerk, M. and van de Koppel, J. (2008) Regular pattern formation in 71 Waclaw, B. et al. (2011) Dynamical phase transition in a model for
real ecosystems. Trends Ecol. Evol. 23, 169–175 evolution with migration. Phys. Rev. Lett. 105, 268101
47 Murray, J.D. (2002) Mathematical Biology, II: Spatial Models and 72 Lloyd, A.L. (2001) Destabilization of epidemic models with the
Biomedical Applications, Springer-Verlag inclusion of realistic distributions of infectious periods. Proc. R. Soc.
48 Butler, T. and Goldenfeld, N. (2009) Robust ecological pattern Lond. B 268, 985–993
formation induced by demographic noise. Phys. Rev. E 80, 030902 73 Ross, J.V. et al. (2009) On parameter estimation in population models
49 Biancalani, T. et al. (2010) Stochastic Turing patterns in the II: multi-dimensional processes and transient dynamics. Theor. Popul.
Brusselator model. Phys. Rev. E 81, 046215 Biol. 75, 123–132
50 Scott, M. et al. (2010) Approximating intrinsic noise in continuous 74 Powell, C.R. and Boland, R.P. (2009) The effects of stochastic
multispecies models. Proc. R. Soc. A 467, 718–737 population dynamics on food web structure. J. Theor. Biol. 257,
51 Woolley, T.E. et al. (2011) Power spectra methods for a stochastic 170–180
description of diffusion on deterministically growing domains. Phys. 75 Murase, Y. et al. (2010) Effects of demographic stochasticity on
Rev. E 84, 021915 biological community assembly on evolutionary time scales. Phys.
52 Bartlett, M.S. (1960) Stochastic Population Models in Ecology and Rev. E 81, 041908
Epidemiology, Methuen 76 Galla, T. (2010) Independence and interdependence in the nest-site
53 Nasell, I. (2002) Stochastic models of some endemic infections. Math. choice by honeybee swarms: agent-based models, analytical
Biosci. 179, 1–19 approaches and pattern formation. J. Theor. Biol. 262, 186–196
54 Stollenwerk, N. and Jansen, V.A.A. (2003) Meningitis, pathogenicity 77 Spence, L.A. et al. (2011) Disturbance affects short-term facilitation,
near criticality: the epidemiology of meningococcal disease as a model but not long-term saturation, of exotic plant invasion in New Zealand
for accidental pathogens. J. Theor. Biol. 222, 347–359 forest. Proc. R. Soc. B 278, 1457–1466
55 Bauch, C.T. and Earn, D.J.D. (2003) Transients and attractors in 78 Levins, R. (1969) Some demographic and genetic consequences of
epidemics. Proc. R. Soc. Lond. B 270, 1573–1578 environmental heterogeneity for biological control. Bull. Entomol.
56 Keeling, M.J. and Rohani, P. (2007) Modelling Infectious Diseases in Soc. Am. 15, 227–240
Humans and Animals, Princeton University Press 79 Grimm, V. et al. (2005) Pattern-oriented modeling of agent-based
57 Metcalf, C.J.E. et al. (2011) The epidemiology of rubella in Mexico: complex systems: lessons from ecology. Science 310, 987–991
seasonality, stochasticity and regional variation. Epidemiol. Infect. 80 Ferguson, N.M. et al. (2005) Strategies for containing an emerging
139, 1029–1038 influenza pandemic in Southeast Asia. Nature 437, 209–214
58 Altizer, S. et al. (2006) Seasonality and the dynamics of infectious 81 Hubbell, S.P. (2001) The Unified Neutral Theory of Biodiversity and
diseases. Ecol. Lett. 9, 467–484 Biogeography, Princeton University Press
59 Mantilla-Beniers, N.B. et al. (2010) Decreasing stochasticity through 82 Moran, P.A.P. (1962) The Statistical Processes of Evolutionary Theory,
enhanced seasonality in measles epidemics. J. R. Soc. Interface 7, Oxford University Press
727–739 83 Alonso, D. et al. (2006) The merits of neutral theory. Trends Ecol. Evol.
60 Earn, D. et al. (2000) A simple model for complex dynamical transitions 21, 451–457
in epidemics. Science 287, 667–670 84 Rosindell, J. et al. (2011) The unified neutral theory of biodiversity and
61 Bauch, C.T. and Earn, D.J.D. (2003) Interepidemic intervals in forced biogeography at age ten. Trends Ecol. Evol. 26, 340–348
and unforced SEIR models. Fields Inst. Comm. 36, 33–44 85 Warren, P.B. (2010) Biodiversity on island chains: neutral model
62 Coulson, T. et al. (2004) Skeletons, noise and population growth: the simulations. Phys. Rev. E 82, 051922
end of an old debate? Trends Ecol. Evol. 19, 359–364 86 Economo, E.P. and Keitt, T.H. (2008) Species diversity in neutral
63 Nguyen, H.T.H. and Rohani, P. (2008) Noise, nonlinearity and metacommunities: a network approach. Ecol. Lett. 11, 52–62
seasonality: the epidemics of whooping cough revisited. J. R. Soc. 87 Hanski, I. (1999) Metapopulation Ecology, Oxford University Press
Interface 5, 403–413 88 Ross, J.V. et al. (2008) Metapopulation persistence in a dynamic
64 Black, A.J. and McKane, A.J. (2010) Stochasticity in staged models of landscape: more habitat or better stewardship? Ecol. Appl. 18,
epidemics: quantifying the dynamics of whooping cough. J. R. Soc. 590–598
Interface 7, 1219–1227 89 Levin, S.A. and Segel, L.A. (1976) Hypothesis for origin of planktonic
65 Rohani, P. et al. (1999) Opposite patterns of synchrony in sympatric patchiness. Nature 259, 659
disease metapopulations. Science 286, 968–971 90 Waddell, J.N. et al. (2010) Demographic stochasticity versus spatial
66 Traulsen, A. et al. (2005) Coevolutionary dynamics: from finite to variation in the competition between fast and slow dispersers. Theor.
infinite populations. Phys. Rev. Lett. 95, 238701 Popul. Biol. 77, 279–286

345