SPRINGER BRIEFS IN APPLIED SCIENCES AND

TECHNOLOGY  MATHEMATICAL METHODS

Ryszard Rudnicki
Marta Tyran-Kamińska

Piecewise
Deterministic
Processes in
Biological Models

123
SpringerBriefs in Applied Sciences
and Technology

Mathematical Methods

Series editor
Anna Marciniak-Czochra, Heidelberg, Germany
More information about this series at http://www.springer.com/series/11219
Ryszard Rudnicki Marta Tyran-Kamińska
•

Piecewise Deterministic
Processes in Biological
Models

123
Ryszard Rudnicki Marta Tyran-Kamińska
Institute of Mathematics Institute of Mathematics
Polish Academy of Sciences University of Silesia
Katowice Katowice
Poland Poland

ISSN 2191-530X ISSN 2191-5318 (electronic)

SpringerBriefs in Applied Sciences and Technology
SpringerBriefs in Mathematical Methods
ISBN 978-3-319-61293-5 ISBN 978-3-319-61295-9 (eBook)
DOI 10.1007/978-3-319-61295-9
Library of Congress Control Number: 2017945225

© The Author(s) 2017

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part
of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission
or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar
methodology now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publication does not imply, even in the absence of a specific statement, that such names are exempt from
the relevant protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authors or the editors give a warranty, express or implied, with respect to the material contained herein or
for any errors or omissions that may have been made. The publisher remains neutral with regard to
jurisdictional claims in published maps and institutional affiliations.

Printed on acid-free paper

This Springer imprint is published by Springer Nature

The registered company is Springer International Publishing AG
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
Preface

The aim of this book is to give a short mathematical introduction to piecewise

deterministic Markov processes (PDMPs) and to present biological models where
they appear. The book is divided into six chapters. In the first chapter we present
some examples of biological phenomena such as gene activity and population
growth leading to different type of PDMPs: continuous-time Markov chains,
deterministic processes with jumps, dynamical systems with random switching and
point processes. The second chapter contains some theoretical results concerning
Markov processes and the construction of PDMPs. The next chapter is an intro-
duction to the theory of semigroups of linear operators which provide the primary
tools in the study of continuous-time Markov processes. In the fourth chapter we
introduce stochastic semigroups, provide some theorems on their existence and
find generators of semigroups related to PDMPs considered in the first chapter. The
next chapter is devoted to the long-time behaviour (asymptotic stability and
sweeping) of the stochastic semigroups induced by PDMPs. In the last chapter we
apply the general results, especially concerning asymptotic behaviour, to biological
models.
The book is dedicated to both mathematicians and biologists. The first group will
find here new biological models which lead to interesting and often new mathe-
matical questions. Biologists can observe how to include seemingly different bio-
logical processes into a unified mathematical theory and deduce from this theory
interesting biological conclusions. We try to keep the required mathematical
and biological background to a minimum so that the topics are accessible to
students.

v
vi Preface

Acknowledgements

This research was partially supported by the National Science Centre (Poland)
Grant No. 2014/13/B/ST1/00224. The authors are grateful to Michael C. Mackey
and Katarzyna Pichór who read the manuscript and made useful suggestions for
improvements.

Katowice, Poland Ryszard Rudnicki

May 2017 Marta Tyran-Kamińska
Contents

1 Biological Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Birth-Death Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Grasshopper and Kangaroo Movement . . . . . . . . . . . . . . . . . . . . . 3
1.4 Velocity Jump Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.5 Size of Cells in a Single Line . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.6 Two-Phase Cell Cycle Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.7 Stochastic Billiard as a Cell Cycle Model . . . . . . . . . . . . . . . . . . 9
1.8 Stochastic Gene Expression I . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.9 Stochastic Gene Expression II . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.10 Gene Regulatory Models with Bursting . . . . . . . . . . . . . . . . . . . . 16
1.11 Neural Activity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
1.12 Processes with Extra Jumps on a Subspace . . . . . . . . . . . . . . . . . 19
1.13 Size-Structured Population Model . . . . . . . . . . . . . . . . . . . . . . . . . 21
1.14 Age-Structured Population Model . . . . . . . . . . . . . . . . . . . . . . . . . 24
1.15 Asexual Phenotype Population Model . . . . . . . . . . . . . . . . . . . . . 25
1.16 Phenotype Model with a Sexual Reproduction . . . . . . . . . . . . . . . 26
1.17 Coagulation-Fragmentation Process in a Phytoplankton
Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .... 28
1.18 Paralog Families . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .... 29
1.19 Definition of PDMP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .... 30
2 Markov Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.1 Transition Probabilities and Kernels . . . . . . . . . . . . . . . . . . . . . . . 33
2.1.1 Basic Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.1.2 Transition Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.1.3 Substochastic and Stochastic Operators . . . . . . . . . . . . . . . 38
2.1.4 Integral Stochastic Operators . . . . . . . . . . . . . . . . . . . . . . . 39
2.1.5 Frobenius–Perron Operator . . . . . . . . . . . . . . . . . . . . . . . . 41
2.1.6 Iterated Function Systems . . . . . . . . . . . . . . . . . . . . . . . . . 42

vii
viii Contents

2.2 Discrete-Time Markov Processes . . . . . . . . . . . . . . . . . . . . . . . . . 44

2.2.1 Markov Processes and Transition Probabilities . . . . . . . . . 44
2.2.2 Random Mapping Representations . . . . . . . . . . . . . . . . . . 46
2.2.3 Canonical Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
2.3 Continuous-Time Markov Processes . . . . . . . . . . . . . . . . . . . . . . . 49
2.3.1 Basic Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
2.3.2 Processes with Stationary and Independent
Increments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
2.3.3 Markov Jump-Type Processes . . . . . . . . . . . . . . . . . . . . . . 52
2.3.4 Generators and Martingales . . . . . . . . . . . . . . . . . . . . . . . . 54
2.3.5 Existence of PDMPs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
2.3.6 Transition Functions and Generators of PDMPs . . . . . . . . 59
3 Operator Semigroups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
3.1 Generators and Semigroups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
3.1.1 Essentials of Banach Spaces and Operators . . . . . . . . . . . . 63
3.1.2 Definitions and Basic Properties . . . . . . . . . . . . . . . . . . . . 65
3.1.3 The Resolvent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
3.2 Basic Examples of Semigroups . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
3.2.1 Uniformly Continuous Semigroups . . . . . . . . . . . . . . . . . . 69
3.2.2 Multiplication Semigroups . . . . . . . . . . . . . . . . . . . . . . . . 70
3.2.3 Translation Semigroups . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
3.3 Generators of Contraction Semigroups . . . . . . . . . . . . . . . . . . . . . 73
3.3.1 The Hille–Yosida Theorem . . . . . . . . . . . . . . . . . . . . . . . . 73
3.3.2 The Lumer–Phillips Theorem . . . . . . . . . . . . . . . . . . . . . . 76
3.3.3 Perturbations of Semigroups . . . . . . . . . . . . . . . . . . . . . . . 78
3.3.4 Perturbing Boundary Conditions . . . . . . . . . . . . . . . . . . . . 80
4 Stochastic Semigroups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
4.1 Aspects of Positivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
4.1.1 Positive Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
4.1.2 Substochastic Semigroups . . . . . . . . . . . . . . . . . . . . . . . . . 84
4.1.3 Resolvent Positive Operators . . . . . . . . . . . . . . . . . . . . . . . 86
4.1.4 Generation Theorems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
4.1.5 Positive Perturbations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
4.1.6 Positive Unbounded Perturbations . . . . . . . . . . . . . . . . . . . 91
4.1.7 Adjoint and Transition Semigroups . . . . . . . . . . . . . . . . . . 94
4.2 Stochastic Semigroups for PDMPs . . . . . . . . . . . . . . . . . . . . . . . . 96
4.2.1 Jump-Type Markov Processes . . . . . . . . . . . . . . . . . . . . . . 96
4.2.2 Semigroups for Semiflows . . . . . . . . . . . . . . . . . . . . . . . . 99
4.2.3 PDMPs Without Boundaries . . . . . . . . . . . . . . . . . . . . . . . 101
4.2.4 Dynamical Systems with Jumps . . . . . . . . . . . . . . . . . . . . 103
4.2.5 Randomly Switched Dynamical Systems. . . . . . . . . . . . . . 104
Contents ix

4.2.6 Jumps from Boundaries . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

4.2.7 Semigroups for the Stein Model . . . . . . . . . . . . . . . . . . . . 109
5 Asymptotic Properties of Stochastic Semigroups—General
Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
5.1 Asymptotic Stability and Sweeping . . . . . . . . . . . . . . . . . . . . . . . 115
5.1.1 Definitions of Asymptotic Stability and Sweeping . . . . . . 115
5.1.2 Lower Function Theorem . . . . . . . . . . . . . . . . . . . . . . . . . 116
5.1.3 Partially Integral Semigroups and Asymptotic
Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .... 117
5.1.4 Sweeping via the Existence of r-finite Invariant
Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
5.2 Asymptotic Decomposition of Stochastic Semigroups . . . . . . . . . 120
5.2.1 Theorem on Asymptotic Decomposition . . . . . . . . . . . . . . 120
5.2.2 Sweeping and the Foguel Alternative . . . . . . . . . . . . . . . . 122
5.2.3 Asymptotic Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
5.2.4 Hasminskiĭ Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
6 Asymptotic Properties of Stochastic Semigroups—Applications .... 129
6.1 Continuous-Time Markov Chains . . . . . . . . . . . . . . . . . . . . . .... 129
6.1.1 Foguel Alternative . . . . . . . . . . . . . . . . . . . . . . . . . . . .... 129
6.1.2 Non-explosive Markov Chains and Asymptotic
Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
6.1.3 Markov Chain with an Absorbing State . . . . . . . . . . . . . . 131
6.1.4 Asymptotics of Paralog Families . . . . . . . . . . . . . . . . . . . . 132
6.1.5 Applications to Pure Jump-Type Markov Processes . . . . . 133
6.2 Dynamical Systems with Random Switching . . . . . . . . . . . . . . . . 135
6.2.1 General Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
6.2.2 Applications to Stochastic Gene Expression Models . . . . . 137
6.3 Cell Maturation Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
6.3.1 Introductory Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
6.3.2 Flows with Jumps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
6.3.3 Size-Structured Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
6.3.4 Two-Phase Cell Cycle Model . . . . . . . . . . . . . . . . . . . . . . 144
6.3.5 Lebowitz–Rubinow’s Model . . . . . . . . . . . . . . . . . . . . . . . 146
6.3.6 Stein’s Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
Appendix A: Measure and Probability Essentials . . . . . . . . . . . . . . . . . . . 149
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
Chapter 1
Biological Models

1.1 Introduction

Applications of probability theory to study biological and medical questions have a

long history. Likely, probabilistic methods were used for the first time in the biological
and medical sciences in 1766 by Daniel Bernoulli to demonstrate the efficacy of
vaccination against smallpox [14]. From that time on, many problems in biology
were described using probabilistic tools. Examples of applications of birth-death and
branching processes in population dynamics and Bayesian methods in Mendelian
genetics are well known [37]. Modern biology uses more advanced methods of
probability theory to describe and study environmental and molecular processes.
A special role in such applications is played by piecewise deterministic Markov
processes (PDMPs).
According to a non-rigorous definition by Davis [28], the class of PDMPs is a
general family of stochastic models covering virtually all non-diffusion applications.
A formal definition of PDMP will be given at the end of this chapter. For now it is
enough to know that PDMP is a continuous-time Markov process ξ(t) with values
in some metric space X and there is an increasing sequence of random times (tn ),
called jump times, such that sample paths (trajectories) of ξ(t) are defined in a
deterministic way in each interval (tn , tn+1 ). We consider two types of behaviour
of the process at jump times: the process can jump to a new point or can change
the dynamics which defines its trajectories. PDMPs is a large family of different
stochastic processes which includes discrete-time Markov processes, continuous-
time Markov chains, deterministic processes with jumps, dynamical systems with
random switching, stochastic billiards and some point processes.
In this chapter, we present simple biological models to illustrate possible appli-
cations of such processes. Some of these models were presented in the survey paper
[101], which can serve as a short introduction to this monograph. Although discrete-
time Markov processes play important role in applications we will not investigate
them in this book because their theory differs from that of continuous-time PDMPs
and their applications are sufficiently known [1].
© The Author(s) 2017 1
R. Rudnicki and M. Tyran-Kamińska, Piecewise Deterministic Processes
in Biological Models, SpringerBriefs in Mathematical Methods,
DOI 10.1007/978-3-319-61295-9_1
2 1 Biological Models

1.2 Birth-Death Processes

We precede the definition of a birth-death process by a simple example concerning

a cellular population. Consider a homogeneous population of cells and assume that
during the time interval of length Δt a single cell can replicate with probability
bΔt + o(Δt) and it can die with probability dΔt + o(Δt). We also assume that cells
replicate and die independently of each other. Denote by ξ(t) the number of cells at
time t. Then for sufficiently small Δt the process ξ(t) satisfies
⎧
⎪
⎪ biΔt + o(Δt), if j = i + 1,
⎪
⎨diΔt + o(Δt), if j = i − 1,
Prob(ξ(t + Δt) = j|ξ(t) = i) =
⎪
⎪ 1 − (b + d)iΔt + o(Δt), if j = i,
⎪
⎩
o(Δt), if | j − i| > 1.
(1.1)
It is assumed that limΔt→0 o(Δt)
Δt
= 0. The process ξ(t) is a continuous-time Markov
chain with values in the space N = {0, 1, 2, . . .}. The process ξ(t) is time homoge-
neous because the conditional probabilities (1.1) do not depend on t. The functions

p ji (Δt) = Prob(ξ(t + Δt) = j|ξ(t) = i)

are called transition probabilities and the limit

p ji (Δt)
q ji = lim , i = j,
Δt→0 Δt

is called the transition rate from i to j. The process ξ(t) has transition rates qi+1,i =
bi, qi−1,i = di, and q ji = 0 otherwise. We also define

∞

qii = − q ji . (1.2)
j=0, j=i

The infinite matrix Q = [q ji ] is called the transition rate matrix or the infinitesi-
mal generator matrix. The matrix Q plays a crucial role in the theory of Markov
processes. In particular, if pi (t) = Prob(ξ(t) = i) for i ∈ N and t ≥ 0, then the col-
umn vector p(t) = [ p0 (t), p1 (t), . . .]T satisfies the following infinite dimensional
system of differential equations

p  (t) = Qp(t).

The process ξ(t) is one of many examples of birth-death processes. A general

birth-death process is a continuous-time Markov chain defined by (1.1) if we replace
bi with bi and di with di , where (bi ) and (di ) are sequences of non-negative num-
bers (see Fig. 1.1). The coefficients bi and di are called birth and death rates for a
1.2 Birth-Death Processes 3

d1 d2 dn

0 b0 1 b1 2 n–1 bn−1 n

Fig. 1.1 The diagram of the transition between states in the birth-death process

population with i individuals. The problem of the existence of such a process is non-
trivial and we shall consider it in detail in the next chapter. It should be noted that for
given rates we have a family of different birth-death processes and their distributions
depend on the initial distributions (i.e. the distribution of the random variables ξ(0)).
Two special types of birth-death processes are pure birth (if di = 0 for all i) and
pure death (if di = 0 for all i) processes. A Poisson process N (t) with intensity λ is
a pure birth process with bi = λ for all i.
A similar construction based on the transition rate matrix can be applied to define
general continuous-time Markov chains. Let Q = [q ji ] be a transition rate matrix,
i.e. a matrix which has non-negative entries outside the main diagonal and satisfies
(1.2). The matrix Q is a transition rate matrix for a homogeneous continuous-time
Markov chain ξ(t) if the following condition is fulfilled

Prob(ξ(t + Δt) = j|ξ(t) = i) = q ji Δt + o(Δt).

Many applications of continuous-time Markov chains in population dynamics, epi-

demiology and genetics are given in Chap. VII of [1].

1.3 Grasshopper and Kangaroo Movement

Continuous-time Markov chains belong to a slightly larger class called pure jump-
type Markov processes. A pure jump-type Markov process is a Markov process,
which remains constant between jumps. For example, the process used in a simple
description of the grasshopper and kangaroo movement [81] is an example of a pure
jump-type Markov process, which is not a Markov chain. A grasshopper jumps at
random times (tn ) from a point x to the point x + ϑn . We assume that jump times
are the same as for a Poisson process N (t) with intensity λ > 0, i.e. N (tn ) = n,
and that (ϑn ) is a sequence of independent and identically distributed (i.i.d.) random
vectors, which are also independent of {N (t) : t ≥ 0}. Then the position ξ(t) of the
grasshopper is given by
N (t)

ξ(t) = ξ(0) + ϑn . (1.3)
n=1

The process given by (1.3) is called a compound Poisson process.

4 1 Biological Models

A general pure jump-type homogeneous Markov process on a measurable space

(X, Σ) can be defined in the following way. Let ϕ : X → [0, ∞) be a given measur-
able function and let P(x, B) be a given transition probability on X , i.e. P(x, ·) is a
probability measure for each x ∈ X and the function x → P(x, B) is measurable for
each B ∈ Σ. Let t0 = 0 and let ξ(0) = ξ0 be an X -valued random variable. For each
n ≥ 1 one can choose the nth jump time tn as a positive random variable satisfying

Prob(tn − tn−1 ≤ t|ξn−1 = x) = 1 − e−ϕ(x)t , t ≥ 0,

and define 
ξn−1 for tn−1 ≤ t < tn ,
ξ(t) =
ξn for t = tn ,

where the nth post-jump position ξn is an X -valued random variable such that

Prob(ξn ∈ B|ξn−1 = x) = P(x, B).

1.4 Velocity Jump Process

Another type of cellular or organism dispersal considered in biological literature,

e.g. in [81], is a velocity jump process. An individual is moving in the space Rd with
a constant velocity and at jump times (tn ) it chooses a new velocity. As in Sect. 1.3,
we assume that jump times are the same as for a Poisson process N (t) with intensity
λ. It means that F(t) = 1 − e−λt is the distribution function of tn − tn−1 . Let x(t)
be the position and v(t) be the velocity of an individual at time t. We assume that
for every x, v ∈ Rd , there is a probability Borel measure P(x, v, B) on Rd which
describes the change of the velocity after a jump, i.e.

Prob(v(tn ) ∈ B |x(tn− ) = x, v(tn− ) = v) = P(x, v, B)

for every Borel subset B of Rd , where x(tn− ) and v(tn− ) are the left-hand side limits of
x(t) and v(t), respectively, at the point tn . Then ξ(t) = (x(t), v(t)), t ≥ 0, is a PDMP
corresponding to our movement and between jumps the pair (x(t), v(t)) satisfies the
following system of ordinary differential equations

x  (t) = v(t),
(1.4)
v (t) = 0.

There are a number of interesting examples of velocity jump processes. The

simplest one is symmetric movement on the real line R. In this case we assume
that an individual is moving with constant speed, say one, and at a jump time it
changes the direction of movement to the opposite one. A PDMP corresponding to the
1.4 Velocity Jump Process 5

symmetric movement has values in the space R × {−1, 1} and P(x, v, {−v}) = 1 for
v = −1, 1. More advanced examples of velocity jump processes and their comparison
with dispersal of cells, insects and mammals are given in [81, 109]. We can consider
velocity jump processes defined in a bounded domain G. Examples of such processes
are stochastic billiards [36]. Stochastic billiards do not change velocity in the interior
of G but when an individual or a point strikes the boundary, a new direction is
chosen randomly from the directions that point back into the interior of G and
the motion continues. Stochastic billiards can describe not only the movement of
individuals in a bounded domain but also various different biological processes. In
Sect. 1.7, we present Lebowitz-Rubinow cell cycle model which can be identified
with a one-dimensional stochastic billiard. Some examples of PDMPs with jumps
on the boundary that appear in the theory of the gene regulatory systems are presented
in Sect. 1.12.
There have been a number of papers devoted to velocity jump processes, their dif-
fusion approximations and applications to aggregation and chemotaxis phenomena.
The interested reader is referred to [50, 75, 82] and the references therein.

1.5 Size of Cells in a Single Line

We consider a sequence of consecutive descendants of a single cell. Let tn be a time

when a cell from the n-generation splits and let ξ(t), t ∈ [tn−1 , tn ), be the size (mass,
volume) of the cell from the n-generation at time t. We assume that g(x) is the growth
rate of a cell with size x, i.e. the process ξ(t) satisfies the differential equation

ξ  (t) = g(ξ(t)) for t ∈ (tn−1 , tn ), n ≥ 1. (1.5)

Denote by ϕ(x) the division rate of a cell with size x, i.e. a cell with size x replicates
during a small time interval of length Δt with probability ϕ(x)Δt + o(Δt). Finally,
we assume that a daughter cell has a half size of the mother cell. It means that

ξ(tn ) = 21 ξ(tn− ), (1.6)

where ξ(tn− ) is the left-hand side limit of ξ(t) at the point tn . In this way, we define
a homogeneous PDMP which is not a pure jump-type Markov process.
Given growth and division rate functions and the initial size x0 of a cell it is not
difficult to find the distribution of its life-span and the distribution of its size at the
point of division. Let π(t, x0 ) be the size of a cell at age t if its initial size is x0 , i.e.
π(t, x0 ) = x(t), where x is the solution of the equation x  = g(x) with the initial
condition x(0) = x0 . The life-span of a cell is a random variable T which depends
on the initial size x0 of a cell. Let Φx0 (t) = Prob(T > t) be the survival function, i.e.
the probability that the life-span of a cell with initial size x0 is greater than t. Then
6 1 Biological Models

Φx0 (t) − Φx0 (t + Δt)

Prob(t < T ≤ t + Δt | T > t) = = ϕ(π(t, x0 ))Δt + o(Δt).
Φx0 (t)

From this equation, we obtain

Φx 0 (t) = −Φx0 (t)ϕ(π(t, x0 ))

and after simple calculations we get

t
Φx0 (t) = exp − ϕ(π(s, x0 )) ds (1.7)
0

and F(t) = 1 − Φx0 (t) is the distribution function of the life-span. For y ≥ x0 , we
define t (x0 , y) to be the time t such that π(t, x0 ) = y. Since

∂t
· g(π(t, x0 )) = 1,
∂y

we see that
∂t 1
=
∂y g(y)

and
t (x0 ,y)
∂ ϕ(y)
ϕ(π(s, x0 )) ds = .
∂y 0 g(y)

Let ϑ be the size of the cell at the moment of division. Then

t (x0 ,y)
Prob(ϑ > y) = Prob(π(t, x0 ) > y) = exp − ϕ(π(s, x0 )) ds
0
(1.8)
y
ϕ(r )
= exp − dr = exp(Q(x0 ) − Q(y)),
x0 g(r )

x ϕ(r )
where Q(x) = dr. Let ζ be a random variable with exponential distribution
0 g(r )
i.e. Prob(ζ > x) = e−x for x ≥ 0. Then
 
Prob(ϑ > y) = exp(Q(x0 ) − Q(y)) = Prob ζ > Q(y) − Q(x0 )
 
= Prob Q −1 Q(x0 ) + ζ > y ,
 
which means that the random variables ϑ and Q −1 Q(x0 ) + ζ have the same dis-
tribution. From this it follows that
 
ξ(tn ) = 21 Q −1 Q(ξ(tn−1 )) + ζ ,
d
(1.9)
1.5 Size of Cells in a Single Line 7

Fig. 1.2 A flow with jumps

(t2− )
(t3− )

(t4 )
(t1 )

(t1− ) (t2 ) (t4− )

(t3 )

(t0 )

where ζ is a random variable independent of ξ(tn−1 ) and with exponential distribu-

tion.
One can consider a more general model when a cell is characterized by a multidi-
mensional parameter x ∈ x ⊂ Rd which changes according to a system of differential
equations
x  = g(x),

ϕ(x) is the division rate of a cell with parameter x, and P(x, B) is a transition
probability function on X which describes the change of parameters from a mother
to a daughter cell, i.e.

Prob(ξ(tn ) ∈ B|ξ(tn− ) = x) = P(x, B).

The distribution function of tn − tn−1 is given by F(t) = 1 − Φx0 (t), where Φx0 (t)
is as in (1.7) with x0 := ξ(tn−1 ). PDMPs of this type are called flows with jumps (see
Fig. 1.2).

1.6 Two-Phase Cell Cycle Model

The cell cycle is a series of events that take place in a cell leading to its replication.
Usually the cell cycle is divided into four phases. The first one is the growth phase G 1
with synthesis of various enzymes. The duration of the phase G 1 is highly variable
even for cells from one species. The DNA synthesis takes place in the second phase
S. In the third phase G 2 significant protein synthesis occurs, which is required during
8 1 Biological Models

the process of mitosis. The last phase M consists of nuclear division and cytoplasmic
division. From a mathematical point of view, we can simplify the model by consid-
ering only two phases: A = G 1 with a random duration t A and B which consists of
the phases S, G 2 and M. The duration t B of the phase B is almost constant. There are
several models of the cell cycle. Let us mention the models by Lasota and Mackey
[61], Tyson and Hannsgen [112], and the generalization of these model given by
Tyrcha [115]. Here, we describe a continuous-time version of the Tyrcha model and
we show that it can be treated as a PDMP.
The crucial role in the model is played by a parameter x which describes the
state of a cell in the cell cycle. It is not clear what x exactly should be. We simply
assume that x is the cell size. We start with a model similar to that from Sect. 1.5
with only one difference. Here ϕ(x) is the rate of entering the phase B, i.e. a cell with
size x enters the phase B during a small time interval of length Δt with probability
ϕ(x)Δt + o(Δt). It is clear that the process ξ(t) is piecewise deterministic but it
is non-Markovian, because its future ξ(t), t ≥ t0 , depends not only on the random
variable ξ(t0 ) but also depends in which phase it is at the time t0 .
Now we extend the process ξ(t), t ≥ 0, to obtain a homogeneous PDMP. A new
process  ξ (t), t ≥ 0, is defined on the phase space [0, ∞) × [0, t B ] × {1, 2} in the
following way. First, we add additional jump points sn = tn − t B , n ≥ 1. At time sn
a cell from the n-generation enters the phase B (see Fig. 1.3). Let


ξ (t) = (
ξ1 (t), 
ξ2 (t), 
ξ3 (t)) = (ξ(t), y(t), i),

where i = 1 if at time t a cell is in the phase A and i = 2 if it is in the phase B,

y(t) = 0 if the cell is in the phase A and y(t) = t − sn if the cell from the n-generation
is in the phase B. Between jump points the coordinates of the process  ξ (t) satisfy
the following system of ordinary differential equations

Fig. 1.3 The schematic tn

diagram of the transition in
the two-phase cell cycle tn+2
model tB

sn tB
tn+1
sn+2
tB

tA
sn+1
tA

tA

tn−1 tn tn+1
1.6 Two-Phase Cell Cycle Model 9
⎧ 
⎪ 
ξ1 (t) = g(ξ1 (t)),
⎪
⎪ 
⎨
 0, if ξ3 (t) = 1,
ξ2 (t) =  (1.10)
⎪
⎪ 1, if ξ 3 (t) = 2,
⎪
⎩ 
ξ3 (t) = 0.

The transitions at the jump points are given by


ξ1 (sn ) = 
ξ1 (sn− ), 
ξ2 (sn ) = 
ξ2 (sn− ) = 0, 
ξ3 (sn ) = 2,

and

ξ1 (tn ) = 21
ξ1 (tn− ), 
ξ2 (tn ) = 0, 
ξ3 (tn ) = 1.

The distribution function of sn − tn−1 is given by

t
F(t) = 1 − exp − ϕ(π(s, x0 )) ds ,
0

where x0 =  ξ1 (tn−1 ) and π(t, x0 ) = x(t) is the solution of equation x  = g(x) with
initial condition x(0) = x0 , while that of tn − sn by F(t) = 0 for t ≤ t B and F(t) = 1
for t > t B .
The life-span tn − tn−1 of a cell with initial size x0 has the distribution

0, if t < t B ,
F(t) =  t−t B (1.11)
1 − exp − 0 ϕ(π(s, x0 )) ds , if t ≥ t B ,

and the relation between the distributions of the random variables ξ(tn ) and ξ(tn−1 ):
  
ξ(tn ) = 21 π t B , Q −1 Q(ξ(tn−1 )) + ζ ,
d
(1.12)

where ζ is a random variable independent of ξ(tn−1 ) with exponential distribution.

1.7 Stochastic Billiard as a Cell Cycle Model

One of the oldest models of cell cycle introduced by Rubinow [94] is based on the
concept of maturity and maturation velocity. Maturity, also called physiological age,
is a real variable x from the interval [0, 1] which describes the position of a cell in the
cell cycle. A new born cell has maturity 0 and a cell splits at maturity 1. In Rubinow’s
model x grows according to the equation x  = v, where the maturation velocity v
can depend on x and also on other factors such as time, the size of the population,
temperature, light, environmental nutrients, pH, etc. If we neglect environmental
factors, resource limitations, and stochastic variation, and we assume that v depends
10 1 Biological Models

only on x, then all cells will have identical cell cycle in this model, in particular they
have the same cell cycle length l.
However, experimental observations concerning cell populations, cultured under
identical conditions for each member, revealed high variability of l in the popula-
tion. It means that the population is heterogeneous with respect to cell maturation
velocities, and therefore, mathematical models of the cell cycle should take into
account maturation velocities. A model of this type was proposed by Lebowitz and
Rubinow [64]. In their model the cell cycle is determined by its length l, which is
fixed at the birth of the cell. The relation between cycle lengths of mother’s and
daughter’s cells is given by a transition probability. If the maturation velocity of each
individual cell is constant, we can use v instead of l in the description of the model
because lv = 1. Such a model is a special case of the Rotenberg model [93] and we
briefly present it here using PDMP formalism. Each cell matures with individual
velocity v, which is a positive constant. We assume that the relation between the
maturation velocities of mother’s and daughter’s cells is given by a transition proba-
bility P(v, dv ). As in Sect. 1.5, we consider a sequence of consecutive descendants
of a single cell. Let tn be a time when a cell from the n-generation splits and let
ξ(t) = (x(t), v(t)), t ∈ [tn−1 , tn ), be the maturity and the maturation velocity of the
cell from the n-generation at time t. It is clear that ξ(t), t ≥ 0, is a PDMP and between
jumps the pair (x(t), v(t)) satisfies system (1.4). We assume that the process ξ(t),
t ≥ 0, has càdlàg sample paths, i.e. they are right-continuous with left limits. We
have x(tn ) = 0 and Prob(v(tn ) ∈ B |v(tn− ) = v) = P(v, B) for each n ∈ N and each
Borel subset B of (0, ∞). Since the v(t) is constant in the interval (tn−1 , tn ), we have
tn − tn−1 = 1/v(tn−1 ).
Properties of the density of the population with respect to (x, v) in the above model
were studied in [19]. But this model can be also identified with a one-dimensional
stochastic billiard. Namely, consider a particle moving in the interval [0, 1] with a
constant velocity. We assume that when the particle hits the boundary points 0 and 1,
it changes its velocity according to the probability measures P0 (−v, B) = P(v, B)
and P1 (v, −B) = P(v, B), respectively, where v > 0 and B is a Borel subset of
(0, ∞). Observe that the PDMP defined in the Lebowitz–Rubinow model is given
by ξ(t) = (x(t), |v(t)|), where x(t) and v(t) represent position and velocity of the
moving particle at time t. Asymptotic properties of the general one-dimensional
stochastic billiard were studied in [78].
It is not difficult to build a model which connects the classical Rubinow model
with Lebowitz–Rubinow’s one. We still assume that the cell maturity changes from
0 to 1 with velocity v and v depends on x and some specific genetic factor y (say
phenotypic trait) inherited from the mother cell which does not change during the
life of the cell. The trait y  of the daughter cell is drawn from a distribution P(y, dy  ),
where y is the mother cell trait. Similarly to the previous model, let tn be a time when
a cell from the n-generation splits and let ξ(t) = (x(t), y(t)), t ∈ [tn−1 , tn ), be the
maturity and the trait of the cell from the n-generation at time t. Then between jumps
the pair (x(t), y(t)) satisfies the following system:
1.7 Stochastic Billiard as a Cell Cycle Model 11

x  (t) = v(x(t), y(t)),
(1.13)
y  (t) = 0.

We have also x(tn ) = 0 and Prob(y(tn ) ∈ B |y(tn− ) = y) = P(y, B). Then the length
of the interval T = tn − tn−1 depends on the trait y of the cell and T is a positive
number such that x(T ) = 1, where x(t) is the solution of the initial problem x  (t) =
v(x(t), y), x(0) = 0.
As we mentioned above, the Lebowitz–Rubinow model is a special case of the
Rotenberg model [93]. Now we briefly present this model using PDMP formalism.
In the Rotenberg model, the maturation velocity is random and can change also
during the cell cycle. A new born cell inherits the initial maturation velocity from
its mother as in the Lebowitz–Rubinow model according to a transition probability
P(v, dv ). During the cell cycle it can change its maturation velocity with intensity
q(x, v), i.e. a cell with parameters (x, v) can change the maturation velocity in a
small time interval of length Δt with probability q(x, v)Δt + o(Δt). Let x0 be the
maturity of a cell when it receives the maturation velocity v. From (1.7) it follows
that the probability distribution function of the length of the time interval when the
maturation velocity remains v is given by
t
1 − exp − q(x0 + vs, v) ds .
0

We suppose that if (x, v) is the state of the cell at the moment of division, then
a new maturation velocity is drawn from a distribution P(x, v, dv ). The process
ξ(t) = (x(t), v(t)) describing consecutive descendants of a single cell is a PDMP
which has jumps when cells split and random jumps during their cell cycles. Between
jumps the pair (x(t), v(t)) satisfies system (1.4). If a jump is at the moment of division
then it is given by the same formula as in the Lebowitz–Rubinow model. If a jump
is during the cell cycle, then x(tn ) = x(tn− ) and

Prob(v(tn ) ∈ B |x(tn− ) = x, v(tn− ) = v) = P(x, v, B) for each Borel subset B of (0, ∞).

1.8 Stochastic Gene Expression I

Gene expression is a complex process which involves three processes: gene activa-
tion/inactivation, mRNA transcription/decay and protein translation/decay. Now we
consider a very simple model when proteins production is regulated by a single gene
and we omit the intermediate process of mRNA transcription (see Fig. 1.4). A gene
can be in an active or an inactive state and it can be transformed into an active state
or into an inactive state, with intensities q0 and q1 , respectively. The rates q0 and q1
depend on the number of protein molecules ξ(t). If the gene is active then proteins
12 1 Biological Models

Auto-regulation

q0
Inactive Active P
gene gene Protein

q1

degradation

Fig. 1.4 The diagram of gene expression I

are produced with a constant speed P. In both states of the gene, protein molecules
undergo the process of degradation with rate μ. It means that the process ξ(t), t ≥ 0,
satisfies the equation
ξ  (t) = P A(t) − μξ(t), (1.14)

where A(t) = 1 if the gene is active and A(t) = 0 in the opposite case. Then the
process ξ (t) = (ξ(t), A(t)), t ≥ 0, is a PDMP. Since the right-hand side
 of Eq.
 (1.14)
is negative for ξ(t) > μP we can restrict values of ξ(t) to the interval 0, μP and the
 
process ξ (t) is defined on the phase space 0, μP × {0, 1}.
The process  ξ (t) has jump points when the gene changes its activity. Formula
(1.7) allow us to find the distribution of time between consecutive jumps. Observe
that if x0 is the number of protein molecules at a jump point, then after time t we
have
P P −μt
πt0 (x0 ) = x0 e−μt , πt1 (x0 ) = + x0 − e
μ μ

protein molecules, respectively, in an inactive and an active state. From (1.7), it

follows that the probability distribution function of the length of an inactive state is
given by
t
1 − exp − q0 x0 e−μs ds
0

and of an active state by

t
P P −μs
1 − exp − q1 + x0 − e ds .
0 μ μ
1.9 Stochastic Gene Expression II 13

1.9 Stochastic Gene Expression II

A more advanced (and more realistic) model of gene expression was introduced by
Lipniacki et al. [65] and studied in [17]. In this model, mRNA transcription and
decay processes are also taken into consideration. The rates q0 and q1 depend on the
number of mRNA molecules ξ1 (t) and on the number of protein molecules ξ2 (t). If
the gene is active then mRNA transcript molecules are synthesized with a constant
speed R. The protein translation proceeds with the rate Pξ1 (t), where P is a constant
(see Fig. 1.5). The mRNA and protein degradation rates are μ R and μ P , respectively.
Now, instead of Eq. (1.14) we have the following system of differential equations

ξ1 (t) = R A(t) − μ R ξ1 (t),
(1.15)
ξ2 (t) = Pξ1 (t) − μ P ξ2 (t),

where A(t) = 1 if the gene is active and A(t) = 0 in the opposite case. Then the
process ξ (t) = (ξ1 (t), ξ2 (t), A(t)), t ≥ 0, is a homogeneous PDMP.
As in the previous model the process  ξ (t) has jump points when the gene changes
its activity and formula (1.7) allow us to find the distribution of the time between
consecutive jumps. Let x0 and y0 be the initial number of mRNA and protein mole-
cules. Let us denote by πti (x0 , y0 ) = x(t) the solution of the system (1.15) at time
t, where i = 0 and i = 1 correspond to an inactive and an active state, respectively
(see Fig. 1.6). Then the probability distribution function of the length of an inactive
(or active) state is given by
t
1 − exp − qi (πsi (x0 , y0 )) ds .
0

Auto-regulation

q0
Inactive Active R P
mRNA Protein
gene gene
q1

R P

degradation degradation

Fig. 1.5 The diagram of autoregulated gene expression II

14 1 Biological Models

2 2

1 1

1 1
i=0 i=1
1 1

Fig. 1.6 The flows π i for P = R = μ P = μ R = 1: left i = 0, right i = 1. The set between the
curves χ0 and χ1 is a stochastic attractor of the related PDMP, i.e. almost all trajectories of the
PDMP enter this set and do not leave it

In some cases the process of protein production has an additional primary tran-
script step [100]. After the activation of a gene, the DNA code is transformed into
pre-mRNA form of transcript. Then, pre-mRNA molecules are converted to func-
tional forms of mRNA, which is transferred into the cytoplasm, where during the
translation phase mRNA is decoded into a protein. We consider three-dimensional
model of gene expression with variables ξ1 , ξ2 , ξ3 describing evolution of pre-mRNA,
mRNA and protein levels. This model is described by the following system of dif-
ferential equations ⎧ 
⎨ ξ1 (t) = R A(t) − (C + μ p R )ξ1 (t),
⎪
ξ2 (t) = Cξ1 (t) − μ R ξ2 (t), (1.16)
⎪
⎩ 
ξ3 (t) = Pξ2 (t) − μ P ξ3 (t).

Here R is the speed of synthesis of pre-mRNA molecules if the gene is active, C is

the rate of converting pre-mRNA into active mRNA particles, μ p R is the pre-mRNA
degradation rate and the rest constants are the same as in the previous model.

Remark 1.1 Examples from Sects. 1.8 and 1.9 show that using PDMPs to model
autoregulated genetic networks is very natural. New results concerning this subject
can be also found in [71, 77, 122, 123].

Models described in the last two sections lead to the following general scheme
called dynamical systems with random switching (see Fig. 1.7). We consider a family
of deterministic processes indexed by a set I ⊂ R. Each process is a solution of a
system of differential equations in some subset X of the space Rd :
1.9 Stochastic Gene Expression II 15

Fig. 1.7 A dynamical

system with random
switching: x1 = πT11 x0 , x3
x2 = πT22 x1 , …

x4

x1 2
x2

1
1 2

x0

x  (t) = bi (x(t)), (1.17)

i ∈ I . Temporary, elements of I are called states. The set I should be a measurable

set but it can be finite, countable or even uncountable. We assume that for each point
x ∈ X we have a transition probability on I denoted by Px (i, B). We assume that
qi (x) is the intensity of leaving the state i, i.e. if at time t the system is in the state
i and at the point x ∈ Rd , then it leaves the state i in a small time interval of length
Δt with probability
qi (x)Δt + o(Δt).

We choose a point x0 ∈ Rd and a state i ∈ I . We go along the trajectory, which is the

solution of Eq. (1.17) with the initial condition x(0) = x0 . We stay in the state i for a
random time Ti and the random variable Ti has the probability distribution function
t
1 − exp − qi (πsi (x0 )) ds , (1.18)
0

where πti (x0 ) is the solution of Eq. (1.17) with the initial condition x(0) = x0 . At
time Ti we leave the state i and using transition probability Px1 (i, B), x1 = πTi i (x0 ),
we choose the next state, say j. We repeat the procedure starting now from the point
x1 and the state j and continue it in subsequent steps. If i(t) is the state of the system
at time t then the process  ξ (t) = (x(t), i(t)), t ≥ 0, is a homogeneous PDMP. We
can also treat the process ξ (t) as a solution of a single system of equations:
16 1 Biological Models

x  (t) = bi(t) (x(t)),
(1.19)
i  (t) = 0,

with transition at the jump times given by

x(tn ) = x(tn− ), Prob(i(tn ) ∈ B) = Px(tn− ) (i(tn− ), B).

1.10 Gene Regulatory Models with Bursting

The central dogma of molecular biology, suggested 60 years ago, proposes that the
information flows from DNA to mRNA and then to proteins. A gene is expressed
through the processes of transcription and translation. First, in the process of tran-
scription, the enzyme RNA polymerase uses DNA as a template to produce an mRNA
molecule which can be translated to make the protein encoded by the original gene.
Recent advances to monitor the behaviour of single molecules allow experimental-
ists to quantify the transcription of mRNA from the gene as well as the translation
of the mRNA into protein. The visualization revealed that in many cases mRNA
and protein synthesis occur in quantal bursts in which the production is active for a
relatively short and random period of time and during that time a random number of
molecules is generated. There is the well-documented production of mRNA and/or
protein in stochastic bursts in both prokaryotes and eukaryotes. For example, in [25]
the authors observe bursts of expression of beta-galactosidase in E. coli, yeast and
mouse cells.
An analysis of the data obtained from such experiments was used in [40] to predict
the size and frequency of the bursts of proteins and to derive some of the first analytical
expressions for protein distributions in steady states across a population of cells.
Including mRNA dynamics in gene regulatory models introduces more complexity.
We assume that the rates of degradation of mRNA and of protein are γ1 and γ2 ,
respectively, and that k1 is the rate of transcription, while k2 is the rate of translation
of mRNA. Let us first consider translational bursting. It was observed in [25, 121]
that the amplitude of protein production through bursting translation of mRNA is
geometric/exponentially distributed with mean number of protein molecules per burst
equal to b = k2 /γ1 and the frequency of bursts being equal to a = k1 /γ2 . If ξ(t)
denotes the number of protein molecules then ξ(t) can be described in terms of
a continuous-time Markov chain with state space equal to N (see [72]). Here, we
follow the approach of [40] (see also [71, 72]) and we assume that ξ(t) denotes
the concentration of protein in a cell, which is the number of molecules divided
by the volume of the cell, being a continuous variable. We consider the general
situation when the degradation of proteins is interrupted at random times tn when
new molecules are being produced with frequency ϕ depending on the current level x
of the concentration of proteins and according to a distribution with density h(x, y).
Thus between the jump times ξ(t) is the solution of the equation
1.10 Gene Regulatory Models with Bursting 17

ξ  (t) = −γ2 ξ(t)

and at jump time tn we go from the point ξ(tn −) to the point ξ(tn −) + ϑn where ϑn
has a distribution with density h, possibly depending on x, so that

Prob(ϑn ∈ B|ξ(tn −) = x) = h(x, y) dy.

B

In particular, the model considered in [40] assumes that ϕ is a constant equal to a

and that
1
h(x, y) = e−y/b , y > 0,
b
in that case one can easily calculate a steady-state distribution, which is the gamma
distribution with parameters a and b. To get analytical expressions for the distrib-
ution one can also take functions regulating the frequency of bursts to be Hill-type
functions, see [40, 71, 72].
Transcriptional bursts have also been observed [43] with the number of molecules
produced per burst being exponentially distributed. We let ξ1 and ξ2 denote the
concentrations of mRNA and protein respectively. We assume that ξ1 and ξ2 are
solutions of the equations

ξ1 (t) = −γ1 ξ1 (t), ξ2 (t) = k2 ξ1 (t) − γ2 ξ2 (t),

between jump times, and that mRNA are produced in bursts with jump rate ϕ depend-
ing on the current level ξ2 of proteins, see [15]. Regulatory mechanisms and genetic
switches can be modelled using this type of PDMPs [18, 69, 73, 80]. These are
particular examples of the so-called flows with jumps as described at the end of
Sect. 1.5.

1.11 Neural Activity

A neuron is an electrically excitable cell that processes and transmits information

through electrical signals. The neuron’s membrane potential Vm is the inside potential
minus the outside potential. If a cell is in the resting state, then this potential, denoted
by Vm,R , is about −70 mV. The depolarization is defined as

V = Vm − Vm,R .

A cell is said to be excited (or depolarized) if V > 0 and inhibited (or hyperpolarized)
if V < 0. The Stein model [107, 108] describes how the depolarization V (t) is
changing in time. The cell is initially at rest so that V (0) = 0. Nerve cells may be
excited or inhibited through neuron’s synapses—junctions between nerve cells (or
18 1 Biological Models

subthreshold phase
λI λE λI λE
0

V − xI V V + xE V − xI V V + xE θ

tR

refractory phase

Fig. 1.8 Changes of the depolarization in the Stein model

between muscle and nerve cell) such that electrical activity in one cell may influence
the electrical potential in the other. Synapses may be excitatory or inhibitory. We
assume that there are two non-negative constants x E and x I such that if at time
t an excitation occurs then V (t + ) = V (t − ) + x E and if an inhibition occurs then
V (t + ) = V (t − ) − x I . The jumps (excitations and inhibitions) may occur at random
times according to two independent Poisson processes N E (t), N I (t), t ≥ 0, with
positive intensities λ E and λ I , respectively. Between jumps the depolarization V (t)
decays according to the equation V  (t) = −αV (t). When a sufficient (threshold)
level θ > 0 of excitation is reached, the neuron emits an action potential (fires). This
will be followed by an absolute refractory period of duration t R , during which V ≡ 0
and then the process starts again (see Fig. 1.8).
Now, we describe the neural activity as a PDMP. Since the refractory period has
a constant duration we can use a model similar to that of Sect. 1.6 with two phases A
and B, where A is the subthreshold phase and B is the refractory phase of duration
t R . We consider two types of jump points: when the neuron is excited or inhibited
and the ends of refractory periods. Thus, we can have one or more jumps inside
the phase A. Let t0 , t1 , t2 , . . . be the subsequent jump times. We denote by F the
subset of jump points consisting of firing points. Let  ξ (t) = (V (t), 1) if the neuron
is in the phase A, and  ξ (t) = (y, 2) if the neuron is in the phase B, where y is the
time since the moment of firing. The process  ξ (t) = (ξ1 (t), 
ξ2 (t)), t ≥ 0, is defined
on the phase space (−∞, θ ) × {1} ∪ [0, t R ] × {2} and between jumps it satisfies the
following system of equations:

 −αξ1 (t), if 
ξ2 (t) = 1,
ξ1 (t) =
1, if 
ξ2 (t) = 2.

If the neuron is in the phase A, i.e.  ξ2 (t) = 1, the depolarization can jump with
intensity λ = λ E + λ I . It means that F(t) = 1 − e−λt is the distribution function of
tn − tn−1 if tn−1 ∈
/ F , while F(t) = 0 for t < t R and F(t) = 1 for t ≥ t R if tn−1 ∈ F .
The transition at a jump point depends on the state of the neuron (its phase and the
ξ (tn− ) = (t R , 2) then 
value of its depolarization). If  ξ (tn ) = (0, 1) with probability

one; if ξ (tn ) = (V, 1) and V < θ − x E then 
−
ξ (tn ) = (V + x E , 1) with probability
1.11 Neural Activity 19

λ E /λ and 
ξ (tn ) = (V − x I , 1) with probability λ I /λ; while if  ξ (tn− ) = (V, 1) and
V ≥ θ − x E then  ξ (tn ) = (0, 2) with probability λ E /λ and 
ξ (tn ) = (V − x I , 1) with
probability λ I /λ.

Remark 1.2 In [26] a simpler version of Stein’s model is considered without the
refractory period. A PDMP V (t), t ≥ 0, corresponding to this model is defined on
the phase space (−∞, θ ), between jumps it satisfies the equation V  (t) = −αV (t),
and F(t) = 1 − e−λt is the distribution function of tn − tn−1 . The transition at a jump
point are given by two formulae:
if V (tn− ) < θ − x E then V (tn ) = V (tn− ) + x E with probability λ E /λ and V (tn ) =
V (tn− ) − x I with probability λ I /λ;
and if V (tn− ) ≥ θ − x E then V (tn ) = 0 with probability λ E /λ and V (tn ) = V (tn− ) −
x I with probability λ I /λ.

Remark 1.3 There are several different models of the neural activity. The interested
reader is referred to the monographs [7, 110] and papers [23, 24, 39, 42, 111].

1.12 Processes with Extra Jumps on a Subspace

Many interesting applications of PDMPs in biological sciences can be found in books

and papers devoted to stochastic hybrid systems. Such systems have appeared as sto-
chastic versions of deterministic hybrid systems—systems described by differential
equations and jumps. Stochastic hybrid systems are very close to PDMPs but they
also contain diffusion processes. One can find a definition of stochastic hybrid sys-
tems and many examples of their applications in the books [3, 16, 27, 119] and the
papers [51, 106].
In [52, 59], which are devoted to stochastic hybrid systems, we can find models of
the production of subtilin by the bacterium Bachillus subtilis. Although this process
plays an important role in biochemical technology, we do not recall these models
in detail but only draw the reader’s attention to an interesting mathematical aspect
of these models. In order to survive, bacteria produce an antibiotic called subtilin to
eliminate competing microbial species in the same ecosystem. The model consid-
ered in [52] consists of five differential equations describing: the size of B. subtilis
population x1 , the total amount of nutrient available in the environment x2 , and the
concentration of three types of proteins taking part in the process x3 , x4 , x5 . Two of
these equations are without switches, two have switches depending on the states of
two genes and one of them have a switch which depends on the level of the total
amount of nutrient, namely,

−μx3 (t), if x2 (t) ≥ θ,
x3 (t) =
P3 − μx3 (t), if x2 (t) < θ,
20 1 Biological Models

where θ is a positive constant. A PDMP corresponding to this model is defined on

the space R5+ and has extra jumps on the subspace S = {x ∈ R5+ : x2 = θ }.
Processes with jumps at some surfaces appear in a natural way in other models.
For example, if we consider the movement of an individual in a bounded domain G
we can use a velocity jump process to describe this movement in the interior of this
domain (see Sect. 1.4). But if the process reaches the boundary ∂G of this domain
then we can use the theory of classical and stochastic billiards [36] to describe the
changes of the velocity.
We now give two simple examples of processes with jumps on a subspace which
allow us to reveal some problems with definition of PDMPs. The first one is a
modification of the model of gene expression from Sect. 1.8. Again, we denote the
number of protein molecules by ξ(t) and assume that protein molecules degrade
with rate μ and if the gene is active then proteins are produced with rate P, thus
ξ(t) satisfies Eq. (1.14). The gene can be inactivated with intensity q1 (ξ(t)) and
activated if the number of protein molecules ξ(t) reaches the minimal level θ , where
0 ≤ θ < P/μ. 
 The  process ξ (t) = (ξ(t), A(t)), t ≥ 0, is a PDMP defined on the
phase space θ, μP × {0, 1} and has extra jumps at the boundary point (θ, 0). Now,
the intensity of activation cannot be a function, but, if we formally assume that this
intensity is an infinite measure concentrated at the point θ , then formula (1.7) allows
us to find the distribution of the time between consecutive jumps. If x0 is the number of
protein molecules at the last jump from active to inactive state, then πt0 (x0 ) = x0 e−μt
and πt0 (x0 ) = θ for t = T := μ1 log(x0 /θ ). It means that the probability distribution
function of the length of an inactive state is given by

0, if t < T,
F(t) = (1.20)
1, if t ≥ T.

A similar modification can be made in the model from Sect. 1.9, i.e. we assume
that the gene is activated if the number of protein molecules ξ2 (t) reaches the level θ ,
where 0 < θ ≤ P R/μ R μ P . It can happen that ξ1 (t¯) < μ p θ at the moment of gene
activation t¯, and then ξ2 (t¯) < 0. It implies that the number of protein molecules ξ2 (t)
decreases in some interval (t¯, t¯ + ε), thus it goes below the level θ . It is clear that
the gene should remain active until the number of protein molecules ξ2 (t) crosses
at least the level θ , therefore, we assume that the intensity of inactivation q1 (x1 , x2 )
equals zero if x2 ≤ θ . The corresponding ξ (t) = (ξ1 (t), ξ2 (t), A(t)), t ≥ 0
  PDMP

is defined on the phase space X = 0, μRR × 0, μPP μR R × {0, 1} and has extra jumps
 
at the line segment S = 0, μRR × {θ } × {0, 1}. Here, an extra jump can be inside
the phase space. The probability distribution function of the length of an inactive
state is given by Eq. (1.20) and the intensity of activation is a measure such that
μ(X \ S) = 0 and μ{(x, θ, 0)} = ∞ for x ∈ 0, μRR .
An interesting PDMP with jumps on some subset is used in [20, 21] to describe
the motor-driven transport of vesicular cargo to synaptic targets located on the axon
or dendrites of a neuron. In this model, a particle is transported along the interval
[0, L] to some target x0 . The particle can be in one of three states i ∈ I = {−1, 0, 1}
1.12 Processes with Extra Jumps on a Subspace 21

and at a given state i it moves with speed i. Transitions between the three states
are governed by a time homogeneous Markov chain ϑ(t) defined on the set I with
the transition rate from i to j given by q ji . The particle starts from the point x = 0
and moves to the right. We assume that 0 is a reflecting point and L is an absorbing
point. Finally, we assume that if the particle is in some neighbourhood U of the
target x0 and is in the stationary state i = 0, then the particle can be absorbed by
the target at a rate κ. If the particle is absorbed by the target or by the point L then
the process repeats. The movement of the particle between jumps is described by a
velocity jump process ξ(t) = (x(t), i(t)), t ≥ 0, from Sect. 1.4 defined on the phase
space [0, L] × I , where the pair (x(t), i(t)) satisfies the following system of ordinary
differential equations 
x  (t) = i(t),
(1.21)
i  (t) = 0.

If x ∈ (0, L) then the process ξ(t) can jump from (x, i) to (x, j) with intensity q ji
for i, j ∈ I , and if x ∈ U the process ξ(t) can also jump from (x, 0) to (0, 1) with
intensity κ. The process ξ(t) has two extra jumps from (0, −1) and from (L , 1) to
(0, 1) with probability 1.
There are a lot of different stochastic models of intracellular transport. The inter-
ested reader is referred to the review paper [22].
Remark 1.4 We can consider PDMPs with jump intensities given by measures
defined on the trajectories of the processes. Assume that a process ξ(t) is defined
on the space X ⊂ Rd and between jumps satisfies the equation ξ  (t) = g(ξ(t)).
Let π(t, x0 ) = x(t) be the solution of this equation with the initial condition
x(0) = x0 defined on the maximal time interval (a, b), −∞ ≤ a < b ≤ ∞. Let
L = {π(t, x0 ) : t ∈ (a, b)} be the trajectory of x0 . We consider only the case when
the trajectory is not periodic. For x = π(t1 , x0 ) and y = π(t2 , x0 ) with t1 < t2 , we
denote by L(x, y) the arc {π(t, x0 ) : t ∈ (t1 , t2 ]}. The jump intensity on the trajec-
tory L can be a non-negative Borel measure μ on L. If ξ(0) = x, x ∈ L, then the
probability distribution function of the time till the next jump is given by

F(t) = 1 − Φx (t) = 1 − e−μ(L(x,π(t,x))) .

In particular, if μ(x) = c > 0 and ξ(t − ) = x then the process has a jump at time t
with probability 1 − e−c . In this case we should assume that the probability of jump
at the point y = ξ(t) is zero.

1.13 Size-Structured Population Model

We return back to the model of size distribution from Sect. 1.5 but instead of a single
cell we consider the size distribution of all cells in the population. As in Sect. 1.5 the
size x(t) of a cell grows according to the equation
22 1 Biological Models

x  (t) = g(x(t)).

A single cell with size x replicates with rate b(x) and dies with the rate d(x). A
daughter cell has a half size of the mother cell. Let assume that at time t we have k
cells and denote their sizes by x1 (t), x1 (t), . . . , xk (t). We can assume that a state of
the population at time t is the set

{x1 (t), . . . , xk (t)}

and the evolution of the population is a stochastic process ξ(t) = {x1 (t), . . . , xk (t)}.
Since the values of this process are sets of points, the process ξ(t) is called a point
process. Although such approach is a natural one, it has one important disadvantage.
We are not able to describe properly the situation when two cells have the same size.
One of the solution of this problem is to consider ξ(t) as a process whose values are
multisets. We recall that a multiset (or a bag) is a generalization of the notion of set
in which members are allowed to appear more than once. Another solution of this
problem is to consider ξ(t) as a process with values in the space of measures given
by
ξ(t) = δx1 (t) + · · · + δxk (t) ,

where δa denotes the Dirac measure at the point a, i.e. δa is the probability measure
concentrated at the point a. Also this approach has some disadvantages, for example
it is rather difficult to consider differential equations on measures. One solution of this
problem is to consider a state of the system as a k-tuple (x1 (t), . . . , xk (t)). But some
cells can die or split into two cells which changes the length of the tuple. To omit this
problem we introduce an extra “dead state” ∗ and describe the state of the population
at time t as an infinite sequence of elements from the space R+ ∗ = [0, ∞) ∪ {∗}
which has on some positions the numbers x1 (t), . . . , xk (t) and on other positions
has ∗. In order to have uniqueness of states, we introduce an equivalence relation
∼ in the space X of all R+ ∗ -valued sequences x such that x i = ∗ for all but finitely
many i. Two sequences x ∈ X and y ∈ X are equivalent with respect to ∼ if y can be
obtained as a permutation of x, i.e. x ∼ y if and only if there is a bijective function
σ : N → N such that y = (xσ (1) , xσ (2) , . . .). The phase space  X in our model is the
space of all equivalence classes with respect to ∼, i.e.  X = X/ ∼.
We now can describe the evolution of the population as a stochastic process
ξ(t) = [(x1 (t), x2 (t), . . .)] with values in the space  X . The process ξ(t) has jump
points when one of the cells dies or replicates. We define g(∗) = b(∗) = b(∗) = 0
and assume that x(t) = ∗ is the solution of the equation x  (t) = 0 with the initial
condition x(0) = ∗. Between jumps the process ξ(t) satisfies the equation

[(x1 (t) − g(x1 (t)), x2 (t) − g(x2 (t)), . . .)] = [(0, 0, . . .)]. (1.22)

For t ≥ 0 and x 0 ∈ R+
∗ we denote by π(t, x ) the solution x(t) of the equation
0

x (t) = g(x(t)) with the initial condition x(0) = x 0 . Let x0 = [(x10 , x20 , . . .)] ∈ 

X
and define
1.13 Size-Structured Population Model 23

π̃ (t, x0 ) = [(π(t, x10 ), π(t, x20 ), . . .)].

The jump rate function ϕ(x) at the state x = [(x1 , x2 , . . .)] is the sum of rates of
death and division of all cells:
∞

ϕ(x) = (b(xi ) + d(xi )). (1.23)
i=1

If x0 ∈ 
X is the initial state of the population at a jump time tn , then the probability
distribution function of tn+1 − tn is given by
t
1 − exp − ϕ(π̃ (s, x0 ) ds . (1.24)
0

At time tn one of the cells dies or replicates. If a cell dies we change the sequence
by removing its size from the sequence and we have

  di (xi (tn− ))
Prob ξ(tn ) = [(x1 (tn− ), . . . , xi−1 (tn− ), xi+1 (tn− ), xi+2 (tn− ), . . .)] =
ϕ(ξ(tn− ))

for i ∈ N. If a cell replicates, we remove its size from the sequence and add two new
elements in the sequence with the sizes of the daughter cells and we have
 
Prob ξ(tn ) = [(x1 (tn− ), . . . , xi−1 (tn− ), 21 xi (tn− ), 21 xi (tn− ), xi+1 (tn− ),xi+2 (tn− ), . . .)]
bi (xi (tn− ))
=
ϕ(ξ(tn− ))

for i ∈ N. In this way, we have checked that the point process ξ(t), t ≥ 0, is a
homogeneous PDMP with values in  X.
We can identify the space 
X with the space N of finite counting measures on R+
by a map η : 
X → N given by

η(x) = δxi (1.25)
{i: xi =∗}

where x = [(x1 , x2 , . . .)]. It means that the process η(ξ(t)), t ≥ 0, is a homogeneous

PDMP with values in N .
Remark 1.5 In order to describe the jump transformation at times tn we need, for-
mally, to introduce a σ -algebra Σ of subset of  X to define a transition probability
P: X × Σ → [0, 1]. Usually, Σ is a σ -algebra of Borel subsets of  X , thus we need
to introduce a topology on the space  X . Since the space N is equipped with the
topology of weak convergence of measures, we can define open sets in  X as preim-
ages through the function η of open sets in N . We can also construct directly a metric
on the space X . Generally, a point process describes the evolution of a configuration
24 1 Biological Models

of points in a state space which is a metric space (S, ρ). First, we extend the state
space S adding “the dead element” ∗. We need to define a metric on S ∪ {∗}. The best
situation is if S is a proper subset of a larger metric space S  . Then we simply choose
∗ as an element from S  which does not belong to S and we keep the same metric.
In the opposite case, first we choose x0 ∈ S and define ρ(∗, x) = 1 + ρ(x0 , x) for
x ∈ S. Next, we define a metric d on the space X by

d(x, y) = max ρ(xi , yi )

i∈N

and, finally, we define a metric d on the space 

X by


d([x], [y]) = min{d(a, b) : a ∈ [x], b ∈ [y]}.

1.14 Age-Structured Population Model

We consider a population of one sex (usually females) in which all individuals are
characterized by their age x ≥ 0. We assume that an individual with age x dies with
rate d(x) and gives birth to one child with rate b(x) p1 or twins with rate b(x) p2 , where
p1 + p2 = 1. The model is very similar to that of the size-structured population. We
assume that g(∗) = b(∗) = d(∗) = 0 and g(a) = 1 for a ∈ R+ . Then the evolution
of the population is described by a stochastic process ξ(t) = [(x1 (t), x2 (t), . . .)] with
values in the space  X with jumps when an individual dies or gives birth. Between
jumps the process ξ(t) satisfies Eq. (1.22) and the jump rate function ϕ(x) is given by
(1.23). Let us assume that at the jump time tn−1 the population consists of k individuals
with ages x1 , x2 , . . . , xk . It is easy to check that the probability distribution function
of tn+1 − tn is given by

t 
k
1 − exp − (b(xi + s) + d(xi + s)) ds .
0 i=1

If ξ(tn− ) = x = [(x1 , x2 , . . .)], then ξ(tn ) has one of the following values:

[(x1 , . . . , xi−1 , xi+1 , . . .)] with probability d(xi )/ϕ(x) for i ∈ N,

[(0, x1 , x2 , . . .)] with probability b(x) p1 /ϕ(x), (1.26)
[(0, 0, x1 , x2 , . . .)] with probability b(x) p2 /ϕ(x),
∞
where b(x) = i=1 b(xi ).
One can consider more general model when the birth and death rates depend not
only on the age of an individual but also on the state of the whole population (e.g. on
the size of the population). Let x = [(x1 , x2 , . . .)] be the state of the population and
b(xi , x), d(xi , x) be the birth and death rates, respectively, of an individual with age
1.14 Age-Structured Population Model 25

xi . Even in this case the stochastic process ξ(t) is a homogeneous PDMP. Between
jumps, the process ξ(t) satisfies Eq. (1.22). The jump rate function is given by
∞

ϕ(x) = (b(xi , x) + d(xi , x)). (1.27)
i=1

We also need to replace b(xi ) and d(xi ) by b(xi , x) and d(xi , x) in formulas (1.26)
for the transition at a jump time.

1.15 Asexual Phenotype Population Model

As in Sects. 1.13 and 1.14 we consider a population of one sex in which all individuals
are characterized by their phenotype which is described by a vector x from a set
F ⊂ Rd . The phenotype remains constant during the whole life of an individual. As in
the previous model the state space of the population is  X . Let x = [(x1 , x2 , . . .)] be the
state of the population. We assume that an individual with phenotype xi dies with rate
d(xi , x) and gives birth to one child with rate b(xi , x) p1 or twins with rate b(xi , x) p2 ,
where p1 + p2 = 1. Phenotypes of progeny can differ from the parent’s phenotype.
We assume that if x is the phenotype of the parent then the progeny phenotype is from
a set B ⊂ F with probability P(x, B), where P : F × B(F) → [0, 1] is a transition
probability on F and B(F) is a σ -algebra of Borel subsets of F. We assume that the
phenotype of twins is the same (but it is not difficult to consider a model which is
based on a different assumption). The point process ξ(t) = [(x1 (t), x2 (t), . . .)] which
describes the phenotype structure of the population is a pure jump-type homogeneous
Markov process, i.e. ξ  (t) = 0. The jump rate function ϕ(x) is given by (1.27). Since
ξ(t) is a pure jump-type homogeneous Markov process the probability distribution
function of tn+1 − tn is given by 1 − e−ϕ(x)t . If ξ(tn− ) = x = [(x1 , x2 , . . .)], then

d(xi , x)
Prob(ξ(tn ) = [(x1 , . . . , xi−1 , xi+1 , . . .)]) = for i ∈ N,
ϕ(x)
∞
 p1 b(xi , x)P(xi , B)
Prob(ξ(tn ) ∈ [(x, x1 , x2 , . . .)] : x ∈ B) = ,
i=1
ϕ(x)
∞
p2 b(xi , x)P(xi , B)
Prob(ξ(tn ) ∈ [(x, x, x1 , x2 , . . .)] : x ∈ B) = .
i=1
ϕ(x)

It should be noted that in cellular populations we need to change the last two
formulas because progeny consists of two cells ( p1 = 0), and we lose a cell which
replicates. In this case, we have

b(xi , x)P(xi , B)
Prob(ξ(tn ) ∈ [(x, x, x1 , . . . , xi−1 , xi+1 , . . .)] : x ∈ B) = .
ϕ(x)
26 1 Biological Models

Also the assumption that both daughter cells have the same phenotype is generally not
correct. If we consider hematopoietic stem cells—precursors of blood cells living
in the bone marrow—then such cells can be at different levels of morphological
development. After division a daughter cell can remain on the same level as the
mother cell or go to the next level. We can introduce a model with a point process to
describe the evolution of hematopoietic stem cells but a model which is based on a
continuous-time Markov chain seems to be more appropriate.

1.16 Phenotype Model with a Sexual Reproduction

Modelling population with sexual reproduction is a challenging task because such

models depend on many factors including social behaviour of individuals. The main
problem is with the description of the mating process [104]. The simplest case is
when each individual has both male and female reproductive organs which is a normal
condition in most plants and invertebrates. Then we can consider semi-random or
assortative mating. In the case of semi-random mating we assume that an individual
with phenotype x ∈ F has mating rate p(x). It means that if the population consists
of k individuals with phenotypes x1 , x2 , . . . , xk then individuals with phenotype xi
and x j form a pair with rate

p(xi ) p(x j )
p(xi , x j ) = k . (1.28)
r =1 p(xr )

If we exclude self-fertilization then this rate can be given by

1 p(xi ) p(x j ) 1 p(xi ) p(x j )

p(xi , x j ) =  +  (1.29)
2 rk=1,r =i p(xr ) 2 rk=1,r = j p(xr )

for i = j and p(xi , x j ) = 0 if i = j. We assume that if x1 and x2 are the par-

ents’ phenotypes then the progeny phenotype is from a set B ⊂ F with probability
P(x1 , x2 , B), where P : F × F × B(F) → [0, 1] is a transition probability from
F × F to F, i.e. P(x1 , x2 , ·) is a probability measure on B(F) for all x1 , x2 and
for each Borel set B ⊂ F, (x1 , x2 ) → P(x1 , x2 , B) is a measurable function. For
simplicity we consider a model when the result of reproduction is only one child. If
x = [(x1 , x2 , . . .)] is the state of the population then d(xi , x) is the death rate for an
individual with phenotype xi . As in the previous model the population is described
by a pure jump-type homogeneous Markov process ξ(t) = [(x1 (t), x2 (t), . . .)]. The
jump rate function ϕ(x) is given by
∞
 ∞ 
 ∞
ϕ(x) = d(xi , x) + p(xi , x j ), (1.30)
i=1 i=1 j=1
1.16 Phenotype Model with a Sexual Reproduction 27

where d(∗, x) = 0 and p(∗, xi ) = p(xi , ∗) = 0 for i ∈ N. If ξ(tn− ) = x = [(x1 ,

x2 , . . .)], then

d(xi , x)
Prob(ξ(tn ) = [(x1 , . . . , xi−1 , xi+1 , . . .)]) = for i ∈ N,
ϕ(x)
∞ 
 ∞
p(xi , x j )P(xi , x j , B)
Prob(ξ(tn ) ∈ [(x, x1 , x2 , . . .)] : x ∈ B) = .
i=1 j=1
ϕ(x)

We now consider assortative mating. Then individuals with similar traits mate
more often than they would choose a partner randomly. Then we can use matching
theory, according to which each participant ranks all potential partners according
to its preferences and attempts to pair with the one with highest-ranking [2], or we
can adapt a model based on a preference function [41] used in two-sex populations
models. In assortative mating, a preference function a(x, y) is usually of the form
a(x, y) = ψ(x − y), where ψ : [0, ∞) → [0, ∞) is a continuous and decreasing
function. We assume that two individuals with phenotypes xi and x j form a pair with
rate
a(xi , x j ) ψ(xi − x j )
p(xi , x j , x) = ∞ = ∞ (1.31)
l=1 a(x i , xl ) l=1 ψ(x i − xl )

and the rest of the model is similar to that with semi-random mating.
If we consider a two-sex population (i.e. each individual is exclusively male or
female) then the mating process is more complex and there are only a few mathe-
matical models of it (see e.g. [60] for insect populations). Since the role of males
and females is different, it is clear that in such models we should characterize an
individual by its sex and phenotype. An important factor is a sexual selection [57]
which often depends on the social behaviour of individuals. Males usually produce
enough sperm to inseminate many females and in the case when they do not take part
in parental care their reproductive success depends on their phenotype. We give some
hints how to build a model in this case. First each individual is described by a pair
(x, s), where x is its phenotype, s = 0 if it is a male and s = 1 if it is a female. Let
x = [((x1 , s1 ), (x2 , s2 ), . . .)] be the state of the population. We assume that a female
with phenotype xi gives birth with rate b(xi , x) and a male with phenotype x j has
the competition rate p(x j , x). To simplify notation we set b(xi , x) = 0 if si = 0 and
p(x j , x) = 0 if si = 1. A female with phenotype xi and a male with phenotype x j
form a pair with rate
b(xi , x) p(x j , x)
p(xi , x j , x) = ∞ . (1.32)
r =1 p(xr , x)

The rest of the model is similar to the model of hermaphrodite population and we
omit it here.
28 1 Biological Models

1.17 Coagulation-Fragmentation Process in a

Phytoplankton Model

Mathematical modelling of plankton behaviour is a complex issue involving various

mathematical tools. A review of mathematical models of plankton dynamics can
be found in [102]. Phytoplankton cells tend to form aggregates in which they live
together like colonial organisms. Since the size of aggregates is important in the
study of fish recruitment, the change of size distribution of aggregates is a very
interesting problem both from biological and mathematical point of view. We now
introduce a point process which corresponds to a model of phytoplankton dynamics
introduced in [6], which takes into account growth and death of aggregates as well
as coagulation-fragmentation processes. More advanced models including a space
distribution of aggregates and diffusion can be found in [103].
In this model individuals are aggregates and they are characterized by their size
x > 0, which depends on the number of cells. The division or death of individual
cells changes the size of aggregates according to the equation x  (t) = g(x(t)) but
aggregates can die, for example by sinking to a seabed or whatever cause, with
mortality rate d(x). An aggregate can break with rate b(x) and the size y of its
descendants is given by the conditional density k(y, x), where k satisfies k(y, x) =
k(x − y, x). We have also a coagulation (aggregation) process, by which two distinct
aggregates join together to form a single one. We assume that the ability to glue to
another aggregate is given by the function p(x). Since the same aggregate cannot
join with itself, we assume that two aggregates with sizes xi and x j form a new
aggregate with rate p(xi , x j ) given by (1.29) for i = j and p(xi , x j ) = 0 if i = j.
Our model is similar to that from Sect. 1.13. The population of aggregates is
described by a point process ξ(t) with values in  X and satisfies Eq. (1.22). It remains
to incorporate to the model fragmentation and coagulation processes. We have a
jump if one of aggregates dies, splits or two aggregates form a new one. The jump
rate function ϕ(x) at state x = [(x1 , x2 , . . .)] is the sum of mortality, fragmentation
and coagulation rates:
∞
 ∞ 
 ∞
ϕ(x) = (d(xi ) + b(xi )) + p(xi , x j ), (1.33)
i=1 i=1 j=1

where d(∗) = b(∗) = p(xi , ∗) = p(∗, xi ) = 0 for i ∈ N. The probability distribu-

tion function of tn+1 − tn is given by (1.24). If ξ(tn− ) = x = [(x1 , x2 , . . .)], then
1.17 Coagulation-Fragmentation Process in a Phytoplankton Model 29

d(xi )
Prob(ξ(tn ) = [(x1 , . . . , xi−1 , xi+1 , . . .)]) = ,
ϕ(x)

b(xi ) k(y, xi ) dy
Prob(ξ(tn ) ∈ [(x1 , . . . , xi−1 , x, xi − x, xi+1 , . . .)] : x ∈ B) = B
,
ϕ(x)
p(xi , x j )
Prob(ξ(tn ) = [(xi + x j , x1 , . . . , xi−1 , xi+1 , . . . , x j−1 , x j+1 , . . .)]) =
ϕ(x)

for i, j ∈ N.

1.18 Paralog Families

We now apply a point process to a model of evolution of paralog families in a genome

[98]. Two genes present in the same genome are said to be paralogs if they are genet-
ically identical. Let us assume that xi is a number of copies of a given gene i (we do
not repeat paralogous genes). We want to find a point process whose points are sizes
of paralog families. The model is based on three fundamental evolutionary events:
gene loss, duplication and accumulated change, called for simplicity a mutation. A
single gene can:
• be duplicated with rate d,
• be removed from the genome with rate r ,
• mutate with rate m and then the gene starts a new one-element paralog family.
Moreover, we assume that all elementary events are independent of each other.
As a result of fundamental evolutionary events a family of size xi increases to size
xi + 1 with rate d xi , decreases to size xi − 1 with rate r xi , and decreases to size xi − 1
and forms a new family of size 1 with rate mxi (Fig. 1.9). We have a jump at any
of fundamental evolutionary events. The point process ξ(t) = [(x1 (t), x2 (t), . . .)]
which describes the structure of paralogs is a pure jump-type homogeneous Markov
process with values in the space  X = X/ ∼. In this model we set ∗ = 0. Thus X is
the space of all sequences with non-negative integer values. The jump rate function
ϕ(x) at state x = [(x1 , x2 , . . .)] is given by
∞

ϕ(x) = (d + r + m)xi . (1.34)
i=1

The probability distribution function of tn+1 − tn is given by 1 − e−ϕ(x)t , where x =

ξ(tn ). If ξ(tn− ) = x = [(x1 , x2 , . . .)], then
30 1 Biological Models

0 1 2 3 n–1 n

Fig. 1.9 The diagram of the transition between states in the paralog families model

d xi
Prob(ξ(tn ) = [(x1 , . . . , xi−1 , xi + 1, xi+1 , . . .)]) = ,
ϕ(x)
r xi
Prob(ξ(tn ) = [(x1 , . . . , xi−1 , xi − 1, xi+1 , . . .)]) = ,
ϕ(x)
mxi
Prob(ξ(tn ) = [(1, x1 , . . . , xi−1 , xi − 1, xi+1 , . . .)]) = ,
ϕ(x)

for i ∈ N.
Remark 1.6 The main subject of the paper [98] is the size distribution of paralogous
gene families in a genome. Let sk (t) be the number of k-element families in our model
at time t. It means that sk (t) = #{i : xi (t) = k}. It follows from the description of
our model that
∞

s1 (t) = −(d + r )s1 (t) + 2(2m + r )s2 (t) + m js j (t), (1.35)
j=3

sk (t) = d(k − 1)sk−1 (t) − (d + r + m)ksk (t) + (r + m)(k + 1)sk+1 (t) (1.36)

for k ≥ 2.

1.19 Definition of PDMP

Having introduced examples of PDMPs, we are ready to give a formal definition of

a piecewise deterministic Markov process. In order to do it we need some auxiliary
definitions.
Let X̃ be a topological space. A function π : R+ × X̃ → X̃ such that
(a) π(0, x) = x for x ∈ X̃ ,
(b) π(s + t, x) = π(t, π(s, x)) for x ∈ X̃ , s, t ∈ R+ ,
(c) for each t ≥ 0 the mapping πt : X̃ → X̃ is continuous with πt (x) = π(t, x),
x ∈ X̃ ,
(d) for each x ∈ X̃ the mapping t → π(t, x) is càdlàg,, i.e. right-continuous and
has left-hand limits
1.19 Definition of PDMP 31

π(t + , x) := lim π(t, x) = π(t, x) and π(t − , x) := lim π(s, x) exists in X̃ ,

s↓t s↑t

is called a semiflow or dynamical system on X̃ .

Given a semiflow on X̃ we define a survival function [0, ∞) × X̃  (t, x) →
Φx (t) ∈ [0, 1] as a mapping with the following properties:
(a) for each t the transformation x → Φx (t) is measurable,
(b) for each x ∈ X̃ the mapping t → Φx (t) is right-continuous nonincreasing func-
tion,
(c) it satisfies
Φx (t + s) = Φx (t)Φπ(t,x) (s), t, s ≥ 0, x ∈ X̃ . (1.37)

In particular, 1 − Φx is the distribution function of a non-negative finite random

variable, provided that Φx (∞− ) := limt→∞ Φx (t) = 0. Otherwise, we extend Φx
to [0, ∞] by setting Φx (∞) = 1 − Φx (∞− ). The function Φ defines a probability
measure on [0, ∞] by

Φx ((s, t]) = Φx (s) − Φx (t), t ≥ s ≥ 0.

An example of a survival function is a mapping defined by

 t 
Φx (t) = exp − ϕ(π(r, x)) dr , t ≥ 0, (1.38)
0

where ϕ : X̃ → R+ , called a jump rate function, is a measurable function such that

the function r → ϕ(π(r, x)) is integrable on [0, ε(x)) for some ε(x) > 0 and all x.
To see that (1.37) holds note that
t+s t t+s
ϕ(π(r, x)) dr = ϕ(π(r, x)) dr + ϕ(π(r, x)) dr
0 0 t
t s
= ϕ(π(r, x)) dr + ϕ(π(r, π(t, x))) dr.
0 0

Another example of a survival function is given in Remark 1.4.

Suppose now that X is a Borel subset of X̃ . We define the exit time from X by

t∗ (x) = sup{t > 0 : π(s, x) ∈ X for s ∈ [0, t)}, x ∈ X. (1.39)

Let Γ be a subset of the boundary ∂ X of X defined by

Γ = {π(t∗ (x)− , x) : 0 < t∗ (x) < ∞, x ∈ X }. (1.40)

We call Γ the active boundary of the set X . If the active boundary Γ of a set X is
non-empty then we can also consider the following survival function
32 1 Biological Models
 t 
Φx (t) = 1[0, t∗ (x)) (t) exp − ϕ(π(r, x)) dr , t ≥ 0, (1.41)
0

where ϕ is a jump rate function.

A piecewise deterministic Markov process (PDMP) with values in X ⊂ X̃ is
determined through three characteristics (π, Φ, P) which are a semiflow π : R+ ×
X → X̃ , a survival function Φ, and a jump distribution P : (X ∪ Γ ) × B(X ) →
[0, 1] which is a transition probability, i.e. for each set B ∈ B(X ) the function
x → P(x, B) is measurable and for each x ∈ X ∪ Γ the function B → P(x, B) is
a probability measure.
Let us briefly describe the construction of the piecewise deterministic process
(PDP) {ξ(t)}t≥0 with characteristics (π, Φ, P). We extend the state space X to X Δ =
X ∪ {Δ} where Δ is a fixed state outside X representing a ’dead’ state for the process
and being an isolated point. For each x ∈ X , we set P(x, {Δ}) = 0 and π(t, x) = Δ
if t = ∞. We also set π(t, Δ) = Δ for all t ≥ 0, P(Δ, {Δ}) = 1, and ΦΔ (t) = 1 for
all t ≥ 0. Let (, F , P) be a probability space. Let τ0 = σ0 = 0 and let ξ(0) = ξ0
be an X -valued random variable. For each n ≥ 1, we can choose a positive random
variable σn satisfying

P(σn > t|ξn−1 = x) = Φx (t), t ∈ [0, ∞].

Define the nth jump time by

τn = τn−1 + σn

and set 
π(t − τn−1 , ξn−1 ) for τn−1 ≤ t < τn ,
ξ(t) =
ξn for t = τn ,

where the nth post-jump position ξn is an X Δ -valued random variable such that

P(ξn ∈ B|ξ(τn− ) = x) = P(x, B), x ∈ X Δ ∪ Γ,

and ξ(τn− ) = limt↑τn ξ(t). Thus, the trajectory of the process is defined for all t <
τ∞ := limn→∞ τn and τ∞ is called the explosion time. To define the process for all
times, we set ξ(t) = Δ for t ≥ τ∞ . The process {ξ(t)}t≥0 is called the minimal PDP
corresponding to (π, Φ, P). It has right-continuous sample paths, by construction. In
the next chapter, we will show that the minimal PDP exists and is a Markov process.
Chapter 2
Markov Processes

In this chapter, we provide a background material that is needed to define and study
Markov processes in discrete and continuous time. We start by giving basic examples
of transition probabilities and the corresponding operators on the spaces of functions
and measures. An emphasis is put on stochastic operators on the spaces of integrable
functions. The importance of transition probabilities is that the distribution of a
stochastic process with Markov property is completely determined by transition
probabilities and initial distributions. The Markov property simply states that the past
and the future are independent given the present. We refer the reader to Appendix
A for the required theory on measure, integration, and basic concepts of probability
theory.

2.1 Transition Probabilities and Kernels

2.1.1 Basic Concepts

In this section, we introduce transition kernels, stressing the interplay between ana-
lytic and stochastic interpretations. Let X be a set and let Σ be a σ -algebra of subsets
of X . The pair (X, Σ) is called a measurable space. A function P : X × Σ → [0, 1]
is said to be a transition kernel if
(1) for each set B ∈ Σ, P(·, B) : X → [0, 1] is a measurable function;
(2) for each x ∈ X the set function P(x, ·) : Σ → [0, 1] is a measure.
If P(x, X ) = 1 for all x ∈ X then P is said to be a transition probability on (X, Σ).
Let (Ω, F , P) be a probability space. A mapping ξ : Ω → X is said to be an
X -valued random variable if it is measurable, i.e. for any B ∈ Σ

© The Author(s) 2017 33

R. Rudnicki and M. Tyran-Kamińska, Piecewise Deterministic Processes
in Biological Models, SpringerBriefs in Mathematical Methods,
DOI 10.1007/978-3-319-61295-9_2
34 2 Markov Processes

{ξ ∈ B} = ξ −1 (B) = {ω ∈ Ω : ξ(ω) ∈ B} ∈ F .

The (probability) distribution or the (probability) law of ξ is a probability measure

μξ on (X, Σ) defined by

μξ (B) = P(ξ ∈ B), B ∈ Σ.

Given an X -valued random variable ξ on the probability space (Ω, F , P) the family

σ (ξ ) = {ξ −1 (B) : B ∈ Σ}

is a σ -algebra of subsets of Ω and it is called the σ -algebra generated by ξ . Note that

ξ is an X -valued random variable if and only if σ (ξ ) ⊆ F . If G is a sub-σ -algebra
of F then ξ is said to be G -measurable if σ (ξ ) ⊆ G . Thus one can also say that
σ (ξ ) is the smallest σ -algebra G such that ξ is G -measurable. Recall that sub-σ -
algebras F1 and F2 of F are called independent if P(A1 ∩ A2 ) = P(A1 )P(A2 ) for
any A1 ∈ F1 and A2 ∈ F2 . If a random variable ϑ with values in any measurable
space is independent of the σ -algebra G , i.e. if σ (ϑ) and G are independent, and if
ξ is an G -measurable X -valued random variable, then the conditional expectation of
g(ξ, ϑ) with respect to G is given by

E(g(ξ, ϑ)|G ) = E(g(ξ, ϑ)|ξ ) and E(g(ξ, ϑ)|ξ = x) = E(g(x, ϑ)) (2.1)

for all x ∈ X and for any measurable non-negative function g; see Lemma A.3.
Let Y be a metric space. Consider a probability measure ν on the Borel σ -algebra
B(Y ). Given a measurable mapping κ : X × Y → X , we define

P(x, B) = 1 B (κ(x, y))ν(dy), x ∈ X, B ∈ Σ, (2.2)

where 1 B is the indicator function, equal to one on the set B and zero otherwise.
Clearly, P is a transition probability. For a Y -valued random variable ϑ with distrib-
ution ν and for every x ∈ X , the X -valued random variable κ(x, ϑ) has distribution
P(x, ·), since

P(κ(x, ϑ) ∈ B) = E(1 B (κ(x, ϑ))) = 1 B (κ(x, y))ν(dy) = P(x, B), B ∈ Σ.

As a particular example, we can take Y to be the unit interval [0, 1] and ν to be the
Lebesgue measure on [0, 1]; in that case ϑ is said to be uniformly distributed on the
unit interval [0, 1]. Now let ξ0 be an X -valued random variable independent of ϑ and
let ξ1 = κ(ξ0 , ϑ). We can write

P(ξ1 ∈ B|ξ0 = x) = P(κ(ξ0 , ϑ) ∈ B|ξ0 = x) = E(1 B (κ(ξ0 , ϑ))|ξ0 = x)

2.1 Transition Probabilities and Kernels 35

and if we take g(x, y) = 1 B (κ(x, y)) in (2.1) then E(g(x, ϑ)) = P(x, B). Thus

P(ξ1 ∈ B|ξ0 = x) = P(x, B),

which gives the conditional distribution of ξ1 given ξ0 . Moreover, if μξ0 is the dis-
tribution of ξ0 then the joint distribution μ(ξ0 ,ξ1 ) of ξ0 and ξ1 satisfies

μ(ξ0 ,ξ1 ) (B0 × B1 ) = P(ξ0 ∈ B0 , ξ1 ∈ B1 ) = P(x, B1 )μξ0 (d x)
B0

for all B0 , B1 ∈ Σ.
We need an extension of the concept of transition probabilities. Let (X 0 , Σ0 ) and
(X 1 , Σ1 ) be two measurable spaces. A function P : X 0 × Σ1 → [0, 1] is said to be
a transition probability from (X 0 , Σ0 ) to (X 1 , Σ1 ) if P(·, B) is measurable for every
B ∈ Σ1 and P(x, ·) is a probability measure on (X 1 , Σ1 ) for every x ∈ X 0 . The next
result ensures existence of probability measures on product spaces.

Proposition 2.1 Let P be a transition probability from (X 0 , Σ0 ) to (X 1 , Σ1 ). If μ0

is a probability measure on (X 0 , Σ0 ) then there exists a unique probability measure
μ on the product space (X 0 × X 1 , Σ0 ⊗ Σ1 ) such that

μ(B0 × B1 ) = P(x0 , B1 ) μ0 (d x0 ), B0 ∈ Σ0 , B1 ∈ Σ1 .
B0

Proof Let B x0 = {x1 ∈ X 1 : (x0 , x1 ) ∈ B} for every x0 ∈ X 0 and B ∈ Σ0 ⊗ Σ1 .

Clearly, if B = B0 × B1 then we have B x0 = B1 for x0 ∈ B0 and B x0 = ∅ for x0 ∈
/ B0 .
By the monotone class theorem (see Theorem A.1) for each B ∈ Σ0 ⊗ Σ1 the func-
tion x0 → P(x0 , B x0 ) is measurable. Define

μ(B) = P(x0 , B x0 ) μ0 (d x0 ), B ∈ Σ0 ⊗ Σ1 ,
X0

which is clearly a measure on the product space. Since the product σ -algebra is gen-
erated by rectangles, i.e. Σ0 ⊗ Σ1 = σ (C ), where C = {B0 × B1 : B0 ∈ Σ0 , B1 ∈
Σ1 }, and C is a π -system, the result follows from the π − λ lemma.

We now show that, for a large class of measurable spaces, any transition probability
is as in (2.2). We call a measurable space (X, Σ) a Borel space if X is isomorphic to
a Borel subset of [0, 1], i.e. there exists a Borel subset Y of [0, 1] and a measurable
bijection ψ : X → Y such that its inverse ψ −1 is also measurable. Any Borel subset
of a Polish space, i.e. a complete separable metric space, with Σ = B(X ) is a Borel
space (see e.g. [55, Theorem A1.2]). Recall that a metric space (X, ρ) is complete
if every sequence (xn ) satisfying the Cauchy condition limm,n→∞ ρ(xn , xm ) = 0 is
convergent to some x ∈ X : limn→∞ ρ(xn , x) = 0, and it is separable if there is a
countable set X 0 ⊂ X which is dense in X , so that for any x ∈ X there exists a
sequence from X 0 converging to x.
36 2 Markov Processes

Proposition 2.2 Let P be a transition probability from a measurable space (X 0 , Σ0 )

to a Borel space (X 1 , Σ1 ). Then there exists a measurable function κ : X 0 × [0, 1] →
X 1 such that if a random variable ϑ is uniformly distributed on the unit interval [0, 1]
then κ(x, ϑ) has distribution P(x, ·) for every x ∈ X 0 .

Proof Since ψ : X 1 → ψ(X 1 ) with ψ(X 1 ) ⊂ [0, 1] is a bijection and we have

ψ −1 ([0, t]) = ψ −1 ([0, t] ∩ ψ(X 1 )), we can define

Fx (t) = P(x, ψ −1 ([0, t])), t ∈ (0, 1].

For each x the function t → Fx (t) is right-continuous with Fx (1) = 1. Consider its
generalized inverse

Fx← (q) := inf{t ≥ 0 : Fx (t) ≥ q}, q ≥ 0,

with the convention that the infimum of an empty set is equal to +∞, and define
κ(x, q) = ψ −1 (Fx← (q)) if Fx← (q) ∈ ψ(X 1 ) and κ(x, q) = x1 if Fx← (q) ∈
/ ψ(X 1 ),
where x1 is an arbitrary point from X 1 . We have

Leb{q ∈ [0, 1] : κ(x, q) ∈ B} = P(x, B) for x ∈ X 0 , B ∈ Σ1 ,

where Leb denotes the Lebesgue measure on [0, 1].

2.1.2 Transition Operators

It is convenient to associate with each transition kernel two linear mappings acting in
two Banach spaces, one which is a space of bounded functions and the other which
is a space of measures.
Let P be a transition kernel on a measurable space (X, Σ). Consider first the
Banach space B(X ) of bounded, measurable, real-valued functions on X with the
supremum norm
g u = sup |g(x)|.
x∈X

Given g ∈ B(X ) we define


T g(x) = g(y)P(x, dy), x ∈ X.
X

If B ∈ Σ then the indicator function 1 B ∈ B(X ) and

T 1 B (x) = P(x, B), x ∈ X.

2.1 Transition Probabilities and Kernels 37

Thus T 1 B ∈ B(X ). By the monotone class theorem, we obtain T g ∈ B(X ) for g ∈

B(X ). From linearity of the integral it follows that the mapping B(X )  g → T g ∈
B(X ) is a linear operator and that

Tg u ≤ g u , g ∈ B(X ).

Now, let M + (X ) be the collection of all finite measures on Σ. For each μ ∈

M + (X ) we define

Pμ(B) = P(x, B) μ(d x), B ∈ Σ.
X

Then Pμ is a finite measure and Pμ(X ) ≤ μ(X ). The space M + (X ) is a cone, i.e.
c1 μ1 + c2 μ2 ∈ M + (X ), if c1 , c2 are non-negative constants and μ1 , μ2 ∈ M + (X ),
and we have
P(c1 μ1 + c2 μ2 ) = c1 Pμ1 + c2 Pμ2 .

Therefore we can extend P so that it is a linear operator on the vector space of all
finite signed measures

M (X ) = {μ1 − μ2 : μ1 , μ2 ∈ M + (X )},

by setting Pμ = Pμ1 − Pμ2 for μ = μ1 − μ2 ∈ M (X ). Define the total variation

norm on M (X ) by
μ T V = |μ|(X ), μ ∈ M (X ),

where the total variation of a finite signed measure μ is given by


|μ|(X ) = sup μ(B) − inf μ(B) = sup{| gdμ| : g ∈ B(X ), g u ≤ 1}.
B∈Σ B∈Σ

Then (M (X ), · TV) is a Banach space and we have

Pμ TV ≤ μ TV, μ ∈ M (X ).

In particular, a transition probability P provides the following stochastic inter-

pretation. If X -valued random variables ξ0 and ξ1 are such that

P(ξ1 ∈ B|ξ0 = x) = P(x, B)

and ξ0 has a distribution μ0 , then μ1 = Pμ0 is the distribution of ξ1 and


E(g(ξ1 )|ξ0 = x) = g(y)P(x, dy) = T g(x), x ∈ X, g ∈ B(X ).
X
38 2 Markov Processes

2.1.3 Substochastic and Stochastic Operators

In this section, we suppose that a measurable space (X, Σ) carries a σ -finite measure
m. The set of all measurable and m-integrable functions is a linear space and it
becomes a Banach space, written L 1 (X, Σ, m), by defining the norm

f = | f (x)| m(d x), f ∈ L 1 (X, Σ, m),
X

and by identifying functions that are equal almost everywhere. Given a measurable
function f : X → R the essential supremum of f is defined by

ess sup| f | = inf{c > 0 : m{x : | f (x)| > c} = 0} =: f ∞.

The set of all functions with a finite essential supremum is denoted by L ∞ (X, Σ, m)
and it becomes a Banach space when we identify functions that are equal almost
everywhere and take · ∞ as the norm.
A linear operator P : L 1 (X, Σ, m) → L 1 (X, Σ, m) is called substochastic if it
is positive, i.e. if f ≥ 0 then P f ≥ 0 for f ∈ L 1 (X, Σ, m), and if P f ≤ f for
f ∈ L 1 (X, Σ, m). A substochastic operator is called stochastic if P f = f for
f ∈ L 1 (X, Σ, m) and f ≥ 0. We denote by P ∗ : L ∞ (X, Σ, m) → L ∞ (X, Σ, m)
the adjoint operator of P, i.e. for every f ∈ L 1 (X, Σ, m) and g ∈ L ∞ (X, Σ, m)
 
P f (x)g(x) m(d x) = f (x)P ∗ g(x) m(d x).
X X

Let P be a transition kernel on (X, Σ). Take f ∈ L 1 (X, Σ, m) with f ≥ 0 and

f = 1. The measure μ ∈ M + (X ) given by dμ = f dm, i.e.

μ(B) = f (x) m(d x), B ∈ Σ,
B

is absolutely continuous with respect to m and we call f the density of μ (with

respect to m). If the measure ν = Pμ given by

ν(B) = Pμ(B) = P(x, B) f (x) m(d x), B ∈ Σ,
X

is absolutely continuous with respect to m then, by the Radon–Nikodym theorem,

there is an essentially unique g ∈ L 1 , g ≥ 0, such that

ν(B) = g(x) m(d x), B ∈ Σ.
B
2.1 Transition Probabilities and Kernels 39

The function g is the Radon–Nikodym derivative dν/dm of the measure ν with

respect to m. If P is a transition probability then we have Pμ(X ) = 1. Thus f is
mapped to g, and we can write

dμ
P f = g, if f = ,
dm
which is the density of the measure ν = Pμ

Pμ(B) = P f (x) m(d x).
B

This motivates the following definitions.

Denote by D the subset of L 1 = L 1 (X, Σ, m) which contains all densities, i.e.

D = { f ∈ L 1 : f ≥ 0, f = 1}.

Observe that a linear operator P on L 1 (X, Σ, m) is a stochastic operator if and only

if P(D) ⊆ D. The transition kernel P corresponds to a substochastic operator P if
 
P(x, B) f (x) m(d x) = P f (x) m(d x) for all B ∈ Σ, f ∈ D,
X B

or, equivalently, the adjoint of P is of the form


∗
P g(x) = g(y) P(x, dy) for g ∈ L ∞ (X, Σ, m). (2.3)

On the other hand, if a transition kernel P has the following property

m(B) = 0 =⇒ P(x, B) = 0 for m-a.e. x and B ∈ Σ, (2.4)

then there exists a substochastic operator P such that (2.3) holds.

Remark 2.1 There exists a stochastic operator which does not have a transition ker-
nel [32]. But if X is a Polish space (a complete separable metric space) and Σ is a
σ -algebra of Borel subsets of X , then each substochastic operator on L 1 (X, Σ, m)
has a transition kernel [48]. The same result holds if X is a Borel subset of a Polish
space and Σ is the σ -algebra of Borel subsets of X .

2.1.4 Integral Stochastic Operators

Let (X, Σ, m) be a σ -finite measure space. Any measurable function k : X × X →

[0, ∞) such that
40 2 Markov Processes

k(x, y) m(d x) = 1 for all y ∈ X
X

defines a transition probability by


P(x, B) = k(y, x) m(dy), B ∈ Σ,
B

called an integral kernel. If we set


P f (x) = k(x, y) f (y) m(dy), f ∈ L 1 (X, Σ, m),
X

then for f ≥ 0 we have P f ≥ 0 and

   
P f (x) m(d x) = k(x, y) m(d x) f (y) m(dy) = f (y) m(dy),
X X X X

which shows that P is a stochastic operator. If, instead, k is such that


k(x, y) m(d x) ≤ 1 for all y ∈ X,
X

then P(x, B) is a transition kernel. It defines a substochastic operator on L 1 (X, Σ, m),

called an integral operator.
Suppose that the state space is discrete, i.e. the set X is finite or countable and
Σ is the family of all subsets of X . Since any probability measure on Σ is uniquely
defined by its values on the singleton sets {y}, y ∈ X , any transition probability P
satisfies 
P(x, B) = P(x, {y}), x ∈ X, B ⊆ X.
y∈B

If the state space is discrete and the measure m is the counting measure then (X, Σ, m)
is a σ -finite measure space and every kernel is an integral kernel.
We now consider the case of X = N = {0, 1, . . .}, where we use the notation l 1 =
L (X, Σ, m) with m being the counting measure on X . We represent
1
∞ any function
f as a sequence u = (u i )i∈N . We have (u i )i∈N ∈ l 1 if and only if i=0 |u i | < ∞. If

∞

ki j = 1 for all j,
i=0

then the operator P : l 1 → l 1 defined by

∞

(Pu)i = ki j u j , i ∈ N,
j=0
2.1 Transition Probabilities and Kernels 41

is a stochastic operator. It can be identified with a matrix P = [ki j ]i, j∈N , called a
stochastic matrix. It has non-negative elements and the elements in each column
sum up to one. The transposed matrix

P ∗ = [k ji ]i, j∈N , k ji = P(i, { j}),

is called a transition matrix and it is such that the elements in each row sum up to
one. Note that k ji is the probability of going from state i to state j.

2.1.5 Frobenius–Perron Operator

Consider a measurable transformation S : X → X , where (X, Σ, m) is a space with

a σ -finite measure m. Let μ be a probability measure on (X, Σ) and let us observe the
evolution of this measure under the action of the system. For example, if we start with
the probability measure concentrated at the point x, i.e. the Dirac measure δx , then
under the action of the system we obtain the measure δ S(x) . In general, if a measure μ
describes the distribution of points in the phase space X , then the measure ν given by
the formula ν(B) = μ(S −1 (B)) describes the distribution of points after the action
of the transformation S. Assume that μ is absolutely continuous with respect to m
with density f . If the measure ν is also absolutely continuous with respect to m, and
g = dν/dm, then we define an operator PS by PS f = g. This operator corresponds
to the transition probability function P(x, B) on (X, Σ) given by

1, if S(x) ∈ B,
P(x, B) = (2.5)
0, if S(x) ∈
/ B.

The operator PS is correctly defined if the transition probability function P(x, B)

satisfies condition (2.4). Condition (2.4) now takes the form

m(B) = 0 =⇒ m(S −1 (B)) = 0 for B ∈ Σ (2.6)

and the transformation S which satisfies (2.6) is called non-singular. This operator
can be extended to a bounded linear operator PS : L 1 → L 1 , and PS is a stochastic
operator. The operator PS is called the Frobenius–Perron operator for the transfor-
mation S or the transfer operator or the Ruelle operator.
We now give a formal definition of the Frobenius–Perron operator. Let (X, Σ, m)
be a σ -finite measure space and let S be a measurable non-singular transformation
of X . An operator PS : L 1 → L 1 which satisfies the following condition
 
PS f (x) m(d x) = f (x) m(d x) for B ∈ Σ and f ∈ L 1 (2.7)
B S −1 (B)
42 2 Markov Processes

is the Frobenius–Perron operator for the transformation S. The adjoint of the

Frobenius–Perron operator PS∗ : L ∞ → L ∞ is given by PS∗ g(x) = g(S(x)) and
is called the Koopman operator or the composition operator. In particular, if
S : X → X is one to one and non-singular with respect to m, then

d(m ◦ S −1 )
PS f (x) = 1 S(X ) (x) f (S −1 (x)) (x) for m-a.e. x ∈ X,
dm

where d(m ◦ S −1 )/dm is the Radon–Nikodym derivative of the measure m ◦ S −1

with respect to m.
We next show how to find the Frobenius–Perron operator for piecewise smooth
transformations of subsets of Rd . Let X be a subset of Rd with non-empty interior
and with the boundary of zero Lebesgue measure. Let S : X → X be a measurable
transformation. We assume that there exist pairwise disjoint open subsets U1 ,…,Un
of X having the following properties:
n
(a) the sets X 0 = X \ i=1 Ui and S(X 0 ) have zero Lebesgue measure,

(b) maps Si = S  are diffeomorphisms from Ui onto S(Ui ), i.e. Si are C 1 and
Ui
invertible transformations with det Si (x) = 0 at each point x ∈ Ui .
Then transformations ψi = Si−1 are also diffeomorphisms from S(Ui ) onto Ui and
the Frobenius–Perron operator PS exists and is given by the formula

PS f (x) = f (ψi (x))| det ψi (x)|, (2.8)
i∈I x

where Ix = {i : ψi (x) ∈ Ui }. Indeed,

 n 
 n 

f (x) d x = f (x) d x = f (x) d x
S −1 (B) i=1 S
−1 (B)∩ U
i i=1 ψi (B)

n 
= f (ψi (x))| det ψi (x)| d x
i=1 B∩ S(Ui )
  
= f (ψi (x))| det ψi (x)| d x = P f (x) d x.
B i∈I B
x

2.1.6 Iterated Function Systems

We now consider a finite set of different non-singular transformations S1 , . . . , Sk of a

σ -finite measure space (X, Σ, m). Let p1 (x), . . . , pk (x) be non-negative measurable
functions defined on X such that p1 (x) + · · · + pk (x) = 1 for all x ∈ X . Take a point
x. We choose a transformation S j with probability p j (x) and the position of x after
2.1 Transition Probabilities and Kernels 43

the action of the system is given by S j (x). Thus we consider the following transition
probability
 k
P(x, B) = p j (x)δ S j (x) (B)
j=1

for x ∈ X and measurable sets B. Hence, for any measure μ, we have

k 
 k 

Pμ(B) = p j (x)δ S j (x) (B)μ(d x) = p j (x)μ(d x).
j=1 X j=1 S −1
j (B)

If μ is absolutely continuous and f = dμ/dm then

k 
 k 

Pμ(B) = p j (x) f (x) m(d x) = PS j ( p j f )(x) m(d x),
j=1 S −1
j (B) j=1 B

where PS1 , . . . , PSk are the corresponding Frobenius–Perron operators. Consequently,

the stochastic operator on L 1 = L 1 (X, Σ, m) corresponding to P is of the form


k
Pf = PS j ( p j f ), f ∈ L 1.
j=1

We can extend the iterated function system in the following way. Consider a family
of measurable transformations S y : X → X , y ∈ Y , where Y is a metric space which
carries a Borel measure ν, and a family of measurable functions p y : X → [0, ∞),
y ∈ Y , satisfying 
p y (x)ν(dy) = 1, x ∈ X,
Y

so that the transition probability P is of the form


P(x, B) = 1 B (S y (x)) p y (x)ν(dy), x ∈ X. (2.9)
Y

If each S y is a non-singular transformation of the space (X, Σ, m) then the stochastic

operator P corresponding to the transition probability as in (2.9) is of the form

Pf = PSy ( p y f )ν(dy), f ∈ L 1,
Y

where PSy is the Frobenius–Perron operator for S y , y ∈ Y .

As a final example, we look at the transition probability describing the switching
mechanism in Sects. 1.8 and 1.9. Let I be at most a countable set. We define the
transformation S j : X × I → X × I , j ∈ I , by
44 2 Markov Processes

S j (x, i) = (x, j), x ∈ X, i, j ∈ I.

Each transformation defined on X = X × I is non-singular with respect to the prod-

uct measure m of a measure on X and the counting measure on I . We assume that
q ji (x), j = i, are non-negative measurable functions satisfying j=i q ji (x) < ∞
for all i ∈ I , x ∈ X . Then we define the jump rate function by

qi (x) = q ji (x)
j=i

and the probabilities p j , j ∈ I , by pi (x, i) = 0 and


1, qi (x) = 0, j = i,
p j (x, i) = q ji (x)
qi (x)
, qi (x) > 0, j = i.

The stochastic operator P on L 1 = L 1 (X, Σ, m) corresponding to the transition

probability
P((x, i), {(x, j)}) = p j (x, i)

is given by 
P f (x, i) = pi (x, j) f (x, j).
j=i

In particular, if I = {0, 1} and qi (x) > 0 for x ∈ X , i = 0, 1, then


0, j = i,
p j (x, i) =
1, j = 1 − i,

and P f (x, i) = PS1−i (x, i) = f (x, 1 − i) for (x, i) ∈ X = X × {0, 1}.

2.2 Discrete-Time Markov Processes

2.2.1 Markov Processes and Transition Probabilities

A family ξn , n ∈ N, of X -valued random variables defined on a probability space

(Ω, F , P) is called a discrete-time stochastic process with state space (X, Σ). It is
said to have the (weak) Markov property if for each time n ≥ 0

P(ξn+1 ∈ B|ξ0 , . . . , ξn ) = P(ξn+1 ∈ B|ξn ), B ∈ Σ. (2.10)

2.2 Discrete-Time Markov Processes 45

ξ
The sequence Fn = σ (ξ0 , . . . , ξn ), n ≥ 0, is called the natural filtration or the his-
tory of the process (ξn ). We have

σ (ξ0 , . . . , ξn ) = {{ξ0 ∈ B0 , . . . , ξn ∈ Bn } : B0 , . . . , Bn ∈ Σ}.

The monotone class theorem and properties of the conditional expectation imply that
if (ξn ) has the weak Markov property then for each k ≥ 0 and all g0 , . . . , gk ∈ B(X )
the following holds

E(g0 (ξn ) . . . gk (ξn+k )|ξ0 , . . . , ξn ) = E(g0 (ξn ) . . . gk (ξn+k )|ξn ).

We can extend it further as follows. A sequence (ξn ) has the weak Markov property
if and only if the future σ (ξm : m ≥ n) and the past σ (ξm : m ≤ n) are conditionally
independent given the present σ (ξn ) (see Lemma A.4), i.e. for each n and all A ∈
σ (ξm : m ≥ n) and F ∈ σ (ξm : m ≤ n) we have

P(A ∩ F|ξn ) = P(A|ξn )P(F|ξn ).

A discrete-time stochastic process ξ = (ξn )n≥0 is called a (homogeneous) Markov

process with transition probability P and initial distribution μ on (X, Σ) if there is
a probability space (Ω, F , P) such that

μ(B) = P(ξ0 ∈ B) and P(ξn+1 ∈ B|ξ0 , . . . , ξn ) = P(ξn , B) (2.11)

for all B ∈ Σ and n ≥ 0. Taking the conditional expectation of (2.11) with respect
to σ (ξn ) we obtain
P(ξn+1 ∈ B|ξn ) = P(ξn , B)

which implies that ξ = (ξn )n≥0 has the weak Markov property. Moreover, we have

P(ξn+k ∈ B|ξ0 , . . . , ξn ) = P(ξn+k ∈ B|ξn ) = P k (ξn , B), k ≥ 1,

where P k , called the kth step transition probability, is defined inductively by


P 0 (x, B) = δx (B), P k+1 (x, B) = P(y, B) P k (x, dy).
X

The kernels P k , k ≥ 1, satisfy the Chapman–Kolmogorov equation


P k+n
(x, B) = P k (y, B) P n (x, dy). (2.12)
X

We can regard the stochastic process ξn , n ≥ 0, as an X N -valued random variable,

where X N is the product space
46 2 Markov Processes

X N = {x = (x0 , x1 , . . .) : xn ∈ X, n ≥ 0}

with product σ -algebra Σ N = σ (C ) which is the smallest σ -algebra of subsets of

X N containing the family C of all finite-dimensional rectangles

C = {{x ∈ X N : x0 ∈ B0 , . . . , xn ∈ Bn } : B0 , . . . , Bn ∈ Σ, n ≥ 0}. (2.13)

To see this let for each ω the mapping n → ξn (ω) denotes the sequence ξ(ω) =
(ξn (ω))n≥0 ∈ X N . Thus we obtain the mapping ξ : Ω → X N and ξ is measurable
if and only if ξn : Ω → X is measurable for each n ∈ N. The distribution of the
process ξ is a probability measure μξ on (X N , Σ N ) and it is uniquely defined through
its values on finite-dimensional rectangles. In other words, the finite-dimensional
distributions μ0,1,...,n = μ(ξ0 ,...,ξn )

μ0,1,...,n (B0 × · · · × Bn ) = P(ξ0 ∈ B0 , . . . , ξn ∈ Bn ), B0 , . . . , Bn ∈ Σ, n ≥ 0,

uniquely determine the law of the process, since

μξ (B0 × · · · × Bn × X N ) = P(ξ ∈ B0 × · · · × Bn × X N )
= μ(ξ0 ,...,ξn ) (B0 × · · · × Bn )

and all finite-dimensional rectangles B0 × · · · × Bn × X N generate Σ N . A transition

probability and an initial distribution determine the finite-dimensional distributions
of a discrete-time Markov process, as stated in the next theorem. Its proof is based
on the monotone class theorem and properties of conditional expectations.

Theorem 2.1 Let μ be a probability measure, P be a transition probability, and

(ξn )n≥0 be defined on a probability space (Ω, F , P). Then (ξn )n≥0 is a Markov
process with transition probability P and initial distribution μ if and only if

P(ξ0 ∈ B0 , ξ1 ∈ B1 , . . . , ξn ∈ Bn )
  
= ... P(xn−1 , Bn )P(xn−2 , d xn−1 ) . . . P(x0 , d x1 ) μ(d x0 )
B0 B1 Bn−1
(2.14)

for all sets B0 , B1 , . . . , Bn ∈ Σ, n ≥ 0.

2.2.2 Random Mapping Representations

Suppose that (ϑn )n≥1 is a sequence of i.i.d. random variables with values in a metric
space Y . Let ξ0 be an X -valued random variable independent of (ϑn )n≥1 and with dis-
tribution μ. Consider a measurable function κ : X × Y → X and define a sequence
2.2 Discrete-Time Markov Processes 47

ξn of X -valued random variables by

ξn = κ(ξn−1 , ϑn ), n ≥ 1. (2.15)

For Fn = σ (ξ0 , . . . , ξn ) we have Fn ⊆ σ (ξ0 , ϑ1 , . . . , ϑn ) and ϑn+1 is independent

of Fn and of ϑn . Thus, we see that (ξn ) has the weak Markov property, by (2.1).
Hence, ξ is Markov with transition probability of the form

P(x, B) = P(κ(x, ϑ1 ) ∈ B) = 1 B (κ(x, y))ν(dy)
Y

and initial distribution μ. A particular example is the random walk on Rd defined by

ξn = ξn−1 + ϑn , n ≥ 1.

From Proposition 2.2 it follows that a typical discrete-time Markov process can
be defined by a recursive formula as in (2.15).
Theorem 2.2 Let P be a transition probability on a Borel space (X, Σ). Then there
exists a measurable function κ : X × [0, 1] → X such that for any sequence (ϑn ) of
independent random variables with uniform distribution on [0, 1] and for any x ∈ X
the sequence defined by (2.15) is a discrete-time Markov process with transition
probability P and initial distribution δx .
To define a discrete-time Markov process with transition probability P on a Borel
space (X, Σ), we can take Ω, by Theorem 2.2, to be the countable product [0, 1]N of
the unit interval [0, 1] with the product σ -algebra F = B([0, 1])N and the measure P
as a countable product of the Lebesgue measure on [0, 1]. Then we define a sequence
of i.i.d. random variables ϑn : Ω → [0, 1], n ≥ 0, by ϑn (ω) = ωn for ω = (ωn )n≥0 ∈
Ω, and ξn by (2.15).

2.2.3 Canonical Processes

We next construct a canonical discrete-time Markov process with a given transi-

tion probability and an initial distribution on an arbitrary measurable space (X, Σ).
The existence of the process with given finite-dimensional distributions amounts to
showing that there is a probability measure on the product space (X N , Σ N ) satisfying
(2.14) with ξ being the identity mapping. This can be realized through the discrete-
time version of the Kolmogorov extension theorem. For its proof we refer the reader
to [55]. Note that no regularity condition is needed on the state space.

Theorem 2.3 (Ionescu–Tulcea) Let P be a transition probability on a measurable

space (X, Σ). For every x ∈ X there exists a unique probability measure Px on the
product space (X N , Σ N ) such that
48 2 Markov Processes

Px (B0 × · · · × Bn × X N )
 
= 1 B0 (x) ... P(xn−1 , Bn )P(xn−2 , d xn−1 ) . . . P(x, d x1 )
B1 Bn−1

for all B0 , . . . , Bn ∈ Σ, n ≥ 0. Moreover, for every set A ∈ Σ N , the map x → Px (A)

is Σ-measurable.

The function Px (A) is a transition probability from (X, Σ) to (X N , Σ N ). Thus,

by Proposition 2.1, for each probability measure μ on (X, Σ) there exists a unique
probability measure Pμ on (X × X N , Σ ⊗ Σ N ) such that

Pμ (B × A) = Px (A) μ(d x), B ∈ Σ, A ∈ Σ N .
B

The sequence space

Ω = X N = {ω = (x0 , x1 , . . .) : xn ∈ X, n ≥ 0}

is called a canonical space and the coordinate mappings ξn (ω) = xn , n ≥ 0, define a

process ξ = (ξn )n≥0 on (Ω, F ) = (X N , Σ N ) with distribution Pμ , called a Markov
canonical process. Note that if we let the process start at x ∈ X so that μ = δx then
we have Pδx = Px and if A ∈ Σ N is the rectangle

A = {ω : ξ0 (ω) ∈ B0 , . . . , ξn (ω) ∈ Bn }

then we have

Pμ (A) = Pμ (ξ0 ∈ B0 , ξ1 ∈ B1 , . . . , ξn ∈ Bn )
  
= ... P(xn−1 , Bn )P(xn−2 , d xn−1 ) . . . P(x0 , d x1 )μ(d x0 ).
B0 B1 Bn−1

Consequently, for each probability measure Pμ on (Ω, F ) the process ξ = (ξn )n≥0
is homogeneous Markov with transition probability P and initial distribution μ. We
write Ex for the integration with respect to Px , x ∈ X , and we have

Eμ (η) = Ex (η) μ(d x)
X

for any bounded or non-negative random variable η : Ω → R. In particular, we can

rewrite the Markov property as: for each n and any bounded measurable g defined
on the sequence space we have

Eμ (g(ξn , ξn+1 , . . .)|Fn ) = h(ξn ), where h(x) = Ex (g(ξ0 , ξ1 , . . .)).

2.3 Continuous-Time Markov Processes 49

2.3 Continuous-Time Markov Processes

2.3.1 Basic Definitions

A family ξ(t), t ∈ [0, ∞), of X -valued random variables is called a continuous-

time stochastic process with state space X , where (X, Σ) is a measurable space.
For each t ≥ 0, let Ft = σ (ξ(r ) : r ≤ t) be the σ -algebra generated by all random
variables ξ(r ), 0 ≤ r ≤ t. Since Fs ⊆ Ft for all 0 ≤ s < t, the collection Ft , t ≥ 0,
is called a history of the process or a filtration. If X is a Borel space then the
process ξ = {ξ(t) : t ≥ 0} is said to be right-continuous (càdlàg) if its sample paths
t → ξ(t)(ω), ω ∈ Ω, are right-continuous (càdlàg).
Let ξ = {ξ(t) : t ≥ 0} be an X -valued process defined on a probability space
(Ω, F , P). The process ξ is said to be a Markov process if for any times s, t we
have
P(ξ(s + t) ∈ B|Fs ) = P(ξ(s + t) ∈ B|ξ(s)), B ∈ Σ. (2.16)

By the monotone class theorem the Markov property (2.16) holds if and only if

E(g(ξ(s + t))|Fs ) = E(g(ξ(s + t))|ξ(s))

for all g ∈ B(X ), where B(X ) is the space of bounded and measurable functions
g : X → R. Note that a process is Markov if and only if for any 0 ≤ s1 < · · · < sn
and g ∈ B(X )

E(g(ξ(sn + t))|ξ(s1 ), . . . , ξ(sn )) = E(g(ξ(sn + t))|ξ(sn )).

A family P = {P(t, ·) : t ≥ 0} of transition probabilities on (X, Σ) is said to be

a transition (probability) function if it satisfies the Chapman–Kolmogorov equation

P(s + t, x, B) = P(s, y, B)P(t, x, dy), s, t ≥ 0, (2.17)
X

and P(0, x, B) = δx (B) for every x ∈ X , B ∈ Σ. We say that ξ is a homogeneous

Markov process with transition function P and starting at x if

P(ξ(0) = x) = 1 and P(ξ(s + t) ∈ B|Fs ) = P(t, ξ(s), B) (2.18)

for all B ∈ Σ and for all times s and t. Equivalently,


E(g(ξ(s + t))|Fs ) = g(y)P(t, ξ(s), dy)
X

for all g ∈ B(X ), s, t ≥ 0. We interpret P(t, x, B) as the probability that the sto-
chastic process ξ moves from state x at time 0 to a state in B at time t. The transition
50 2 Markov Processes

function P defines a family of bounded linear operators


T (t)g(x) = g(y)P(t, x, dy)
X

on the Banach space B(X ) with supremum norm · u . It follows from the Chapman–
Kolmogorov equation (2.17) that T (t), t ≥ 0, is a semigroup, i.e.

T (s + t)g = T (t)(T (s)g), g ∈ B(X ), s, t ≥ 0.

The family {T (t)}t≥0 is called the transition semigroup associated to ξ .

A random variable τ : Ω → [0, ∞] is called a stopping time for the filtration (Ft )
if it satisfies {τ ≤ t} ∈ Ft for all t ≥ 0. The σ -algebra Fτ giving the history known
up to time τ is defined as

Fτ = {A ∈ F : A ∩ {τ ≤ t} ∈ Ft for all t ≥ 0}.

If τ1 and τ2 are two stopping times and τ1 ≤ τ2 then Fτ1 ⊆ Fτ2 . A Markov process
ξ is said to be strong Markov if for any stopping time τ , the strong Markov property
at τ holds

P(ξ(τ + t) ∈ B|Fτ ) = P(ξ(τ + t) ∈ B|ξ(τ )) a.s. on τ < ∞ (2.19)

for all B ∈ Σ and all times t. If X is a Borel space and the process has right-continuous
sample paths then ξ(τ ) is Fτ -measurable on {τ < ∞} for any stopping time τ for
the filtration (Ft ).
As in the discrete-time case we can consider a canonical space Ω = X R+ , which
now is the space of all functions ω : R+ → X , with product σ -algebra Σ R+ = σ (C )
which is the smallest σ -algebra of subsets of X R+ containing the family C of all
cylinder sets:

{ω ∈ X R+ : ω(t0 ) ∈ B0 , . . . , ω(tn ) ∈ Bn }, B0 , . . . , Bn ∈ Σ, 0 ≤ t0 < · · · < tn , n ≥ 0.

Let Ft = σ (ξ(r ) : r ≤ t), t ≥ 0, where the canonical process ξ(t), t ≥ 0, is defined

as the identity map on (X R+ , Σ R+ ). Suppose that P is a transition probability func-
tion on a Borel space (X, Σ). It follows from a continuous-time version of the
Kolmogorov extension theorem [55] that for every x ∈ X there exists a probability
measure Px on (Ω, F ) such that ξ(t), t ≥ 0, is a homogeneous Markov process with
transition function P and starting at x. Moreover, for every set A ∈ Σ N , the map
x → Px (A) is Σ-measurable. We have

Px (ξ(t) ∈ B) = P(t, x, B), t ≥ 0, B ∈ Σ,

and
2.3 Continuous-Time Markov Processes 51

Px (ξ(t1 ) ∈ B1 , . . . , ξ(tn ) ∈ Bn )
 
= ... P(tn − tn−1 , xn−1 , Bn )P(tn−1 − tn−2 , xn−2 , d xn−1 ) . . . P(t1 , x, d x1 )
B1 Bn−1

for all 0 < t1 < · · · < tn , B1 , . . . , Bn ∈ Σ, n ∈ N.

Given a transition function on a Borel space, for each x ∈ X , there exists a
probability space (Ω, F , Px ) and a homogeneous Markov process {ξ(t) : t ≥ 0}
defined on (Ω, F , Px ) with transition function P and starting at x. We will
also denote the process ξ(t) started at x by ξx (t). Then the family of processes
ξ = {ξx (t) : t ≥ 0, x ∈ X } is called a Markov family. We will simply say that ξ(t)
is a Markov process with state space X defined on (Ω, F , Px ). However, in general,
a transition function is unknown in advance and we need to construct the processes
directly from other processes.

2.3.2 Processes with Stationary and Independent Increments

Our first example is a continuous time extension of random walks. An Rd -valued

process ξ(t), t ≥ 0, is said to have independent increments, if ξ(s + t) − ξ(s) is
independent of Fs = σ (ξ(r ) : r ≤ s) for all t, s ≥ 0. Given such a process, we have
by (2.1)

P(ξ(s + t) ∈ B|Fs ) = P(ξ(s + t) − ξ(s) + ξ(s) ∈ B|ξ(s)) = P(ξ(s + t) ∈ B|ξ(s))

for B ∈ B(Rd ), since ξ(s + t) = ξ(s + t) − ξ(s) + ξ(s) and ξ(s + t) − ξ(s) is
independent of Fs , which shows that ξ has the Markov property.
The process ξ has stationary increments, if the distribution of ξ(s + t) − ξ(s) is
the same as the distribution of ξ(t) − ξ(0) for all s, t ≥ 0. In that case

E1 B (ξ(s + t) − ξ(s) + x) = E1 B (ξ(t) − ξ(0) + x)

for all x and s, t. In particular, we have ξ(s + t) − ξ(0) = ξ(s + t) − ξ(t) + ξ(t) −
ξ(0) and the random variables ξ(s + t) − ξ(t) and ξ(t) − ξ(0) are independent.
Hence, if μt is the distribution of ξ(t) − ξ(0), then μs+t is the convolution of μs and
μt , i.e. 
μs+t (B) = (μs ∗ μt )(B) = μs (B − x)μt (d x),

and 
P(t, x, B) = 1 B (x + y)μt (dy).
Rd

A process is called a Lévy process if it has stationary independent increments,

it starts at zero, i.e. ξ(0) = 0 a.s., and it is continuous in probability, i.e. for every
52 2 Markov Processes

ε > 0 we have
lim P(|ξ(t)| > ε) = 0.
t→0

2.3.3 Markov Jump-Type Processes

In this section, we provide a simple construction of pure jump-type processes

with bounded jump rate function. They were introduced in Sect. 1.3 and are par-
ticular examples of PDMPs defined in Sect. 1.19. Here we show that they are
Markov processes. Suppose that (X, Σ) is a measurable space, P is a transition
probability on X and that ϕ is a bounded non-negative measurable function. Set
λ = sup{ϕ(x) : x ∈ X } and define the transition probability P̄ by

P̄(x, B) = λ−1 (ϕ(x)P(x, B) + (λ − ϕ(x))δx (B)), x ∈ X, B ∈ Σ. (2.20)

Let (ξn )n≥0 be a discrete-time Markov process with transition probability P̄ and
(σn )n≥1 be a sequence of independent random variables, exponentially distributed
with mean λ−1 , and independent of the sequence (ξn )n≥0 . We set τ0 = 0 and we
define
n
ξ(t) = ξn−1 if τn−1 ≤ t < τn , where τn = σk , n ≥ 1.
k=1

It follows from the strong law of large numbers that τn → ∞, as n → ∞, a.s. The
sample paths of the process ξ are constant between consecutive τn and the random
variables σn are called holding times.
Let N (t) be the number of jump times τn in the time interval (0, t], i.e. we have
∞

N (t) = max{n ≥ 0 : τn ≤ t} = 1(0,t] (τn ).
n=0

Then N (t) = 0 if t < τ1 , and

N (t) = n ⇐⇒ τn ≤ t < τn+1 .

We show that N (t) is Poisson distributed with parameter λt, i.e.

(λt)n
P(N (t) = n) = e−λt , n ≥ 0.
n!

We have P(N (t) = 0) = P(τ1 > t) = e−λt and for n ≥ 1

P(N (t) = n) = P(τn ≤ t < τn+1 ) = P(τn+1 > t) − P(τn > t),
2.3 Continuous-Time Markov Processes 53

since {τn > t} ⊆ {τn+1 > t}. The random variable τn , being the sum of n indepen-
dent exponentially distributed random variables with parameter λ, has a gamma
distribution with density

λn x n−1 −λx
f τn (x) = e for x ≥ 0.
(n − 1)!

Hence,
 ∞
λn x n−1 −λx (λt)n
P(τn > t) = e d x = −e−λt + P(τn+1 > t).
t (n − 1)! n!

We can write ξ(t) = ξ N (t) for t ≥ 0. In particular, if we take X = N and the trivial
Markov chain ξn = n for all n, then we have ξ(t) = N (t) for all t ≥ 0 and N (t),
t ≥ 0, is a Poisson process with intensity λ > 0. If (ξn )n≥0 is a random walk then ξ(t)
is the compound Poisson process as in (1.3). The Poisson process N has stationary
independent increments, equivalently, for any n, t, s and A ∈ σ (N (r ) : r ≤ s)

P({N (t + s) − N (s) = n} ∩ A) = P(N (t) = n)P(A).

Consider again the general process ξ(t) = ξ N (t) for t ≥ 0. We denote by ξx (t)
the particular process ξ(t) starting at ξ(0) = ξ0 = x. We can easily calculate the
distribution of ξx (t). By independence of N (t) and (ξn ), we obtain
∞

P(ξx (t) ∈ B) = P(ξ(t) ∈ B|ξ0 = x) = P(N (t) = n)P(ξn ∈ B|ξ0 = x).
n=0

Since the random variable N (t) is Poisson distributed with parameter λt, we have
∞
 (λt)n n
P(t, x, B) = e−λt P̄ (x, B), (2.21)
n=0
n!

where P̄ n is the nth step transition probability.

We now check that condition (2.18) holds with Fs = σ (ξ(r ) : r ≤ s). We may
write

E(1 B (ξ(t + s))|Fs ) = E(1 B (ξ N (t+s) )|Fs ) = E(1 B (ξ N (t+s)−N (s)+N (s) )|Fs ),

which gives
∞

E(1 B (ξ(t + s))|Fs ) = E(1{N (t+s)−N (s)=n} 1 B (ξn+N (s) )|Fs ).
n=0
54 2 Markov Processes

Since N (t + s) − N (s) is independent of σ (N (r ) : r ≤ s) and σ (ξn : n ≥ 0), it is

also independent of Fs . It is Poisson distributed with parameter λt. Thus
∞

E(1 B (ξ(t + s))|Fs ) = P(N (t + s) − N (s) = n)E(1 B (ξn+N (s) )|Fs )
n=0
∞
(λt)n
= e−λt E(1 B (ξn+N (s) )|Fs ).
n=0
n!

The family

{A1 ∩ A2 ∩ {N (s) = m} : A1 ∈ σ (N (r ) ≤ s), A2 ∈ σ (ξk : k ≤ m), m ≥ 0}

generates the σ -algebra Fs . Thus, it is enough to show that

 
1 B (ξn+N (s) ) dP = P̄ n (ξ(s), B) dP.
A1 ∩A2 ∩{N (s)=m} A1 ∩A2 ∩{N (s)=m}

To this end, observe that ξ(s) = ξm on {N (s) = m} and A1 ∩ {N (s) = m} is inde-

pendent of A2 and ξm . This together with the Markov property for (ξn )n≥0 leads
to
 
P̄ n (ξ(s), B) dP = P(A1 ∩ {N (s) = m}) P̄ n (ξm , B) dP
A1 ∩A2 ∩{N (s)=m} A2

= P(A1 ∩ {N (s) = m}) 1 B (ξn+m ) dP.
A2

Now, making use of the independence of A1 ∩ {N (s) = m} and A2 ∩ {ξn+m ∈ B},

completes the proof of the Markov property.

2.3.4 Generators and Martingales

Assume that ξ(t) is a Markov process with state space X defined on (Ω, F , Px ). Let

T (t)g(x) = Ex g(ξ(t)) = g(y)P(t, x, dy), g ∈ B(X ), x ∈ X, t ≥ 0,
X

where Ex is the expectation with respect to Px . Consider the class D(L) of all bounded
and measurable functions g : X → R such that the limit

Ex (g(ξ(t))) − g(x)
lim
t↓0 t
2.3 Continuous-Time Markov Processes 55

exists uniformly for all x ∈ X . We denote the limit by Lg(x) and we call L the
infinitesimal generator of the Markov process ξ . In particular,

1
Lg(x) = lim (g(y) − g(x))P(t, x, dy)
t↓0 t X

for all x ∈ X , whenever g ∈ D(L).

For a pure jump-type process with bounded jump rate function ϕ as considered
in Sect. 2.3.3 and with transition function of the form (2.21) we have
 
−λt −λt
(g(y) − g(x))P(t, x, dy) = (e − 1)g(x) + e λt g(y) P̄(x, dy)
X X
∞
 
(λt)n
+ e−λt g(y) P̄ n (x, dy),
n=2
n! X

which shows that D(L) consists of all bounded measurable functions and that

Lg(x) = λ (g(y) − g(x)) P̄(x, dy), x ∈ X, g ∈ B(X ).
X

Using (2.20) we conclude that the infinitesimal generator of this process is of the
form 
Lg(x) = ϕ(x) (g(y) − g(x))P(x, dy), x ∈ X, g ∈ B(X ).
X

The significance of the infinitesimal generator (L , D(L)) is related to the Dynkin

formula, which states that if g ∈ D(L) then the process
 t
η(t) = g(ξ(t)) − g(ξ(0)) − Lg(ξ(r )) dr
0

is a martingale, i.e. the random variable η(t) is integrable, Ft -measurable for each
t ≥ 0, and
E(η(t + s)|Fs ) = η(s)

for all t, s ≥ 0. In particular, the Dynkin formula holds if ξ(t) is a Markov process
with a metric state space and right-continuous paths. To see this take g and Lg
bounded and observe that
 t+s
Ex (η(t + s)|Fs ) = Ex (g(ξ(t + s))|Fs ) − g(x) − Ex (Lg(ξ(r ))|Fs ) dr.
0

Since Lg(ξ(r )) is Fs -measurable for all r ≤ s, we can write

56 2 Markov Processes
 t+s  s  t
Ex (Lg(ξ(r ))|Fs ) dr = Lg(ξ(r )) dr + Ex (Lg(ξ(s + r ))|Fs ) dr,
0 0 0

and, by the Markov property, we have

 s  t
Ex (η(t + s)|Fs ) = T (t)g(ξ(s)) − g(x) − Lg(ξ(r )) dr − T (r )(Lg)(ξ(s)) dr,
0 0

which gives the claim, by using the identity (see (3.4) in Sect. 3.1.2)
 t
T (t)g = g + T (r )(Lg) dr, t ≥ 0.
0

There are several different versions of generators for Markov processes. For exam-
ple, one can consider instead of the uniform convergence in B(X ), the pointwise
convergence or the so-called bounded pointwise convergence (see [35]). Another
approach, given by [28], introduces the extended generator using the concept of
local martingales and allowing unbounded functions in the domain of the generator;
here we adopt this definition. Let M(X ) be the space of all measurable functions
g : X → R. An operator L is called the extended generator of the Markov process
ξ , if its domain D( L) consists of those g ∈ M(X ) for which there exists f ∈ M(X )
such that for each x ∈ X , t > 0,
 t
Ex (g(ξ(t))) = g(x) + Ex f (ξ(r )) dr
0

and  t
Ex (| f (ξ(r ))|) dr < ∞,
0

in which case we define Lg = f .

2.3.5 Existence of PDMPs

In this section, we consider the general setting from Sect. 1.19. We assume that (X, Σ)
is a Borel space and that (π, Φ, P) are three characteristics representing, respectively,
a semiflow, a survival function, and a jump distribution, being a transition probability
from X ∪ Γ to X where Γ is the active boundary. We assume that P(x, X \ {x}) = 1
for all x ∈ X ∪ Γ . Since (X, Σ) is a Borel space, we can find a measurable mapping
κ : (X ∪ Γ ) × [0, 1] → X such that

P(x, B) = Leb{r ∈ [0, 1] : κ(x, r ) ∈ B}, x ∈ X ∪ Γ, B ∈ Σ. (2.22)

2.3 Continuous-Time Markov Processes 57

We extend the state space and the characteristics to (X Δ , ΣΔ ) as described in

Sect. 1.19. We define κ(Δ, r ) = Δ for r ∈ [0, 1]. Thus formula (2.22) remains valid
for x ∈ X Δ ∪ Γ . We extend every function g defined on X to X Δ by setting g(Δ) = 0.
For each x ∈ X Δ we define the generalized inverse of t → Φx (t) by

Φx← (q) = inf{t : Φx (t) ≤ q}, q ≥ 0. (2.23)

If ϑ is a random variable uniformly distributed on [0, 1], then we have

P(σ > t) = Φx (t), t ∈ [0, ∞], where σ = Φx← (ϑ).

Let (Ω, F , P) be a probability space and let (ϑn )n≥1 be a sequence of independent
random variables with uniform distribution on [0, 1]. We define τ0 = σ0 = 0, ξ0 = x
and
σ1 = Φξ← 0
(ϑ1 ), τ1 = σ1 + τ0 ,

and we set ξ(t) = π(t − τ0 , ξ0 ) for t < τ1 . Since the function t → π(t, x) has a
left-hand limit, which belongs to the set X Δ ∪ Γ , we can define

ξ1 = κ(ξ(τ1− ), ϑ2 ) and ξ(τ1 ) = ξ1 .

On the set {τ1 = ∞} the process is defined for all times t. On {τ1 < ∞} we continue
the construction of the process inductively. We define σ2 = Φξ← 1
(ϑ3 ), τ2 = σ2 + τ1 ,
and we set

ξ(t) = π(t − τ1 , ξ1 ) if τ1 ≤ t < τ2 , ξ(τ2 ) = ξ2 = κ(ξ(τ2− ), ϑ4 ),

and so on. Consequently, we define the minimal process {ξ(t)}t≥0 starting at ξ(0) = x
by 
π(t − τn , ξn ), if τn ≤ t < τn+1 , n ≥ 0,
ξ(t) = (2.24)
Δ, if t ≥ τ∞ ,

where

τn = σn + τn−1 , σn = Φξ←
n−1
(ϑ2n−1 ), ξn = κ(ξ(τn− ), ϑ2n ), n ≥ 1. (2.25)

Let N (t) be the number of jump times τn in the time interval [0, t]

N (t) = sup{n ≥ 0 : τn ≤ t}. (2.26)

Then N (t) = 0 if t < τ1 , N (t) = n if and only if τn ≤ t < τn+1 , and N (t) = ∞ for
t ≥ τ∞ . If we set ξ∞ = Δ and τ∞+1 = ∞ then we have

ξ(t) = π(t − τn , ξn ) on {τn ≤ t < τn+1 }

58 2 Markov Processes

for some n ∈ N̄ = {0, 1, . . .} ∪ {∞}, t ∈ [0, ∞], which we can rewrite as

ξ(t) = π(t − τ N (t) , ξ N (t) ), t ∈ [0, ∞].

Theorem 2.4 The minimal process ξ(t), t ≥ 0, as defined in (2.24) is a strong

Markov process.

We outline the main steps of the proof. It is similar to the proof given in [28, 29]
for processes with Euclidean state space. Let Ft be the filtration generated by ξ(t).
Then each τn is a stopping time. Observe that for any t ≥ 0

Ft = σ (1 B (ξk )1[0,r ] (τk ) : k ∈ N̄, B ∈ ΣΔ , 0 ≤ r ≤ t).

It is easy to see that Fτn = σ (τk , ξk : k ≤ n), n ∈ N̄, and that

Fs ∩ {τn ≤ s < τn+1 } = Fτn ∩ {τn ≤ s < τn+1 }, n ∈ N̄.

Note that for q, r ≥ 0 we have

Φx← (q) > r ⇐⇒ Φx (r ) > q.

Thus we obtain

P(τn+1 > r |Fτn ) = P(σn+1 > r − τn |Fτn ) = P(Φξ←

n
(ϑ2n+1 ) > r − τn |Fτn )
= Φξn (r − τn )1{τn ≤r } + 1{τn >r } ,

which implies that

P(τn+1 > t + s|Fs ) = Φξ(s) (t) on {τn ≤ s < τn+1 } (2.27)

and leads to the weak Markov property. To show the strong Markov property, we
take a stopping time τ . We have

Fτ ∩ {τn ≤ τ < τn+1 } = Fτn ∩ {τn ≤ τ < τn+1 }, n ∈ N̄,

and for each n there exists Fτn -measurable random variable ζn such that

τ 1{τ <τn+1 } = ζn 1{τ <τn+1 } ,

so that (2.27) remains valid for s = τ .

Let Px be the distribution of the process ξ(t) starting at x. The transition probability
function is given by

P(t, x, B) = Px (ξ(t) ∈ B) = Px (ξ(t) ∈ B, t < τ∞ ) + Px (ξ(t) ∈ B, t ≥ τ∞ ).

2.3 Continuous-Time Markov Processes 59

For x = Δ we have ξ(t) = Δ, thus P(t, Δ, B) = δΔ (B) for all t ≥ 0. If x ∈ X and

Δ∈ / B, then Px (ξ(t) ∈ B, t ≥ τ∞ ) = 0 for all t. Thus for any x ∈ X and B ∈ Σ we
have
∞

P(t, x, B) = Px (ξ(t) ∈ B, t < τ∞ ) = Px (ξ(t) ∈ B, τn ≤ t < τn+1 ). (2.28)
n=0

Note that if

Ex (N (t)) = Ex 1(0,t] (τn ) < ∞ for all t > 0, x ∈ X, (2.29)
n

then the process ξ is non-explosive, i.e. τ∞ = ∞ a.s. In that case we have P(t, x, X ) =
1 for all t > 0 and x ∈ X .

Remark 2.2 If P(x, {x}) = 0 for some x ∈ X then we can extend the state space X
to   by
X = X × {0, 1} and define a transition probability P


P((x, i), B × {1 − i}) = P(x, B), 
P((x, i), B × {i}) = 0, (x, i) ∈ (X ∪ Γ ) × {0, 1}.

κ: 
It follows from (2.22) that  X × [0, 1] → 
X given by 
κ ((x, i), r ) = (κ(x, r ), 1 −
i) satisfies


P((x, i), B × { j}) = Leb{r ∈ [0, 1] : 
κ ((x, i), r ) ∈ B × { j}}).

We define the semiflow   by

π and the survival function Φ

 (x,i) (t) = Φx (t), (x, i) ∈ 

π (t, x, i) = (π(t, x), i) and Φ X , t ≥ 0.

Using the characteristics ( π , Φ,  we construct the process 

 P) ξ (t) = (ξ(t), i(t)),
t ≥ 0, on X Δ = X Δ × {0, 1} as in (2.24). It is strong Markov by Theorem 2.4. Its
restriction ξ to the state space X remains a strong Markov process.

2.3.6 Transition Functions and Generators of PDMPs

We consider the minimal PDMP ξ with characteristics (π, Φ, P) and jump times
(τn ) as given in Sect. 2.3.5. Let Px be the distribution of the process ξ(t) starting at
x. For any non-negative measurable functions h defined on X Δ × [0, ∞], we have

Ex [h(ξ(τ1 ), τ1 )] = h(y, s)P(π(s − , x), dy)Φx (ds).
X Δ ×[0,∞]

We define the transition kernel

60 2 Markov Processes

K (x, B × J ) = Ex [1 B (ξ(τ1 ))1 J (τ1 )], x ∈ X Δ ,

for B ∈ ΣΔ , J ∈ B([0, ∞]). The strong Markov property of the process ξ(t) at τn
implies that the sequence (ξ(τn ), τn ), n ≥ 0, is a Markov chain on X Δ × [0, ∞] such
that for B ∈ ΣΔ and J ∈ B([0, ∞])

P(ξ(τn+1 ) ∈ B, τn+1 − τn ∈ J |Fτn ) = K (ξ(τn ), B × J ).

We have the iterative formula


K n (x, B × J ) = Px (ξ(τn ) ∈ B, τn ∈ J ) = K n−1 (y, B × (J − s))K (x, d y, ds)
X Δ ×[0,∞]

for n ≥ 1, K 1 = K , and K 0 (x, B × J ) = 1 B (x)δ0 (J ). Note that

Px (ξ(t) ∈ B, t < τ1 ) = 1 B (π(t, x))Px (τ1 > t) = 1 B (π(t, x))Φx (t).

Since ξ(t) = π(t − τn , ξ(τn )) on {τn ≤ t < τn+1 }, it follows form (2.28) that
∞ 

P(t, x, B) = 1 B (π(t − s, y))Φ y (t − s)K n (x, dy, ds) (2.30)
n=0 X ×[0,t]

for all x ∈ X , B ∈ Σ, t > 0.

We now show that the transition function P of the process ξ satisfies the following
Kolmogorov equation
 t
P(t, x, B) = 1 B (π(t, x))Φx (t) + P(t − s, y, B)K (x, dy, ds) (2.31)
0 X

for x ∈ X , t > 0, B ∈ Σ. To this end define for each n ≥ 0 and t ≥ 0

Pn (t, x, B) = Px (ξ(t) ∈ B, t < τn+1 ), x ∈ X, B ∈ Σ. (2.32)

It follows from the monotone convergence theorem and (2.28) that

Pn (t, x, B) = Ex (1 B (ξ(t))1{t<τn+1 } ) ↑ Ex (1 B (ξ(t))1{t<τ∞ } ) = P(t, x, B).

For any non-negative measurable function g we have

Ex g(ξ(t))1{t<τn+1 } = Ex g(ξ(t))1{t<τ1 } + Ex g(ξ(t))1{τ1 ≤t<τn+1 }

and the strong Markov property implies that


Ex 1 B (ξ(t))1{τ1 ≤t<τn+1 } = Pn−1 (t − s, y, B)K (x, dy, ds).
X ×[0,t]
2.3 Continuous-Time Markov Processes 61

Hence, the monotone convergence theorem completes the proof of (2.31).

Let M(X )+ (respectively B(X )+ ) be the space of all non-negative (bounded)
measurable functions on X . We define

Tn (t)g(x) = g(y)Pn (t, x, dy) (2.33)
X

for t ≥ 0, x ∈ X , g ∈ M(X )+ , n ≥ 0, where Pn is as in (2.32). Let T be the transition

operator corresponding to the jump distribution P. It is defined for g ∈ M(X )+ by

T g(x) = g(y)P(x, dy), x ∈ X.
X

Then, for t ≥ 0, we have

  t
Tn−1 (t − s)g(y)K (x, dy, ds) = Tn−1 (t − s)g(y)P(π(s − , x), dy)Φx (ds)
X ×[0,t] 0 X
 t
= T (Tn−1 (t − s)g)(π(s − , x))Φx (ds).
0

We now suppose that the semiflow is continuous in X , i.e. π(s − , x) = π(s, x) for
all s < t∗ (x), x ∈ X . We consider Φ as in (1.38) defined with the help of a jump rate
function ϕ. Then
 t  t
T (Tn−1 (t − s)g)(π(s, x))Φx (ds) = T (Tn−1 (t − s)g)(π(s, x))ϕ(π(s, x))Φx (s) ds
0 0

and we obtain
 t
Tn (t)1 B (x) = T0 (t)1 B (x) + T0 (s)(ϕT (Tn−1 (t − s)1 B ))(x) ds (2.34)
0

for all x ∈ X , t ≥ 0, and n ≥ 1.

We conclude this section with a description of the extended generator ( L, D( L))
in the case when the active boundary might be non-empty, the survival function is
as in (1.41), and the minimal process ξ satisfies (2.29). In particular, if the jump
rate function ϕ in condition (1.41) is bounded then (2.29) holds. Given the active
boundary Γ defined in (1.40) we write that g ∈ MΓ (X ) if g : X → R is measurable
and the function t → g(π(t, x)) has a finite limit as t → t∗ (x) for x ∈ X with finite
t∗ (x), where t∗ (x) is the exit time from X as defined in (1.39). If g ∈ MΓ (X ) has the
following properties
(1) for each x ∈ X the function t → g(π(t, x)) is absolutely continuous on (0, t∗ (x)),
(2) for each x ∈ Γ we have

g(x) = g(y)P(x, dy),
X
62 2 Markov Processes

(3) for each t ≥ 0, x ∈ X ,

 

Ex |g(ξ(τn )) − g(ξ(τn− ))| < ∞,
τn ≤t

then g ∈ D( L) and

Lg(x) = L 0 g(x) + ϕ(x) (g(y) − g(x))P(x, dy), x ∈ X, (2.35)
X

with L 0 g defined by
 t
g(π(t, x)) − g(x) = L 0 g(π(s, x)) ds, t < t∗ (x), x ∈ X.
0

A more general condition instead of (3) characterizes all elements of the domain
of the extended generator as defined and showed in [28, 29]. Note that if g is bounded
and condition (2.29) holds, then g satisfies condition (3).
Chapter 3
Operator Semigroups

Semigroups of linear operators provide the primary tools in the study of continuous-
time Markov processes. They arise as the solutions of the initial value problem
for the differential equation u  (t) = Au(t), where A is a linear operator acting on a
Banach space. We describe what is generally regarded as the basic theory. We provide
basic definitions, examples and theorems characterizing the operators as being the
generators of semigroups. The aim here is to provide necessary foundations for
studying semigroups on L 1 spaces in the next chapter.

3.1 Generators and Semigroups

3.1.1 Essentials of Banach Spaces and Operators

In this chapter, we assume that (X ,  · ) is a real Banach space. That is, X is a

real vector space and  · , called the norm, is a non-negative function defined on X
satisfying:

 f  = 0 if and only if f = 0,
c f  = |c|  f , whenever c ∈ R and f ∈ X ,
 f + g ≤  f  + g, whenever f, g ∈ X ,

and the metric space (X , ρ) with ρ( f, g) =  f − g is complete. Usually, X will

be a space of real-valued functions defined on a state space X .
A linear operator A on X is a linear mapping A : D(A) → X , where D(A)
is a linear subspace of X , called the domain of A. It is said to be bounded if
D(A) = X and A = sup f ≤1 A f , called the norm of A, is finite. Note that a
© The Author(s) 2017 63
R. Rudnicki and M. Tyran-Kamińska, Piecewise Deterministic Processes
in Biological Models, SpringerBriefs in Mathematical Methods,
DOI 10.1007/978-3-319-61295-9_3
64 3 Operator Semigroups

linear operator A with D(A) = X is bounded if and only if it is continuous, i.e. the
mapping f → A f is continuous for all f ∈ X . The operator A is a contraction if
A ≤ 1. The operator A is said to be densely defined if its domain D(A) is dense
in X , so that every f ∈ X is a limit of a sequence of elements from D(A). The
operator (A, D(A)) is closed if its graph {( f, A f ) : f ∈ D(A)} is a closed set in the
product space X × X or equivalently if f n ∈ D(A), n ≥ 1,

lim f n = f, and lim A f n = g,

n→∞ n→∞

then f ∈ D(A) and g = A f .

A linear operator (A, D(A)) on a Banach space X is said to be invertible if there
is a bounded operator A−1 on X such that A−1 A f = f for all f ∈ D(A) and
A−1 g ∈ D(A) and A A−1 g = g for all g ∈ X . Observe that A is invertible if and
only if A is closed, A is onto {A f : f ∈ D(A)} = X and A is one to one, i.e. if
A f = 0 then f = 0. The set {A f : f ∈ D(A)}, called the range of (A, D(A)),
is denoted by R(A), and the set { f ∈ D(A) : A f = 0}, called the nullspace, by
N (A). Analogous definitions are valid for operators acting between two different
Banach spaces.
We denote by X ∗ the space of all continuous linear functionals α : X → R. It
is a real Banach space with the norm

α = sup |α( f )|, α ∈ X ∗ ,

 f ≤1

and it is called the dual space of X . We use the duality notation α, f := α( f )

for f ∈ X , α ∈ X ∗ . In particular, the Hahn–Banach theorem allows us to extend
a nonzero continuous functional defined on a closed linear subspace of X to a
continuous functional on the whole Banach space X . The adjoint operator A∗ of a
densely defined linear operator A is a linear operator from D(A∗ ) ⊂ X ∗ into X ∗
defined as follows. We let α ∈ D(A∗ ) if there exists β ∈ X ∗ such that

α, A f = β, f , f ∈ D(A), (3.1)

in which case we set β = A∗ α.

We can define the Riemann integral for a function u : [a, b] → X . similarly to
that for real-valued functions given a partition Δ of the interval [a, b], i.e. a = t0 <
t1 < . . . < tn = b we define the Riemann sum by


n
S(Δ, u) = u(sk )|tk − tk−1 |
k=1

where sk ∈ (tk−1 , tk ]. Let f ∈ X . If for each ε > 0 there exists δ > 0 such that for
every partition Δ with supk |tk − tk−1 | < δ we have  f − S(P, u) < ε, then the
3.1 Generators and Semigroups 65

function u : [a, b] → X is said to be Riemann integrable on the interval [a, b] with

the integral equal to f , which is denoted by
 b
f = u(s) ds.
a

Most properties of the Riemann integral for real-valued functions can be proved
for normed space-valued functions. It is easy to see that we have the following: if
u : [a, b] → X is continuous then u is uniformly continuous on [a, b] and Riemann
integrable on every interval [a, t] for t ∈ [a, b]; in particular
 t
w(t) = u(s) ds
a

is differentiable on [a, b] and w (t) = u(t).

3.1.2 Definitions and Basic Properties

Let, for each t ≥ 0, S(t) : X → X be a bounded linear operator. The family

{S(t)}t≥0 is called a semigroup whenever it satisfies the following conditions:
(a) S(0) = I , where I is the identity operator, i.e. I f = f for f ∈ X ,
(b) S(t + s) = S(t)S(s), s, t ≥ 0,
and it is said to be strongly continuous or a C0 -semigroup if
(c) for each f ∈ X , S(t) f − f  → 0 as t ↓ 0.
If, additionally, S(t) ≤ 1 for every t ≥ 0, so that each operator S(t) is a contraction,
then {S(t)}t≥0 is called a contraction semigroup.
The infinitesimal generator (briefly the generator) of {S(t)}t≥0 is by definition
the operator A with domain D(A) ⊂ X defined as

1
D(A) = { f ∈ X : lim (S(t) f − f ) exists in X },
t↓0 t
1
A f = lim (S(t) f − f ), f ∈ D(A).
t↓0 t

Note that D(A) is a linear subspace of X and A : D(A) → X is a linear operator.

We provide basic properties of strongly continuous semigroups and their genera-
tors. We refer to [34] for their proofs. If {S(t)}t≥0 is a strongly continuous semigroup
with generator (A, D(A)) then the following holds:
(1) There exist constants ω ∈ R and M ≥ 1 such that

S(t) ≤ Meωt , t ≥ 0. (3.2)

66 3 Operator Semigroups

(2) For each f ∈ X the mapping [0, ∞)  t → S(t) f ∈ X is continuous.

(3) If f ∈ X then
 t  t
S(s) f ds ∈ D(A) and S(t) f − f = A S(s) f ds, t > 0. (3.3)
0 0

(4) If f ∈ D(A) then S(t) f ∈ D(A),

 t
d
S(t) f = S(t)A f = AS(t) f and S(t) f − f = S(s)A f ds, t ≥ 0.
dt 0
(3.4)
(5) The operator (A, D(A)) is closed and densely defined.
We now show that the generator uniquely determines a strongly continuous semi-
group. Let A1 , A2 be two linear operators with domains D(A1 ), D(A2 ), respectively.
We say that A2 is an extension of A1 or write A1 ⊆ A2 , if D(A1 ) ⊆ D(A2 ) and
A2 f = A1 f for f ∈ D(A1 ).

Proposition 3.1 Let {S1 (t)}t≥0 and {S2 (t)}t≥0 be strongly continuous semigroups
on a Banach space X with generators (A1 , D(A1 )) and (A2 , D(A2 )), respectively.
If A1 ⊆ A2 then S1 (t) f = S2 (t) f for all f and t ≥ 0.

Proof Let f ∈ D(A1 ) and t > 0. Define u(s) = S2 (s)S1 (t − s) f for s ∈ [0, t]. The
function s → u(s) is differentiable at every s ∈ (0, t) with the derivative

d
u  (s) = (S2 (s)S1 (t − s) f ) = S2 (s)A2 S1 (t − s) f + S2 (s)(−A1 S1 (t − s) f ).
ds

We have S1 (t − s) f ∈ D(A1 ) for all t > s > 0 and A2 S1 (t − s) f = A1 S1 (t − s) f ,

which shows that u  (s) = 0. Thus the function u is constant on [0, t], which implies
that S1 (t) f = u(t) = u(0) = S2 (t) f . Since D(A1 ) is dense in X and the operators
S1 (t) and S2 (t) are bounded, the result follows.

Remark 3.1 Suppose that (A, D(A)) is the generator of a strongly continuous semi-
group. If we let u(t) = S(t) f , t ≥ 0, for a fixed f ∈ D(A) then it follows from
(3.4) that u(t) ∈ D(A), the function t → u(t) is differentiable with a continuous
derivative, and u(t) is the solution of the initial value problem

u  (t) = Au(t), t > 0, u(0) = f. (3.5)

Similar arguments as in the proof of Proposition 3.1 show that u is the only solution
of (3.5) with values in D(A). Consequently, for each f ∈ D(A) equation (3.5) has
one and only one classical solution and it is given by u(t) = S(t) f for t ≥ 0. For
f ∈ X the function t → S(t) f is only a mild solution of (3.5), i.e. it is continuous
and satisfies (3.3).
3.1 Generators and Semigroups 67

Remark 3.2 Observe that if a strongly continuous semigroup {S(t)}t≥0 satisfies (3.2)
then passing to the rescaled semigroup T (t) = e−λt S(t), t ≥ 0, where λ ≥ ω,
yields a bounded semigroup on (X ,  · ), i.e. T (t) ≤ M for all t ≥ 0. Moreover,
(A, D(A)) is the generator of {S(t)}t≥0 if and only if (A −λI, D(A)) is the generator
of {T (t)}t≥0 , since
 
1 1 1
lim (T (t) f − f ) − (S(t) f − f ) = lim (e−λt − 1)S(t) f = −λ f.
t↓0 t t t↓0 t

Next if {S(t)}t≥0 is a bounded semigroup on (X ,  · ), then we can introduce

 f 1 = sup{S(s) f  : s ≥ 0}, f ∈X,

which is a norm on X satisfying  f  ≤  f 1 ≤ M f  and S(t) f 1 ≤  f 1 for

all t ≥ 0, f ∈ X . Thus {S(t)}t≥0 is a contraction semigroup on (X ,  · 1 ).

3.1.3 The Resolvent

Let (A, D(A)) be a linear operator. We say that λ ∈ R belongs to the resolvent
set ρ(A) of A, if the operator λI − A : D(A) → X is invertible. The operator
R(λ, A) := (λI − A)−1 for λ ∈ ρ(A) is called the resolvent operator of A at λ.

Proposition 3.2 Suppose that μ ∈ ρ(A) and that |λ − μ| < R(μ, A)−1 . Then
λ ∈ ρ(A) and
∞

R(λ, A) = (λ − μ)n (R(μ, A))n+1 ,
n=0

where the series converges in the operator norm.

Proof Observe that

R(μ, A)R(λ, A) = R(λ, A)R(μ, A), μ, λ ∈ ρ(A),

since (λI − A)(μI − A) = (μI − A)(λI − A). We also have

μI − A = λI − A + (μ − λ)I = [I + (μ − λ)R(λ, A)](λI − A)

which implies that R(λ, A), λ ∈ ρ(A), satisfies the resolvent identity

R(λ, A) − R(μ, A) = (μ − λ)R(μ, A)R(λ, A), μ, λ ∈ ρ(A).

If (λ−μ)R(μ, A) < 1 then the bounded operator I −(μ−λ)R(μ, A) is invertible

and its inverse is given by Neumann series expansion
68 3 Operator Semigroups

∞

(I − (λ − μ)R(μ, A))−1 = [(λ − μ)R(μ, A)]n ,
n=0

this together with the resolvent identity shows that

R(λ, A) = (I − (λ − μ)R(μ, A))−1 R(μ, A).

We now provide the integral representation of the resolvent operator of the gen-
erator of a strongly continuous semigroup. Let {S(t)}t≥0 be a strongly continuous
semigroup with generator (A, D(A)) and let ω ∈ R and M ≥ 1 be constants such
that (3.2) holds. For each λ > ω we consider
 ∞  r
R(λ) f = e−λt S(t) f dt := lim e−λt S(t) f dt, f ∈X. (3.6)
0 r →∞ 0

We will show that R(λ) is the resolvent operator of A at λ. Since the mapping
t → e−λt S(t) f is continuous and e−λt S(t) f  ≤ Me−(λ−ω)t  f  for f ∈ X , we
see that R(λ) is well defined and that
 ∞  ∞
−λt M
R(λ) f  ≤ e S(t) f  dt ≤ Me−(λ−ω)t dt f  =  f .
0 0 λ−ω

Thus R(λ) is a bounded linear operator and it is the Laplace transform of the semi-
group. Observe that for h > 0 and f ∈ X we have

1 1 ∞ −λt
(S(h) − I )R(λ) f = e (S(h + t) f − S(t) f ) dt
h h 0
 ∞  h
1 1
= (eλh − 1) e−λt S(t) f dt − eλh e−λt S(t) f dt,
h 0 h 0

which implies that

R(λ) f ∈ D(A) and A R(λ) f = λR(λ) f − f,

so that (λI − A)R(λ) = I . Finally, if f ∈ D(A) then

 ∞  ∞
−λt
R(λ)A f = e S(t)A f dt = A e−λt S(t) f dt = A R(λ) f,
0 0

since (3.4) holds and the operator A is closed. Consequently, (ω, ∞) ⊂ ρ(A) and
R(λ) = R(λ, A) for λ > ω.
We close this section with a simple result which will be useful to identify gener-
ators through their resolvents.
3.1 Generators and Semigroups 69

Lemma 3.1 Let (A2 , D(A2 )) be an extension of the operator (A1 , D(A1 )). Assume
that 1 ∈ ρ(A1 ) and that the operator I − A2 is one to one. Then D(A2 ) = D(A1 )
and A2 = A1 .
Proof Let f ∈ D(A2 ) and consider g = f − A2 f . Since 1 ∈ ρ(A1 ), the operator
(I − A1 , D(A1 )) is invertible. Thus we can find h ∈ D(A1 ) such that g = h − A1 h.
We have A1 ⊆ A2 which implies that f − A2 f = h − A1 h = h − A2 h. Since the
operator I − A2 is one to one, we conclude that f = h, showing that D(A2 ) ⊆ D(A1 ).

3.2 Basic Examples of Semigroups

3.2.1 Uniformly Continuous Semigroups

Let A be a bounded operator on a Banach space X . Then, for each f ∈ X and

t ≥ 0, the sequence
m
tn n
A f
n=0
n!

is Cauchy, thus it converges and its limit is denoted by et A f . The family of operators
∞ n
 t
S(t) = et A = An
n=0
n!

is a semigroup and it is a strongly continuous semigroup, since

∞ n
 ∞ n

t t
S(t) f − f  =  An f  ≤ An  f  = (etA − 1) f .
n=1
n! n=1
n!

Moreover, we have
S(t) − I  ≤ etA − 1,

which shows that the semigroup is also uniformly continuous, i.e.

lim S(t) − I  = 0.
t↓0

Observe that
∞ n
 t
S(t) f − f − t A f  ≤ An  f  ≤ (etA − 1 − tA) f ,
n=2
n!

which implies that A is the generator of the semigroup {S(t)}t≥0 .

70 3 Operator Semigroups

We now show that uniformly continuous semigroups have bounded generators.

Note that if a semigroup {S(t)}t≥0 is uniformly continuous then the function [0, ∞) 
t → S(t) ∈ L (X ) is continuous, where the vector space L (X ) of all bounded
linear operators on the Banach space X is itself a Banach space when equipped with
the operator norm. Since S : [0, ∞) → L (X ) is continuous, we obtain
 t
1
lim T (t) = I, where T (t) = S(s)ds,
t↓0 t 0

which implies that there is δ > 0 such that T (t) − I  < 1/2 for 0 ≤ t < δ. Hence,
the operator T (t) is invertible for a sufficiently small t > 0. Since for any f ∈ X
we have
1
(S(t) f − f ) = AT (t) f,
t

we see that the linear operator AT (t) is bounded, thus

A f  = AT (t)T (t)−1 f  ≤ AT (t)T (t)−1  f ,

which shows that D(A) = X and that A is bounded.

Remark 3.3 Note that the use of L ∞ spaces is quite limited in the theory of operator
semigroups. It follows from the result of [66] that if the Banach space X is L ∞ =
L ∞ (X, Σ, m) and {S(t)}t≥0 is a strongly continuous semigroup on L ∞ , then the
generator of {S(t)}t≥0 is a bounded operator. Thus strongly continuous semigroups
on L ∞ are necessarily uniformly continuous.

3.2.2 Multiplication Semigroups

Let (X, Σ, m) be a σ -finite measure space. Consider a measurable non-negative

function ϕ : X → [0, ∞). Define the operator S(t) on L 1 = L 1 (X, Σ, m) by

S(t) f (x) = e−tϕ(x) f (x).

We have
 
−tϕ(x)
S(t) f  = |e f (x)| m(d x) ≤ | f (x)| m(d x) =  f 
X X

for every f ∈ L 1 and t ≥ 0. Thus {S(t)}t≥0 is a semigroup on L 1 . From the

dominated convergence theorem it follows that

lim S(t) f − f  = lim |e−tϕ(x) f (x) − f (x)| m(d x) = 0
t↓0 X t↓0
3.2 Basic Examples of Semigroups 71

for every f ∈ L 1 , which implies that {S(t)}t≥0 is a strongly continuous semigroup

and it is a contraction semigroup.
We now show that the generator of {S(t)}t≥0 is given by

A f = −ϕ f, f ∈ L 1ϕ = { f ∈ L 1 : ϕ f ∈ L 1 }.

To this end denote by (A1 , D(A1 )) the generator of {S(t)}t≥0 . If f ∈ D(A1 ) then

1
lim (S(t) f − f ) = A1 f ∈ L 1 ,
t↓0 t

thus there exists a sequence tn ↓ 0 such that

1
lim (e−tn ϕ(x) − 1) f (x) = A1 f (x)
n→∞ tn

for m-a.e. x ∈ X , which implies that

A1 f (x) = −ϕ(x) f (x) for m-a.e. x ∈ X

and that ϕ f ∈ L 1 . Thus D(A1 ) ⊆ L 1ϕ . Conversely, suppose that f ∈ L 1ϕ . Since

ϕ(x) ≥ 0, we have |e−tϕ(x) − 1| ≤ tϕ(x), which implies that
1 
 
 (S(t) f (x) − f (x)) + ϕ(x) f (x) ≤ 2ϕ(x)| f (x)|.
t

Hence, f ∈ D(A1 ) and A1 f = −ϕ f , since the dominated convergence theorem

gives   
1 
lim  (S(t) f (x) − f (x)) + ϕ(x) f (x) m(d x) = 0.
t↓0 t

3.2.3 Translation Semigroups

Here we give an example showing that not every semigroup is strongly continuous.
Let X be either L 1 (R), the space of Lebesgue integrable functions on R, or B(R),
the space of bounded functions on R. Define the operator S(t) on X by

S(t) f (x) = f (x − t), x ∈ R, t ≥ 0.

It is easy to see that {S(t)}t≥0 is a semigroup on X .

For any f ∈ X = L 1 (R) we have
 
S(t) f  = | f (x − t)| d x = | f (x)| d x =  f .
R R
72 3 Operator Semigroups

Thus {S(t)}t≥0 is a semigroup of contractions on L 1 (R). If f ∈ Cc (R), the space of

continuous functions with compact support, then we have

lim f (x − t) = f (x)
t↓0

for every x ∈ R. By the Lebesgue dominated convergence theorem we obtain


lim S(t) f − f  = lim | f (x − t) − f (x)| d x = 0
t↓0 R t↓0

for every f ∈ Cc (R). Since the set Cc (R) is a dense subset of L 1 (R), the semigroup
is strongly continuous on L 1 (R).
The generator of the translation semigroup on L 1 (R) is given by A f = − f  with
domain

D(A) = { f ∈ L 1 (R) : f is absolutely continuous and f  ∈ L 1 (R)}.

To see this let us denote by (A1 , D(A1 )) the generator of the translation semigroup
{S(t)}t≥0 . Recall that we have S(t) f (x) = f (x − t), x ∈ R, t ≥ 0, f ∈ L 1 (R). First
take f ∈ D(A1 ) so that

1
lim (S(t) f − f ) = g ∈ L 1 (R).
t↓0 t

Then for every compact interval [a, b] ⊂ R we have

    
 1  1 
 ( f (x − t) − f (x)) d x − g(x) d x  ≤  (S(t) f − f ) − g ,
 t t
[a,b] [a,b]

which implies that

 
f (x − t) − f (x)
lim dx = g(x) d x.
t↓0 [a,b] t [a,b]

For all sufficiently small t > 0 we have

  
1 1 1
( f (x − t) − f (x)) d x = f (x) d x − f (x) d x.
[a,b] t t [a−t,a] t [b−t,b]

Since 
1
lim f (x) d x = f (b)
t↓0 t [b−t,b]

for a.e. b ∈ R, we conclude that

3.2 Basic Examples of Semigroups 73

g(x) d x = f (a) − f (b)
[a,b]

for a.e. a, b ∈ R. Thus f is absolutely continuous and its derivative being equal to
−g in the L 1 (R) space is integrable, which shows that the operator (A, D(A)) is an
extension of the generator (A1 , D(A1 )). We have 1 ∈ ρ(A1 ). Note that the general
solution of the differential equation f (x) + f  (x) = 0 is of the form f (x) = ce−x
for x ∈ R, where c is a constant. Thus the operator I − A is one to one which implies
that A = A1 by Lemma 3.1.
Consider now the semigroup {S(t)}t≥0 on the space X = B(R). Then {S(t)}t≥0
is a semigroup of contractions, since

S(t) f u = sup |S(t) f (x)| ≤ sup | f (x)| =  f u .

x∈R x∈R

However, it is no longer strongly continuous on the space B(R). To see this take
f (x) = 1[0,1) (x), x ∈ R. We have

|S(t) f (x) − f (x)| = |1[t,1+t) (x) − 1[0,1) (x)| = |1[1,1+t) (x) − 1[0,t) (x)|

for t ∈ (0, 1) and x ∈ R, which implies that

S(t) f − f u = sup |S(t) f (x) − f (x)| = 1 for all t ∈ (0, 1).

x∈R

Remark 3.4 Note that if {S(t)}t≥0 is a semigroup on a Banach space X , then

X0 = { f ∈ X : lim S(t) f − f  = 0}
t↓0

is a closed linear subspace of X and S(t)(X0 ) ⊆ X0 for all t ≥ 0. Thus {S(t)}t≥0

is a strongly continuous semigroup on the Banach space X0 .
For the translation semigroup on X = B(R) it is easy to see that the subspace
X0 contains all uniformly continuous functions on R.

3.3 Generators of Contraction Semigroups

3.3.1 The Hille–Yosida Theorem

The abstract theory of contraction semigroups was developed independently by Hille

[49] and Yosida [120]. The first major result in the semigroup theory is the following.

Theorem 3.1 (Hille–Yosida) A linear operator (A, D(A)) on a Banach space X

is the generator of a contraction semigroup if and only if D(A) is dense in X , the
74 3 Operator Semigroups

resolvent set ρ(A) of A contains (0, ∞), and for every λ > 0

λR(λ, A) ≤ 1.

In that case, the semigroup {S(t)}t≥0 with generator (A, D(A)) is given by
∞
 λn t n
S(t) f = lim e−λt (λR(λ, A))n f, f ∈ X , t ≥ 0. (3.7)
λ→∞ n!
n=0

For a proof see [34] or [35]. To check that an operator A with a dense domain is the
generator by using the Hille–Yosida theorem, we need to show that for each λ > 0
and g ∈ X there exists a unique solution f ∈ D(A) of

λ f − A f = g and λ f  ≤ g.

The following is a key notion towards a characterization of the generator of a

contraction semigroup that does not require explicit knowledge of the resolvent
operator. An operator (A, D(A)) is called dissipative, if

λ f − A f  ≥ λ f  for all f ∈ D(A) and λ > 0. (3.8)

Observe that if (A, D(A)) is the generator of a strongly continuous contraction

semigroup then λR(λ, A) = (I − λ−1 A)−1 is a contraction for each λ > 0. Thus if
for a given f ∈ D(A) and λ > 0 we let g = f − λ−1 A f then f = (I − λ−1 A)−1 g
and we have  f − λ−1 A f  = g ≥ (I − λ−1 A)−1 g =  f , which shows that
A is dissipative. Also the converse is true, as we prove next.

Lemma 3.2 Suppose that (A, D(A)) is dissipative. Then for each λ > 0 the opera-
tor I − λ−1 A is one to one and its inverse (I − λ−1 A)−1 : R(λI − A) → D(A) is a
contraction, where R(λI − A) is the range of λI − A. Moreover, if R(λI − A) = X
for some λ > 0 then R(λI − A) = X for all λ > 0 and (0, ∞) ⊆ ρ(A).

Proof The linear operator λI − A is one to one for each λ > 0, since from 0 =
λ f − A f  ≥ λ f , it follows that  f  = 0. For g ∈ R(I − λ−1 A) we let
f = (I − λ−1 A)−1 g which gives

g = (I − λ−1 A)(I − λ−1 A)−1 g ≥ (I − λ−1 A)−1 g.

Hence, (I − λ−1 A)−1 is a contraction.

Suppose now that (λ0 I − A)(D(A)) = X for some λ0 > 0. Then the operator
λ0 I − A defined on X is invertible and λ0 ∈ ρ(A). We have R(λ0 , A)−1 ≥ λ0 .
From Proposition 3.2 it follows that each λ satisfying

|λ − λ0 | < λ0
3.3 Generators of Contraction Semigroups 75

belongs to ρ(A), which implies that (λI − A)(D(A)) = X for λ ∈ (0, 2λ0 ).
Repeating the argument we conclude that (λI − A)(D(A)) = X for all λ > 0.
We come to the key result, the Lumer–Phillips reformulation [67] of the Hille–
Yosida theorem.

Theorem 3.2 A linear operator (A, D(A)) on a Banach space X is the generator of
a contraction semigroup if and only if D(A) is dense in X , (A, D(A)) is dissipative,
and the range of the operator λI − A is X for some λ > 0.

To check that a densely defined operator A is the generator by using this refor-
mulation of the Hille–Yosida theorem, we need to show that A is dissipative and that
there is λ > 0 such that for each g ∈ X there exists a solution f ∈ D(A) of the
equation
λ f − A f = g.

We now provide an equivalent definition of a dissipative operator. Let X ∗ be the

dual space of X . A linear operator (A, D(A)) is dissipative if and only if for each
f ∈ D(A) there exists α ∈ X ∗ with α ≤ 1 such that

α, f =  f  and α, A f ≤ 0. (3.9)

It is easy to prove the “if” part, since for each λ > 0 and f ∈ D(A) condition (3.9)
implies

λ f  = α, λ f ≤ α, λ f − α, A f = α, λ f − A f ≤ αλ f − A f .

For the proof of the converse see, e.g. [34, Proposition II.3.23].
It follows from the Hahn–Banach theorem that the normalized duality set

J ( f ) = {α ∈ X ∗ : α ≤ 1 and α, f =  f }

is not empty. In particular, if P is a contraction then the operator A = P − I is

dissipative, since for any f ∈ X and α ∈ J ( f ) we have

α, A f = α, P f − f = α, P f − α, f ≤ α f  −  f  ≤ 0.

Also the operator A = λ(P − I ) is dissipative for every λ > 0.

We have the following extension of the Hille–Yosida theorem to arbitrary semi-
groups, as discovered independently by Feller, Phillips and Miyadera (see [34]).

Theorem 3.3 A linear operator (A, D(A)) generates a strongly continuous semi-
group on X satisfying (3.2) with constants M ≥ 1 and ω ∈ R if and only if D(A)
is dense in X , the resolvent set ρ(A) of A contains (ω, ∞), and

[(λ − ω)R(λ, A)]n  ≤ M, λ > ω, n = 1, 2, . . . . (3.10)

76 3 Operator Semigroups

An operator A with resolvent operator R(λ, A) satisfying (3.10) for some constants
M, ω is called a Hille–Yosida operator.

Remark 3.5 If an operator A is not densely defined, then we can still generate a
semigroup on a closed subspace of the Banach space X . In particular we have the
following; see [34, Chapter III]. If an operator (A, D(A)) is dissipative and the range
of the operator λI − A is X for some λ > 0, then the part A| of the operator A in
the subspace X0 = D(A), i.e.

A| f = A f for f ∈ D(A| ) = { f ∈ D(A) ∩ X0 : A f ∈ X0 },

is densely defined and it generates a contraction semigroup on X0 . If (A, D(A)) is

a Hille–Yosida operator then it generates a strongly continuous semigroup on X0 .

3.3.2 The Lumer–Phillips Theorem

An alternative form of the Lumer–Phillips theorem is more readily applicable. To

state it we need to introduce the following concepts. The operator A is said to be
closable, if it has an extension which is a closed operator. Another way to state this
is that the closure in X × X of the graph

G (A) = {( f, g) ∈ X × X : f ∈ D(A), g = A f }

of A is a graph of a linear operator, i.e. (0, g) ∈ G (A) implies that g = 0. If A is

closable, then the closure A of A is the closed operator whose graph is the closure
of the graph of A. We have

D(A) = { f ∈ X : there exist f n ∈ D(A) and g ∈ X such that f n → f and A f n → g}

and A f = g. It is easily seen that if (A, D(A)) is dissipative and with dense domain,
then it is closable and its closure (A, D(A)) is dissipative and the range of λI − A
is the closure of the range of λI − A. This gives an extension of the Hille–Yosida
theorem.

Theorem 3.4 (Lumer–Phillips) A linear operator (A, D(A)) on a Banach space X

is closable and its closure is the generator of a contraction semigroup if and only if
D(A) is dense in X , (A, D(A)) is dissipative, and the range of the operator λI − A
is dense in X for some λ > 0.

We now show how to check the range condition from the Lumer–Phillips theorem
using the notion of the adjoint operator. It is easy to see that if A is a densely defined
linear operator and λ ∈ ρ(A) then λ ∈ ρ(A∗ ) and R(λ, A)∗ = R(λ, A∗ ), where A∗
is the adjoint of A. Using the Lumer–Phillips theorem we now prove the following
3.3 Generators of Contraction Semigroups 77

Corollary 3.1 Let (A, D) be a dissipative operator with dense domain D. Then
the closure of (A, D) is the generator of a contraction semigroup if and only if the
operator λI ∗ − A∗ is one to one for some λ > 0, where I ∗ is the identity in X ∗ .

Proof Suppose that the closure of the set (λI − A)D is not equal to X . From the
Hahn–Banach theorem it follows that there must exist α ∈ X ∗ , α = 0, such that

α, λ f − A f = 0

for all f ∈ D. This implies that α ∈ D(A∗ ) and A∗ α = λα. Thus λI ∗ − A∗ is not
one to one.

In general, it is difficult to identify the whole domain of the generator. The follow-
ing concept is useful. If (A, D(A)) is a closed linear operator then a linear subspace
D of D(A) is called a core for A if the closure of the restriction of A to D, denoted
by A|D , is equal to A; in symbols, A|D = A. We give a sufficient condition for D
to be a core.

Theorem 3.5 Let {S(t)}t≥0 be a contraction semigroup with generator (A, D(A)).
Suppose that D0 ⊆ D ⊆ D(A) are subspaces of X such that S(t)(D0 ) ⊆ D for all
t > 0. Then D0 ⊆ R(λI − A|D ) for all λ > 0. In particular, if D0 is dense in X
then D is a core for (A, D(A)).

Proof Let g ∈ D0 and μ > 0. The operator (μA, D(A)) is the generator of the
rescaled semigroup {S(μt)}t≥0 . From (3.6) it follows that we have
 r
g = (I − μA)−1 (I − μA)g = lim e−t S(μt)(I − μA)g dt
r →∞ 0

r  − jr/n
n−1
= lim lim e S(μjr/n)(I − μA)g,
r →∞ n→∞ n
j=0

which implies the first claim, since S(μt)(I − μA)g = (I − μA)S(μt)g ∈ R(I −
μA|D ) for any t ≥ 0. The second claim follows from the Lumer–Phillips theorem
and Lemma 3.1.

Finally, the following result gives the uniqueness of the semigroup generated by
an extension of a given operator.

Theorem 3.6 Let {S(t)}t≥0 be a strongly continuous semigroup with generator

extending an operator (A, D). The following conditions are equivalent:
(1) D is a core for the generator of {S(t)}t≥0 .
(2) The closure of (A, D) is the generator of a strongly continuous semigroup.
(3) The semigroup {S(t)}t≥0 is the only semigroup with generator being an extension
of the operator (A, D).
78 3 Operator Semigroups

Proof Let A1 be a closed extension of (A, D). Since the closure A of (A, D) is the
smallest closed extension of (A, D), we have A ⊆ A1 . If A and A1 are generators
of strongly continuous semigroups, then the semigroups are equal. If D is not a core
for the generator of {S(t)}t≥0 , then one can find an infinite number of extensions of
A which are generators (see [5, A-II, Theorem 1.33]).

3.3.3 Perturbations of Semigroups

An important tool for the construction of a semigroup is perturbation theory; see [34,
Chapter III]. Here we only give the Phillips perturbation theorem which concerns
bounded perturbations and the variation of parameters formula. First, we need the
following result.

Lemma 3.3 Assume that (A, D(A)) and (B, D(B)) are linear operators such that
D(A) ⊆ D(B). Let λ ∈ ρ(A) and let B R(λ, A) be a bounded operator on X . Then
the operator (A + B, D(A)) satisfies

R(λI − A − B) = R(I − B R(λ, A)).

Moreover, λ ∈ ρ(A + B) if and only if 1 ∈ ρ(B R(λ, A)). In that case,

R(λ, A + B) = R(λ, A)(I − B R(λ, A))−1 .

Proof Since λ ∈ ρ(A), we have R(R(λ, A)) = D(A) and (λI − A)R(λ, A) f = f
for f ∈ X , which gives

(λI − A − B)R(λ, A) f = (I − B R(λ, A)) f for all f ∈ X . (3.11)

Thus, the first assertion follows. We also have

(λI − A − B) = (I − B R(λ, A))(λI − A),

this together with (3.11) proves the claim.

We now use the Hille–Yosida theorem to prove the following theorem.

Theorem 3.7 (Phillips Perturbation) Let (A, D(A)) be the generator of a strongly
continuous semigroup and let B be a bounded operator. Then (A + B, D(A)) is the
generator of a strongly continuous semigroup.

Proof We can assume that (A, D(A)) is the generator of a contraction semigroup.
Then for each λ > 0 the operator B R(λ, A) is bounded and B R(λ, A) ≤ B/λ.
If we take λ > B, so that 1 ∈ ρ(B R(λ, A)), then it follows from Lemma 3.3 that
λ ∈ ρ(A + B) and
3.3 Generators of Contraction Semigroups 79

1 1 1
R(λ, A + B) ≤ = .
λ 1 − B/λ λ − B

Since R(λ, A + B) = R(λ − B, A + B − BI ), we conclude that (0, ∞) ⊂

ρ(A+ B −BI ) and μR(μ, A+ B −BI ) ≤ 1. From the Hille–Yosida theorem
it follows that the operator (A + B − BI, D(A)) is the generator of a contraction
semigroup. By rescaling, the operator (A + B, D(A)) is also the generator.

We now consider two strongly continuous semigroups {S(t)}t≥0 and {P(t)}t≥0

with generators (A, D(A)) and (C, D(C)). Suppose that C is an extension of the
operator (A + B, D(A)) for some operator (B, D(B)) with D(B) ⊇ D(A). Then
the semigroup {P(t)}t≥0 satisfies the integral equation
 t
P(t) f = S(t) f + P(t − s)B S(s) f ds (3.12)
0

for any f ∈ D(A) and t ≥ 0. To see this observe that for f ∈ D(A) the function
u(s) = P(t − s)S(s) f is continuously differentiable with derivative

u  (s) = P(t − s)AS(s) f − P(t − s)C S(s) f = −P(t − s)B S(s) f,

since S(s) f ∈ D(A) and C S(s) f = AS(s) f + B S(s) f . This implies that
 t  t

P(t) f − S(t) f = u(0) − u(t) = − u (s)ds = P(t − s)B S(s) f ds.
0 0

If the operator B is bounded then the semigroup {P(t)}t≥0 is also given by the
Dyson–Phillips expansion
∞
P(t) f = Sn (t) f, (3.13)
n=0

where S0 (t) f = S(t) f and

 t
Sn+1 (t) f = Sn (t − s)B S0 (s) f ds, f ∈ D(A), n ≥ 0. (3.14)
0

If B is bounded then equalities (3.12) and (3.14) can be extended to all f ∈ X .

Moreover, instead of (3.12) one can also consider the variation of parameters formula
 t
P(t) f = S(t) f + S(t − s)B P(s) f ds.
0

We close this section by giving the Phillips perturbation theorem for perturbations
of contraction semigroups.
80 3 Operator Semigroups

Corollary 3.2 Let P̄ be a contraction on X and let (A0 , D(A0 )) be the generator
of a contraction semigroup {P0 (t)}t≥0 . If λ > 0 is a constant then the operator
(A0 − λI + λ P̄, D(A0 )) is the generator of a contraction semigroup {P(t)}t≥0 and
∞

P(t) = e−λt λn Sn (t) f, f ∈ L 1 , t ≥ 0, (3.15)
n=0

where Sn are as in (3.14) with S0 (t) = P0 (t) and B = P̄.

3.3.4 Perturbing Boundary Conditions

In this section, we describe, based on [44, 45], a perturbation method related to

operators with boundary conditions. On a Banach space X we consider a linear
operator (A, D), called the maximal operator in the sense that it has a sufficiently
large domain D ⊂ X . We assume that there is a second Banach space ∂X which
will serve here as the boundary space. We consider two operators Ψ0 , Ψ : D → ∂X ,
called boundary operators, and we define

D(A) = { f ∈ D : Ψ0 ( f ) = Ψ ( f )}. (3.16)

We denote by (A0 , D(A0 )) the restriction of the operator (A, D) to the zero boundary
condition
A0 f = A f, f ∈ D(A0 ) = { f ∈ D : Ψ0 ( f ) = 0}. (3.17)

We now describe how we can rewrite the operator (A, D(A)) using perturbations.
We define two operators A , B : D(A ) → X × ∂X with D(A ) = D × {0} by

A ( f, 0) = (A f, −Ψ0 f ) and B( f, 0) = (0, Ψ f ) for f ∈ D.

We have
(A + B)( f, 0) = (A f, Ψ f − Ψ0 f ), f ∈ D.

The part of (A , D(A )) in X × {0} denoted by (A| , D(A| )), i.e.

D(A| ) = {( f, 0) ∈ D(A ) ∩ X × {0} : A ( f, 0) ∈ X × {0}} ,

can be identified with (A0 , D(A0 )); we have

D(A| ) = D(A0 ) × {0}, A| ( f, 0) = (A0 f, 0).

Hence, (A| , D(A| )) is the generator of a strongly continuous semigroup on

X × {0} if and only if (A0 , D(A0 )) is the generator of a strongly continuous semi-
3.3 Generators of Contraction Semigroups 81

group on X . Similarly, the part of (A +B, D(A )) in X ×{0}, denoted by (A +B)| ,

can be identified with (A, D(A)); we have

D((A + B)| ) = D(A) × {0}, (A + B)| ( f, 0) = (A f, 0), f ∈ D(A).

Using ideas of Greiner [44] we are able to compute the resolvent operator of the
operator (A, D(A)) as stated next.

Lemma 3.4 Let (A0 , D(A0 )) be as in (3.17) and let λ ∈ ρ(A0 ). Assume that the
operator Ψ0 : D → ∂X restricted to the nullspace N (λI − A) = { f ∈ D : λ f −
A f = 0} is invertible with bounded inverse Ψ (λ) : ∂X → N (λI − A). For the
operator (A, D(A)) with D(A) as in (3.16) we have λ ∈ ρ(A) if and only if I∂X −
Ψ Ψ (λ) is invertible, where I∂X is the identity operator on ∂X . In that case, if
Ψ R(λ, A0 ) and Ψ Ψ (λ) are bounded then the resolvent operator of A at λ is given
by

R(λ, A) f = (I + Ψ (λ)(I∂X − Ψ Ψ (λ))−1 Ψ )R(λ, A0 ) f, f ∈X. (3.18)

Proof We show that the resolvent of the operator A at λ is given by

R(λ, A )( f, f ∂ ) = (R(λ, A0 ) f + Ψ (λ) f ∂ , 0), ( f, f ∂ ) ∈ X × ∂X .

Let Rλ ( f, f ∂ ) = (R(λ, A0 ) f + Ψ (λ) f ∂ , 0) for ( f, f ∂ ) ∈ X × ∂X . Observe that

(λ − A )Rλ ( f, f ∂ ) is equal to

((λ − A)R(λ, A0 ) f + (λ − A)Ψ (λ) f ∂ , Ψ0 R(λ, A0 ) f + Ψ0 Ψ (λ) f ∂ ).

Since R(λ, A0 ) f ∈ D(A0 ) and Ψ (λ) f ∂ ∈ N (λI − A), we obtain

A R(λ, A0 ) f = A0 R(λ, A0 ) f, Ψ0 R(λ, A0 ) f = 0, and (λ − A)Ψ (λ) f ∂ = 0.

Thus,

(λ − A )Rλ ( f, f ∂ ) = ((λ − A0 )R(λ, A0 ) f, Ψ0 Ψ (λ) f ∂ ) = ( f, f ∂ ).

Similarly, for f ∈ D(A) and f ∂ = 0 we have

Rλ (λ − A )( f, f ∂ ) = (R(λ, A0 )(λI − A) f + Ψ (λ)Ψ0 f, 0) = ( f, f ∂ ),

since f = f − Ψ (λ)Ψ0 f + Ψ (λ)Ψ0 f with f − Ψ (λ)Ψ0 f ∈ D(A) and Ψ (λ)Ψ0 f ∈

N (λI − A) implying that

R(λ, A0 )(λI − A) f = R(λ, A0 )(λ − A0 )( f − Ψ (λ)Ψ0 f ) = f − Ψ (λ)Ψ0 f.

82 3 Operator Semigroups

Observe that I − B R(λ, A ) is invertible if and only if I − Ψ Ψ (λ) is invertible,

since

(I − B R(λ, A ))( f, f ∂ ) = ( f, −Ψ R(λ, A0 ) f + (I − Ψ Ψ (λ)) f ∂ )

for any ( f, f ∂ ) ∈ X × ∂X . The resolvent of A + B is, by Lemma 3.3, equal to

R(λ, A + B) = R(λ, A )(I − B R(λ, A ))−1 ,

which completes the proof.

Remark 3.6 It is assumed in [44] that the operators (A, D) and (Ψ0 , D) are closed,
the operator Ψ is bounded, the range of the operator Ψ0 is equal to ∂X , and that
there are constants γ > 0 and λ0 ∈ R such that

Ψ0 ( f ) ≥ λγ  f , f ∈ N (λI − A), λ > λ0 . (3.19)

These conditions imply that λΨ (λ) ≤ γ −1 for all λ ∈ ρ(A0 ) and that one can
apply the Hille–Yosida theorem to show that the operator A generates a strongly
continuous semigroup. In fact, operators I − Ψ (λ)Ψ and I − Ψ Ψ (λ) are invertible
and (I − Ψ (λ)Ψ )−1 = I + Ψ (λ)(I − Ψ Ψ (λ))Ψ . In Sect. 4.1.5 we shall provide an
extension of Grainer’s result to unbounded perturbations in L 1 space.
Chapter 4
Stochastic Semigroups

In this chapter, we introduce stochastic semigroups as strongly continuous semi-

groups of stochastic operators on L 1 spaces. We provide characterizations of their
generators and we explain their connection with PDMPs as defined in the previous
chapters. We give examples of such semigroups which correspond to pure jump-
type processes, to deterministic processes, to semiflows with jumps and to randomly
switched dynamical systems.

4.1 Aspects of Positivity

4.1.1 Positive Operators

In this chapter, we assume that the Banach space X is L 1 = L 1 (X, Σ, m), where
(X, Σ, m) is a σ -finite measure space, with the norm

f = | f (x)| m(d x), f ∈ L 1.
X

We can write any f as the difference of two non-negative functions f = f + − f − ,

where the positive part f + and the negative part f − are defined by

f + = max{0, f } and f − = (− f )+ = max{0, − f }.

We define the positive cone L 1+ to be the set of f ∈ L 1 which are positive f ≥ 0. A lin-
ear operator A : D(A) → L 1 is said to be positive if A f ≥ 0 for f ∈ D(A)+ , where
D(A)+ = D(A) ∩ L 1+ , and we write A ≥ 0. A positive and everywhere defined oper-

© The Author(s) 2017 83

R. Rudnicki and M. Tyran-Kamińska, Piecewise Deterministic Processes
in Biological Models, SpringerBriefs in Mathematical Methods,
DOI 10.1007/978-3-319-61295-9_4
84 4 Stochastic Semigroups

ator is a bounded operator and its norm is determined through values on the positive
cone, as we show next.

Proposition 4.1 Let A be a linear operator with D(A) = L 1 . If

(A f )+  ≤  f + , f ∈ L 1,

then A is positive. If A is positive then A is a bounded operator and

A = sup A f .
f ≥0,  f =1

Proof We have (A f )− = (−A f )+ = (A(− f ))+ . Thus, if f ≥ 0 then f − = 0 and

(A f )−  = (A(− f ))+  ≤ (− f )+  =  f −  = 0.

This shows that A f = (A f )+ ≥ 0.

Observe that we have −| f | ≤ f ≤ | f | for any f ∈ L 1 , thus |A f | ≤ A| f |, which
implies that A f  ≤ A| f | . If we let a be equal to the supremum in the right-hand
of the equality, then a ≤ A. Since | f | ≥ 0, we see that A f  ≤ A| f | ≤ a f 
for any f ∈ L 1 , which shows that A = a
Suppose that A is not bounded. Then we can find a sequence f n ∈ L 1+ such that
 f n  = 1 and A f n  ≥ n 3 for every n. Since f defined by

∞
fn
f =
n=1
n2

is integrable and non-negative, we conclude that f ≥ f n /n 2 , which implies that

n 2 A f ≥ A f n for all n ≥ 1. Thus we have for any n

n 3 ≤ A f n  ≤ n 2 A f ,

which is impossible, since A f  < ∞.

4.1.2 Substochastic Semigroups

A family {P(t)}t≥0 of linear operators on L 1 is called a substochastic semigroup

(stochastic, positive semigroup) if P(t) is a substochastic (stochastic, positive) oper-
ator on L 1 for every t and {P(t)}t≥0 is a strongly continuous semigroup. Thus a
substochastic semigroup is a positive contraction semigroup on L 1 .
We proceed to characterize generators of substochastic semigroups using the result
of [83].
4.1 Aspects of Positivity 85

Theorem 4.1 (Phillips) An operator (A, D(A)) is the generator of a substochastic

semigroup if and only if D(A) is dense in L 1 , A is dispersive, i.e.

(λ f − A f )+  ≥ λ f + , f ∈ D(A), λ > 0,

and the range of λI − A is L 1 for some λ > 0.

Let us outline the proof. Note that if A is dispersive and f ∈ D(A) then − f ∈ D(A)
and, for any λ > 0,

(λ f − A f )−  = (λ(− f ) − A(− f ))+  ≥ λ(− f )+  = λ f − ,

which shows that A is dissipative, since  f  =  f +  +  f −  for any f ∈ L 1 . More-

over, we have (λR(λ, A) f )+  ≤  f + , which implies that the operator λR(λ, A)
is substochastic for every λ ∈ ρ(A), by Proposition 4.1. This shows that (A, D(A))
is the generator of a substochastic semigroup. For the proof of the converse we refer
the reader to [5, C-II].
We now characterize a dispersive operator using positive functionals. The dual
space (L 1 )∗ can be identified with L ∞ = L ∞ (X, Σ, m), thus for each α ∈ (L 1 )∗
there exists a unique element g ∈ L ∞ such that

α( f ) = g, f = g(x) f (x) m(d x), f ∈ L 1.
X

The operator A is dispersive if and only if for each f ∈ D(A) there exists g ∈ L ∞ ,
g ≥ 0, with g||∞ ≤ 1 such that

g, f =  f +  and g, A f ≤ 0.

In general, it is difficult to check that a given operator generates a substochastic

semigroup using the Phillips theorem. We provide another approach in Sect. 4.1.4.

Remark 4.1 The Phillips theorem is valid in Banach lattices; see [5]. It can be used
to generate a positive contraction semigroup on a closed subspace X of the space
B(X ) of bounded measurable functions with the supremum norm  · u . In particular,
suppose that the operator (A, D(A)) satisfies the positive maximum principle of
the form: for each f ∈ D(A) such that  f + u > 0 there exists x0 ∈ X satisfying
f (x0 ) =  f + u and A f (x0 ) ≤ 0. Then the operator A is dispersive, since

(λ f − A f )+ u ≥ max{0, λ f (x0 ) − A f (x0 )} ≥ λ f (x0 ) ≥ λ f + u .

Remark 4.2 A uniformly continuous semigroup with generator A is a positive semi-

group if and only if the operator A + λI is positive for some constant λ (see [5,
C-II, Theorem 1.11]). Thus a substochastic (stochastic) uniformly continuous semi-
group on L 1 (X, Σ, m) has the generator of the form A = −λI + λP, where λ is
86 4 Stochastic Semigroups

a non-negative constant and P is a substochastic (stochastic) operator (see [96,

Section 3.2]).

Remark 4.3 Let P : L 1 (X, Σ, m) → L 1 (X, Σ, m) be a substochastic operator and

Y ∈ Σ. The family of sets ΣY = {B ∈ Σ : B ⊆ Y } is called the restriction of Σ
to Y . By m Y we denote the restriction of the measure m to ΣY . The restriction of
P to the set Y is the substochastic operator PY : L 1 (Y, ΣY , m Y ) → L 1 (Y, ΣY , m Y )
defined by PY f (x) = P f˜(x) for x ∈ Y , where f˜(x) = f (x) for x ∈ Y and f˜(x) =
0 for x ∈/ Y . Analogously, we define the restriction of a substochastic semigroup
{P(t)}t≥0 to a set Y as a substochastic semigroup {PY (t)}t≥0 on L 1 (Y, ΣY , m Y )
given by PY (t) f (x) = P(t) f˜(x) for x ∈ Y and t ≥ 0.

4.1.3 Resolvent Positive Operators

If (A, D(A)) is the generator of a substochastic semigroup {S(t)}t≥0 then it follows

from (3.6) that for λ > 0 and f ∈ L 1+ we have
 ∞
R(λ, A) f = e−λt S(t) f dt ≥ 0.
0

Thus the resolvent operator of A at each λ > 0 is positive. Following Arendt [4], we
call a linear operator A resolvent positive if there exists ω ∈ R such that (ω, ∞) ⊆
ρ(A) and R(λ, A) ≥ 0 for all λ > ω. The spectral bound of A is defined as

s(A) = inf{ω ∈ R : (ω, ∞) ⊆ ρ(A) and R(λ, A) ≥ 0 for all λ > ω}.

We need the following properties of resolvent positive operators.

Lemma 4.1 Let A be a resolvent positive operator and let s(A) be the spectral
bound of A. Then for each μ > λ > s(A)

R(λ, A) ≥ R(μ, A) ≥ 0.

/ ρ(A) when s(A) > −∞.

Moreover, s(A) ∈

Proof By definition s(A) < ∞. The resolvent identity

R(λ, A) − R(μ, A) = (μ − λ)R(μ, A)R(λ, A),

implies that R(λ, A) ≥ R(μ, A) if μ > λ > s(A). Suppose now that s(A) > −∞
and that μ = s(A) ∈ ρ(A). Then R(μ, A) ≥ 0 and for λ < μ such that (μ −
λ)R(μ, A) < 1 we see, by Proposition 3.2, that λ ∈ ρ(A) and
4.1 Aspects of Positivity 87

∞

R(λ, A) = (μ − λ)n R(μ, A)n+1 ≥ 0,
n=0

which gives a contradiction with the definition of s(A).

We now give the following characterization [117] of positive perturbations of

resolvent positive operators as resolvent positive operators.
Theorem 4.2 Suppose that (A, D(A)) is a resolvent positive operator and that
(B, D(B)) is a positive operator with D(B) ⊇ D(A). The operator (A + B, D(A))
is resolvent positive if and only if

lim (B R(λ, A))n  = 0 for some λ > s(A). (4.1)

n→∞

The following unpublished perturbation result [30] for generators of positive semi-
groups is only valid on an L 1 space. For its proof we refer the reader to the framework
of Miyadera–Voigt perturbation theory (see [10] and [117]).

Theorem 4.3 (Desch) Suppose that (A, D(A)) is the generator of a positive semi-
group on L 1 and that B with D(B) ⊇ D(A) is a positive operator. If (A + B, D(A))
is resolvent positive, then (A + B, D(A)) is the generator of a positive semigroup.

4.1.4 Generation Theorems

We now provide a generation theorem for substochastic semigroups on L 1 . Recall

that we denote by D(A)+ the set of all non-negative elements from the domain D(A)
of an operator A.

Theorem 4.4 A linear operator (A, D(A)) is the generator of a substochastic semi-
group on L 1 if and only if D(A) is dense in L 1 , the operator A is resolvent positive,
and 
A f (x) m(d x) ≤ 0 for all f ∈ D(A)+ . (4.2)
X

Proof Suppose first that (A, D(A)) is the generator of a substochastic semigroup
{S(t)}t≥0 . We know that (A, D(A)) is resolvent positive. We also have for any f ∈
D(A)+  
1
A f (x) m(d x) = lim (S(t) f (x) − f (x)) m(d x)
X t→∞ t X
1
= lim (S(t) f  −  f ) ≤ 0.
t→∞ t

To show that (A, D(A)) is the generator we make use of the Hille–Yosida theorem.
Since A is resolvent positive, A is closed and the resolvent R(λ, A) is a positive
88 4 Stochastic Semigroups

operator for all λ > s(A), by Lemma 4.1. Thus, it remains to check that s(A) ≤ 0
and that λR(λ, A) is a contraction for all λ > 0. Take f ∈ D(A)+ . Then, for any
λ > 0, we have
 
(λ − A) f  ≥ (λ − A) f (x) m(d x) = λ f  − A f (x) m(d x) ≥ λ f .
X X

Consequently, if λ > max{0, s(A)} then

g ≥ λR(λ, A)g for all g ∈ L 1+ ,

and λR(λ, A) ≤ 1. Suppose now that s(A) > 0. Let λn > s(A) and λn ↓ s(A).
We have
1 1
R(λn , A) ≤ ≤ , n ≥ 0.
λn s(A)

Since s(A) > −∞, we have s(A) ∈ / ρ(A). Thus R(λn , A) → ∞, which gives a
contradiction and completes the proof.

Recall that an operator is stochastic if and only if it is substochastic and it preserves

the L 1 norm on the positive cone. Thus Theorem 4.4 leads to the following

Corollary 4.1 A linear operator (A, D(A)) is the generator of a stochastic semi-
group on L 1 if and only if D(A) is dense in L 1 , the operator A is resolvent positive,
and 
A f (x) m(d x) = 0 for all f ∈ D(A)+ .
X

Proof If {S(t)}t≥0 is a stochastic semigroup, then S(t) f  =  f  for all f ∈ L 1+ .

Hence

1
A f (x) m(d x) = lim (S(t) f  −  f ) = 0, f ∈ D(A)+ .
X t→∞ t

To show that a substochastic semigroup existing by Theorem 4.4 is stochastic let us

take f ∈ D(A)+ . Then S(t) f ∈ D(A)+ for t > 0 and
 
d d
S(t) f  = S(t) f (x) m(d x) = AS(t) f (x) m(d x) = 0.
dt dt X X

4.1.5 Positive Perturbations

In this section, we provide sufficient conditions for the sum of two operators to be the
generator of a substochastic semigroup. We assume that (A, D(A)) is the generator
of a substochastic semigroup and that (B, D(B)) is an operator with D(B) ⊇ D(A).
4.1 Aspects of Positivity 89

It follows from Theorem 4.4 that (A + B, D(A)) is the generator of a substochastic

semigroup if and only if (A + B, D(A)) is resolvent positive and

(A f (x) + B f (x)) m(d x) ≤ 0 for f ∈ D(A)+ . (4.3)
X

We now suppose that (A, D(A)) is resolvent positive and (B, D(A)) is a positive
operator. Then R(λ, A) f ≥ 0 for every λ > s(A) and B R(λ, A) f ≥ 0 for f ∈ L 1+
which implies that the positive operator B R(λ, A) being defined everywhere is
bounded. If, additionally, (4.3) holds then the operator B R(λ, A) is substochastic
for every λ > 0. To see this take λ > 0 and observe that

(λI − A − B)R(λ, A) f = (I − B R(λ, A)) f

for f ∈ L 1 . Thus

B R(λ, A) f + λR(λ, A) f = f + (A + B)R(λ, A) f for f ∈ L 1+ . (4.4)

This together with (4.3) implies that

B R(λ, A) f  + λR(λ, A) f  ≤  f  for f ∈ L 1+ , (4.5)

showing that B R(λ, A) ≤ 1. If there exists λ > 0 such that B R(λ, A) < 1 then
(A + B, D(A)) generates a substochastic semigroup, as we show next. The case
when B R(λ, A) = 1 for all λ > 0 is discussed in the next section.
Theorem 4.5 Let (A, D(A)) be resolvent positive and (B, D(A)) be a positive
operator such that (4.3) holds. If B R(λ, A) < 1 for some λ > 0 then
∞

R(λ, A + B) = R(λ, A) (B R(λ, A))n
n=0

and the operator (A + B, D(A)) generates a substochastic semigroup.

Proof Since B R(λ, A) < 1, the operator I − B R(λ, A) is invertible. Its inverse
is given by the Neumann expansion
∞

(I − B R(λ, A))−1 = (B R(λ, A))n
n=0

and is a positive operator. The formula for R(λ, A + B) follows from Lemma 3.3.
Thus the operator (A + B, D(A)) is resolvent positive, which together with Theo-
rem 4.4 completes the proof.
We show that positive bounded perturbations of generators of substochastic semi-
groups lead to positive semigroups.
90 4 Stochastic Semigroups

Corollary 4.2 Suppose that (A, D(A)) is the generator of a substochastic semi-
group and that B with D(B) = L 1 is a positive operator. Then (A + B, D(A)) is
the generator of a positive semigroup.

Proof By rescaling, it is enough to show that the operator (A + B − BI, D(A)) is

the generator of a substochastic semigroup. Since B is bounded and λR(λ, A) ≤ 1,
there exists λ > 0 such that B < λ, which implies that B R(λ, A) < 1. We have
R(λ, A) = R(λ − B, A − BI ) and the operator (A − BI, D(A)) generates
a substochastic semigroup. For f ∈ D(A)+ we have
 
(A + B − BI ) f (x) m(d x) = A f (x) m(d x) + B f  − B f  ≤ 0,
X X

and the claim follows from Theorem 4.5.

We now consider positive perturbations of generators on L 1 with boundaries as

studied in [45]. The case of unbounded positive operator B is treated in the next
section.

Theorem 4.6 Let L 10 = L 1 (Γ0 , Σ0 , m 0 ), where (Γ0 , Σ0 , m 0 ) is a σ -finite measure

space. We assume that (A, D) is an operator on L 1 such that
(1) for each λ > 0 the operator Ψ0 : D → L 10 restricted to the nullspace N (λI −
A) of the operator (λI − A, D) is invertible with positive inverse Ψ (λ) : L 10 →
N (λI − A);
(2) the operator Ψ : D → L 10 is positive and there is c > 0 such that λΨ Ψ (λ) ≤ c
for all λ > 0;
(3) the restriction A0 of the operator A to D(A0 ) = N (Ψ0 ) is resolvent positive.
If B is a bounded positive operator and condition (4.3) holds for A with D(A) =
N (Ψ − Ψ0 ), then (A + B, D(A)) is the generator of a substochastic semigroup.

Proof Condition (2) implies that the operator I − Ψ Ψ (λ) is invertible for λ > c.
Since (A0 , D(A0 )) is resolvent positive, the operators Ψ and Ψ (λ) are positive and
the operator I − Ψ Ψ (λ) is invertible for all sufficiently large λ > 0, Lemma 3.4
implies that the operator (A, D(A)) is resolvent positive. Consequently, we have
B R(λ, A) < 1 for some λ > 0 and the result follows from Theorem 4.5.

We now show how we can generate stochastic semigroups by using perturbation

theorems.

Theorem 4.7 Suppose that (A0 , D(A0 )) is the generator of a stochastic semigroup
on L 1 = L 1 (X, Σ, m) and that P is a stochastic operator on L 1 . Then for any
bounded measurable function ϕ : X → [0, ∞) the operator

A1 f = A0 f − ϕ f, f ∈ D(A0 ),

is the generator of a substochastic semigroup and the operator

4.1 Aspects of Positivity 91

A f = A1 f + P(ϕ f ), f ∈ D(A0 ),

is the generator of a stochastic semigroup.

Proof The function ϕ is bounded. Thus we can take λ = sup{ϕ(x) : x ∈ X }. Then
λ − ϕ(x) ≥ 0 for all x. By rescaling, the operator (A0 − λI, D(A0 )) is the generator
of a substochastic semigroup. We have

A1 f = A0 f − ϕ f = A0 f − λ f + (λ − ϕ) f for f ∈ D(A0 ).

The operator (A0 , D(A0 )) is the generator of a stochastic semigroup and ϕ ≥ 0,

hence
 
A1 f (x) m(d x) = − ϕ(x) f (x) m(d x) ≤ 0
X X

for any f ∈ D(A0 )+ . Since the operator f → (λ − ϕ) f is positive and bounded, the
operator (A1 , D(A0 )) generates a substochastic semigroup, by Theorem 4.5. Finally,
the operator B f = P(ϕ f ) for f ∈ L 1 is bounded and positive. Since P preserves
the integral on the cone L 1+ , we have
 
B f (x) m(d x) = ϕ(x) f (x) m(d x), f ∈ L 1+ .
X X

Consequently, the operator (A1 + B, D(A0 )) is resolvent positive and is the generator
of a stochastic semigroup, by Corollary 4.1.
Remark 4.4 Note that if A0 generates a substochastic semigroup and P is a substo-
chastic operator then A as defined in Theorem 4.7 generates a substochastic semi-
group.

4.1.6 Positive Unbounded Perturbations

In this section we describe a generation result of Kato [56] (see also [9, 10, 116])
ensuring that some extension of (A + B, D(A)) is the generator of a substochastic
semigroup when the operator B is positive and unbounded.
Theorem 4.8 Assume that (A, D(A)) is the generator of a substochastic semigroup
on L 1 and that (B, D(A)) is a positive operator such that (4.3) holds. There exists
an extension (C, D(C)) of the operator (A + B, D(A)) which is the generator of a
substochastic semigroup {P(t)}t≥0 . The resolvent operator of C is


N
R(λ, C) f = lim R(λ, A) (B R(λ, A))n f, f ∈ L 1 , λ > 0. (4.6)
N →∞
n=0
92 4 Stochastic Semigroups

The semigroup {P(t)}t≥0 from Theorem 4.8 is called the minimal semigroup
related to A + B. To justify the name we show that the semigroup {P(t)}t≥0 is the
smallest substochastic semigroup whose generator is an extension of (A + B, D(A)),
 t≥0 with generator (C,
i.e. for any positive semigroup { P(t)}  D(C))
 being an exten-
 f ≥ P(t) f for all f ∈ L 1+ , t > 0.
sion of the operator (A + B, D(A)) we have P(t)
To this end for each r ∈ (0, 1) consider the operator Ar = A + r B with domain
D(A). We have r B R(λ, A) ≤ r < 1 and the operator (Ar , D(A)) is resolvent
positive with
∞

R(λ, A + r B) f = R(λ, A) (r B R(λ, A))n f, f ∈ L 1 , λ > 0.
n=0

For f ∈ D(A) we have

 f = (1 − r )B f
(λI − A − r B) f − (λI − C)

which leads to

 − R(λ, A + r B) = (1 − r )R(λ, C)B

R(λ, C)  R(λ, A + r B) ≥ 0

 Hence, for each f ∈ L 1+ we have

for λ > s(C).


N
 f ≥ R(λ, A + r B) f ≥ R(λ, A)
R(λ, C) (r B R(λ, A))n f
n=0

for all r ∈ (0, 1) and N ≥ 0. Taking the limit as r → 1 and then as N → ∞, we

obtain
 f ≥ R(λ, C) f, f ∈ L 1+ ,
R(λ, C)

 f ≥ P(t) f , by (3.7).
which implies that P(t)
The minimal semigroup {P(t)}t≥0 can be defined by

P(t) f = lim Pr (t) f, f ∈ L 1 , t > 0,

r →1

where, for each r ∈ (0, 1), {Pr (t)}t≥0 is a substochastic semigroup generated by
(A + r B, D(A)). The semigroup {P(t)}t≥0 satisfies the integral Eq. (3.12) and it is
also given by the Dyson–Phillips expansion (3.13), where {S(t)}t≥0 is the semigroup
generated by (A, D(A)). Theorem 4.8 and the Lumer–Phillips theorem imply that the
closure of the operator (A + B, D(A)) is the generator of a substochastic semigroup
if and only if the range of λI − (A + B) is dense in L 1 for some λ > 0. Equivalently,
by Corollary 3.1, the operator λI ∗ − (A + B)∗ is one to one. By Theorem 3.6, what
we are missing is the characterisation of a core of the generator of the minimal
semigroup. Necessary and sufficient conditions for the generator of the minimal
4.1 Aspects of Positivity 93

semigroup {P(t)}t≥0 to be the closure of the operator (A + B, D(A)) are to be

found in [10, 113, 114]. In particular, we have the following.
Theorem 4.9 Let λ > 0 and let {P(t)}t≥0 be the minimal semigroup related to A +
B satisfying

(A f (x) + B f (x)) m(d x) = 0 for f ∈ D(A)+ . (4.7)
X

Then the following are equivalent

(1) The minimal semigroup {P(t)}t≥0 related to A + B is stochastic.
(2) For all f ∈ L 1
lim (B R(λ, A))n f  = 0. (4.8)
n→∞

(3) If for some g ∈ L ∞ , g ≥ 0, we have (B R(λ, A))∗ g = g then g = 0, where

(B R(λ, A))∗ denotes the adjoint of B R(λ, A).
(4) The closure of (A + B, D(A)) is the generator of a substochastic semigroup.
Remark 4.5 Observe that condition (4.8) is weaker than condition (4.1), the latter
implying that the operator A + B is resolvent positive which together with (4.7)
implies that (A + B, D(A)) is the generator. In particular, it is shown in [114]
that under assumption (4.3) the operator (A + B, D(A)) is the generator of a sub-
stochastic semigroup if and only if the operator B R(λ, A) is quasi-compact for
some λ > 0, i.e. there exists a linear compact operator K and n ∈ N such that
(B R(λ, A))n − K  < 1. Recall that an operator K is called compact if the clo-
sure of the image of any bounded set under the mapping K is a compact subset of
L 1.
Remark 4.6 If A is the generator of a substochastic semigroup then R(λ, A)∗ =
R(λ, A∗ ) for all λ > 0. Moreover, we have

(B R(λ, A))∗ g = R(λ, A∗ )B ∗ g for g ∈ D(B ∗ ).

Hence, any g ∈ L ∞ , g ≥ 0, is a solution of (B R(λ, A))∗ g = g if and only if g ∈

D(A∗ ) ∩ D(B ∗ ) and
A∗ g + B ∗ g = λg.

Note that if we let

B f = P(ϕ f ) for f ∈ L 1ϕ = { f ∈ L 1 : ϕ f ∈ L 1 },

then D(B ∗ ) = {g ∈ L ∞ : ϕ P ∗ g ∈ L ∞ }. If, additionally, A f = −ϕ f , then D(A∗ ) =

{g ∈ L ∞ : ϕg ∈ L ∞ }) and the adjoint of the operator B R(λ, A) is given by
ϕ
(B R(λ, A))∗ g = P ∗ g.
λ+ϕ
94 4 Stochastic Semigroups

4.1.7 Adjoint and Transition Semigroups

We say that a substochastic semigroup {P(t)}t≥0 corresponds to a transition function

P = {P(t, ·) : t ≥ 0} if for each t > 0 the adjoint operator P ∗ (t) of P(t) is given by

P ∗ (t)g(x) = g(y)P(t, x, dy), g ∈ L ∞ .
X

Equivalently, each transition kernel P(t, ·) satisfies (2.4), or

 
P(t) f (x) m(d x) = P(t, x, B) f (x) m(d x), B ∈ Σ, f ∈ L 1+ . (4.9)
B X

In particular, if a stochastic semigroup {P(t)}t≥0 corresponds to the transition func-

tion P induced by a Markov process ξ = {ξ(t) : t ≥ 0}, i.e.

P(t, x, B) = Px (ξ(t) ∈ B), x ∈ X, B ∈ Σ,

where Px is the distribution of ξ(t) starting at x, then P(t) f is the density of ξ(t) if
the distribution of ξ(0) has a density f .
In this section, we relate a stochastic semigroup {P(t)}t≥0 to the transition
semigroup {T (t)}t≥0 on B(X ) associated to a homogeneous Markov process ξ =
{ξ(t) : t ≥ 0} with transition function P = {P(t, ·) : t ≥ 0} as described in Sect. 2.3.4.
The transition semigroup {T (t)}t≥0 on the Banach space B(X ) with supremum norm
 · u associated to the process ξ is given by

T (t)g(x) = Ex (g(ξ(t)) = g(y)P(t, x, dy), g ∈ B(X ).
X

In particular, if
 
T (t)g(x) f (x) m(d x) = g(x)P(t) f (x) m(d x) (4.10)
X X

holds for all f ∈ L 1+ and all g ∈ B(X ), then (4.9) holds and {P(t)}t≥0 corresponds
to the transition function P.
The transition semigroup {T (t)}t≥0 on B(X ) is strongly continuous on the closed
subspace of B(X )

B0 (X ) = {g ∈ B(X ) : lim T (t)g − gu = 0}

t↓0

and T (t)g ∈ B0 (X ) for g ∈ B0 (X ). The generator (L , D(L)) of the semigroup

{T (t)}t≥0 is densely defined in B0 (X ). On the other hand for each t, the adjoint
P ∗ (t) of the operator P(t) is a contraction on L ∞
4.1 Aspects of Positivity 95
 
P ∗ (t)g(x) f (x) m(d x) = g(x)P(t) f (x) m(d x), g ∈ L ∞ , f ∈ L 1 ,
X X

and {P ∗ (t)}t≥0 is a semigroup on L ∞ , called the adjoint semigroup. We would like

to relate the semigroups by using their generators. We define the subspace of strong
continuity of the adjoint semigroup {P ∗ (t)}t≥0 by

L  = {g ∈ L ∞ : lim P ∗ (t)g − g∞ = 0},

t↓0

where  · ∞ denotes the essential supremum norm in L ∞ . It is called the sun dual
(or the semigroup dual) space. The semigroup given by the part of P ∗ (t) in L  , i.e.

P  (t)g = P ∗ (t)g for g ∈ L  ,

is called the sun dual semigroup. It is strongly continuous, thus a contraction semi-
group on L  . Let (A, D(A)) be the generator of the semigroup {P(t)}t≥0 and let
(A∗ , D(A∗ )) be its adjoint. It is easy to see that D(A∗ ) ⊂ L  and that A∗ is an exten-
sion of the generator A of the sun dual semigroup {P  (t)}t≥0 . In fact, (A , D(A ))
is the part of (A∗ , D(A∗ )) in L  , i.e.

A g = A∗ g for g ∈ D(A ) = {g ∈ D(A∗ ) : A∗ g ∈ L  }.

Consequently, if Eq. (4.10) holds for all g ∈ B0 (X ) and f ∈ L 1 then {T (t)}t≥0 is the
part of the sun dual semigroup {P  (t)}t≥0 in B0 (X ) and we have L ⊆ A ⊆ A∗ .
We would like to obtain (4.10) using generators. We now assume that there exists
a closed subspace X of B0 (X ) such that T (t)g ∈ X for g ∈ X and that D L is a
core for the part of the generator (L , D(L)) of {T (t)}t≥0 in X . We suppose that the
following condition holds
 
Lg(x) f (x) m(d x) = g(x)A f (x) m(d x) (4.11)
X X

for g ∈ D L and f ∈ D A , where D A is a core for the generator (A, D(A)) of the
stochastic semigroup {P(t)}t≥0 on L 1 . By approximations, condition (4.11) extends
to all functions from the domains of the generators. It is easy to see that the function

v(s) = T (s)g(x)P(t − s) f (x) m(d x), s ∈ [0, t], g ∈ D(L), f ∈ D(A),
X

is continuous on [0, t] and differentiable with v (s) = 0 for s ∈ (0, t). This implies
that (4.10) holds for g ∈ D(L) and f ∈ D(A). Since domains of the generators are
dense in the corresponding spaces, we conclude that P ∗ (t)g = T (t)g for g ∈ X .
If X is large enough so that we can extend equality (4.10) to all g being indicator
functions, then {P(t)}t≥0 corresponds to the transition function P. In particular, if X
is a metric space and Σ = B(X ), then, by the uniqueness theorem A.2, if condition
96 4 Stochastic Semigroups

(4.9) holds for all closed subsets B, then it holds for all Borel sets. Since each indicator
function of a closed set can be approximated by a sequence of (globally) Lipschitz
continuous functions, it is enough to check (4.10) for uniformly continuous functions
g : X → R, or only for Lipschitz continuous functions.
To check condition (4.11) we can use the extended generator ( L, D(L)), which
for a class of PDMPs is characterized at the end of Sect. 2.3.6. The next result may
be useful in showing that L is a generator. The notion of resolvent positive operators
as introduced in Sect. 4.1.3 can also be used for operators defined on X ⊂ B(X )
with the supremum norm.

Theorem 4.10 Let (L , D(L)) be a resolvent positive operator in a closed subspace

X of B(X ). If 1 X ∈ D(L) then L is the generator of a positive semigroup on the
closure of D(L). If, additionally, L1 X = 0 then L generates a contraction semigroup.

Proof The assumption 1 X ∈ D(L) implies that the operator L is cofinal, i.e. for each
non-negative g ∈ X there exists a non-negative h ∈ D(L) such that g(x) ≤ h(x) for
all x ∈ X . As in the proof of [4, Theorem 2.1] we conclude that L is a Hille–Yosida
operator. Consequently, L generates a positive semigroup on the closure of D(L)
(see Remark 3.5).

However, in general it is difficult to determine a sufficiently large space X . Thus,

in the next section we will also prove directly condition (4.10) by either using the
formula for the transition function or by using the Dyson–Phillips expansion for the
semigroup {P(t)}t≥0 and the Kolmogorov equation (2.31).

4.2 Stochastic Semigroups for PDMPs

4.2.1 Jump-Type Markov Processes

In Sect. 1.3 we considered a pure jump-type Markov process. This is a Markov process
on the space (X, Σ, m) which remains constant between jumps; see Sect. 2.3.3. In
the case when the state space X is at most a countable set, we obtain continuous-time
Markov chains. We assume that a measurable function ϕ : X → [0, ∞) is a jump
rate function. Let a transition probability function P(x, B) describe the new position
of the point after a jump from x. We assume that P(x, B) satisfies (2.4) and that P
is the stochastic operator on L 1 (X, Σ, m) corresponding to P(x, B).
Suppose first that ϕ is bounded. We now show that if the random variable ξ(t) is
the position of the moving point at time t and ξ(0) has density u(0), then ξ(t) has
density u(t) which satisfies the evolution equation

u  (t) = Au(t) = −ϕu(t) + P(ϕu(t)). (4.12)

To this end we let λ = sup{ϕ(x) : x ∈ X } and

4.2 Stochastic Semigroups for PDMPs 97
 
P̄ f (x) = λ−1 P(ϕ f )(x) + (λ − ϕ(x)) f (x) .

Then P̄ is a stochastic operator and A = −λI + λ P̄. The stochastic operator P̄

corresponds to the transition probability P̄ defined in (2.20). It follows from Corollary
3.2 that the solution of (4.12) is given by
∞
 (λt)k e−λt
u(t) = P̄ k u(0). (4.13)
k=0
k!

The stochastic semigroup {P(t)}t≥0 defined by P(t) f = u(t) with f = u(0) corre-
sponds to the transition function P(t, x, B), induced by the process ξ = {ξ(t) : t ≥ 0}
(see Sect. 4.1.7) and given by (2.21), since
 
P̄ f (x) m(d x) =
k
P̄ k (x, B) f (x) m(d x)
B X

for all k, f ∈ L 1+ and B ∈ Σ.

Suppose now that ϕ is unbounded and that

A f = −ϕ f + P(ϕ f ) for f ∈ L 1ϕ = { f ∈ L 1 : ϕ f ∈ L 1 }.

The operator L 1ϕ  f → −ϕ f is the generator of a substochastic semigroup, by

Sect. 3.2.2, and the operator L 1ϕ  f → P(ϕ f ) is positive. For f ∈ L 1ϕ and f ≥ 0,
we have
  
A f (x) m(d x) = −ϕ(x) f (x) m(d x) + P(ϕ f )(x) m(d x) = 0,
X X X

since P preserves the integral of non-negative integrable elements. Theorem 4.8

implies that there exists the minimal substochastic semigroup {P(t)}t≥0 generated
by an extension of the operator (A, L 1ϕ ). From Theorem 4.9 and Remark 4.6 it follows
that the minimal semigroup related to A is stochastic if and only if every non-negative
and essentially bounded solution g of the equation

ϕ(P ∗ g − g) = λg

is equal to 0 a.e., where P ∗ is the adjoint of P and λ > 0.

We close this section by providing the Kato original result [56, Theorem 3] in the
setting of l 1 = L 1 (I, Σ, m) where I is at most a countable set, Σ is the σ -algebra
of all subsets of I , and m is the counting measure on I . Let Q = [qi j ]i, j∈I be a
matrix, or an infinite dimensional matrix, when I is infinite.  We consider a linear
operator (bounded or unbounded) on l 1 given by (Qu)i = j∈I qi j u j for i ∈ I . The
matrix Q = [qi j ]i, j∈I is called a Kolmogorov matrix, if its entries have the following
properties
98 4 Stochastic Semigroups

(i) i j ≥ 0 for i  = j,
q
(ii) i∈I qi j = 0 for j ∈ I ,

and it is called a sub-Kolmogorov matrix if it satisfies condition (i) and the condition

(ii ) i∈I qi j ≤ 0 for j ∈ I .

Let us define the jump rate function ϕ : I → [0, ∞) by ϕ = (ϕ j ) j∈I with ϕ j =

−q j j , and the jump transition kernel by P( j, {i}) = pi j with pii = 0 and

qi j
pi j = if ϕ j > 0 and pi j = 1 if ϕ j = 0, i = j.
ϕj

The transition kernel P defines the substochastic operator P : l 1 → l 1 by


(Pu)i = pi j u j , i ∈ I, u ∈ l 1 .
j∈I

We have 
(Qu)i = qii u i + qi j u j = −ϕi u i + P(ϕu)i .
j=i

If Q is a sub-Kolmogorov matrix then condition (4.3) holds and the minimal substo-
chastic semigroup {P(t)}t≥0 related to Q is generated by an extension of the operator
Q. Note that the matrix Q is a Kolmogorov matrix if and only if

ϕj = qi j , j ∈ I,
i= j

or equivalently P is a stochastic operator. The matrix Q is a bounded operator on l 1

if and only if ϕ is bounded:
sup |q j j | < ∞.
j

In this case the solution of (4.12) is given by (4.13). If ϕ is unbounded then we have
the following result.

Theorem 4.11 (Kato) Let Q be a Kolmogorov matrix and let λ > 0 be a posi-
tive constant. We denote Q ∗ = (qi∗j )i, j∈I , where qi∗j = q ji for i, j ∈ I . The minimal
semigroup related to Q is a stochastic semigroup on l 1 if and only if the equation
Q ∗ w = λw has no nonzero solution w ∈ l ∞ and w ≥ 0.

A Kolmogorov matrix Q is called non-explosive if the minimal semigroup related to

Q is stochastic.
4.2 Stochastic Semigroups for PDMPs 99

4.2.2 Semigroups for Semiflows

Let (X, Σ, m) be a σ -finite measure space. Consider a measurable semiflow on X ,

i.e. π : [0, ∞) × X → X is such that the mapping (t, x) → π(t, x) is measurable
and the transformations πt : X → X , where πt (x) = π(t, x) for x ∈ X and t ≥ 0,
satisfy the semigroup property

π0 (x) = x and πs+t (x) = πt (πs (x)), t, s ≥ 0, x ∈ X.

We assume that for each t > 0 the transformation πt is non-singular with respect
to m. Let P0 (t) be the Frobenius–Perron operator on L 1 corresponding to πt (see
Sect. 2.1.5). It satisfies
 
1 B (πt (x)) f (x) m(d x) = P0 (t) f (x) m(d x), B ∈ Σ, f ∈ L 1 ,
X B

which, by linearity and approximations, can be extended to

 
g(πt (x)) f (x) m(d x) = g(x)P0 (t) f (x) m(d x) (4.14)
X X

for all measurable and non-negative g, as well as for all g ∈ L ∞ and f ∈ L 1 . Since
{πt }t≥0 is a semigroup on X , it follows from (4.14) that P0 (0) = I and P0 (t + s) =
P0 (t)P0 (s) for all s, t ≥ 0. Thus the family {P0 (t)}t≥0 is a semigroup on L 1 .
We give sufficient conditions for the semigroup of Frobenius–Perron operators to
be a stochastic semigroup. The proof of the following is based on [45].

Theorem 4.12 Assume that L 1 is separable. If the semigroup {P0 (t)}t≥0 of Frobe-
nius–Perron operators corresponding to {πt }t≥0 satisfies P0 (t)| f | = |P0 (t) f | for all
f ∈ L 1 and t > 0, then {P0 (t)}t≥0 is a stochastic semigroup.

Proof We first prove that for any f ∈ L 1 the function t → P0 (t) f is continuous at
each s > 0. If g ∈ L ∞
+ and f ∈ L + then the function (t, x)  → g(π(t, x)) f (x) is
1

measurable and non-negative, thus the function


t → g(π(t, x)) f (x) m(d x)
X

is Borel measurable, by Fubini’s theorem. This together with (4.14) implies that for
any f ∈ L 1 the function

t → g, P0 (t) f = g(x)P0 (t) f (x) m(d x)
X

is Borel measurable for each g ∈ L ∞ . Consequently, the function t → P0 (t) f is

weakly measurable. The space L 1 is separable. By Dunford’s theorem [33], the
100 4 Stochastic Semigroups

function t → P0 (t) f is continuous at s > 0. Next, we prove that

P0 (t) f − f  = P0 (t + s) f − P0 (s) f 

for any t, s > 0 and f ∈ L 1 . Since π(s, x) ∈ X for all x ∈ X and s > 0, we have

P0 (t) f − f  = 1 X (π(s, x))|P0 (t) f (x) − f (x)| m(d x),
X

and, by (4.14), we obtain


P0 (t) f − f  = P0 (s)|P0 (t) f (x) − f (x)| m(d x).
X

By assumption P0 (s)|P0 (t) f − f | = |P0 (t + s) f − P0 (s) f |, which gives the claim

and implies that t → P0 (t) f is continuous at 0 for all f ∈ L 1 .

We now assume that X is a separable metric space and that Σ = B(X ) is the
σ -algebra of Borel subsets of X . In that case the space L 1 = L 1 (X, B(X ), m) is
separable for any σ -finite measure m on (X, B(X )). If π : [0, ∞) × X → X is a
semiflow as in Sect. 1.19, then the mapping (t, x) → π(t, x) is measurable. More-
over, if πt is non-singular and one to one then the Frobenius–Perron operator P0 (t)
for the transformation πt is of the form given in Sect. 2.1.5 and satisfies the condition
|P0 (t) f | = P0 (t)| f | for f ∈ L 1 .
Recall that the survival function t → Φx (t) is right-continuous with Φx (0) = 1
and that for each t ≥ 0 the mapping x → Φx (t) is measurable. Define the operators
S(t), t ≥ 0, on L 1 by
S(t) f = P0 (t)(Φ(t) f ), (4.15)

for t ≥ 0, f ∈ L 1 , where we write Φ(t) for the function x → Φx (t). From (4.14) it
follows that
 
Φx (t)1 B (πt (x)) f (x) m(d x) = S(t) f (x) m(d x) (4.16)
X B

for all t ≥ 0, f ∈ L 1+ , B ∈ Σ.

Theorem 4.13 Suppose that {S(t)}t≥0 is defined as in (4.15) and satisfies (4.16). If
{P0 (t)}t≥0 is a substochastic semigroup then {S(t)}t≥0 is a substochastic semigroup.

Proof Since 0 ≤ Φ(t) ≤ 1, we obtain

0 ≤ S(t) f ≤ P0 (t) f

for f ∈ L 1+ . Thus, each operator S(t) is substochastic. To show the semigroup prop-
erty, we use the equation
4.2 Stochastic Semigroups for PDMPs 101

Φx (t + s) = Φπs (x) (t)Φx (s), x ∈ X, t, s ≥ 0.

This and (4.14) imply that for any measurable set B and any f ∈ L 1+ we have
 
P0 (s)(Φ(t + s) f )(x) m(d x) = 1 B (πs (x))Φπs (x) (t)Φx (s) f (x) m(d x)
B
X
= 1 B (x)Φx (t)P0 (s)(Φ(s) f )(x) m(d x).
X

Consequently, P0 (s)(Φ(t + s) f ) = Φ(t)P0 (s)(Φ(s) f ) for s, t ≥ 0, f ∈ L 1+ , which

shows that

P0 (t + s)(Φ(t + s) f ) = P0 (t)(Φ(t)P0 (s)(Φ(s) f )) = S(t)(S(s) f ).

Finally, observe that

S(t) f − f  ≤ Φ(t) f − f  + P0 (t) f − f 

and that Φ(t) f − f  → 0 as t ↓ 0, which implies the strong continuity of the

semigroup {S(t)}t≥0 .
Remark 4.7 We can use results of this section for the semiflow π : [0, ∞) × X̃ → X̃
as in Sect. 1.19 with X̃ being an extension of the state space X . If the assumptions
of Theorem 4.12 hold for the semigroup { P̃0 (t)}t≥0 satisfying
 
g(πt (x)) f (x) m̃(d x) = g(x) P̃0 (t) f (x) m̃(d x)
X̃ X̃

for f ∈ L 1 ( X̃ , Σ̃, m̃) and g ∈ L ∞ ( X̃ , Σ̃, m̃), then the restriction of { P̃0 (t)}t≥0 to
X is a substochastic semigroup {P0 (t)}t≥0 on L 1 = L 1 (X, Σ, m). In particular, we
have
 
1[0,t∗ (x)) (t)g(πt (x)) f (x) m(d x) = g(x)P0 (t) f (x) m(d x), f ∈ L 1, g ∈ L ∞,
X X

where t∗ (x) is the exit time from the set X as defined in (1.39). If the survival function
Φ is given by (1.41) then the semigroup {S(t)}t≥0 satisfies (4.16).

4.2.3 PDMPs Without Boundaries

In this section we show how we can define stochastic semigroups for a large class
of PDMPs. Let (X, Σ, m) be a σ -finite measure space, where X is a separable
metric space and Σ = B(X ) is the σ -algebra of Borel subsets of X . The character-
istics (π, Φ, P) define a PDMP process ξ = {ξ(t) : t ≥ 0} with state space X , where
102 4 Stochastic Semigroups

π : [0, ∞) × X → X is a semiflow as in Sect. 1.19, Φ is a survival function, and P

is a jump distribution. The semiflow is measurable. To simplify the presentation we
now assume that the active boundary is empty and also that the survival function Φ
is defined by
  t
Φx (t) = exp − ϕ(πr (x)) dr , x ∈ X, t ≥ 0, (4.17)
0

where ϕ is a bounded measurable function such that for each x ∈ X , ϕ(πt (x)) →
ϕ(x) as t → 0.
Theorem 4.14 Let {P0 (t)}t≥0 be a stochastic semigroup of Frobenius–Perron oper-
ators corresponding to {πt }t≥0 . Suppose that {S(t)}t≥0 is a substochastic semigroup
as defined in (4.15) with Φx (t) as in (4.17). Then the generator of {S(t)}t≥0 is given
by
A1 f = A0 f − ϕ f for f ∈ D(A0 ), (4.18)

where (A0 , D(A0 )) is the generator of {P0 (t)}t≥0 .

Proof It follows from Theorem 4.7 that the operator (A1 , D(A0 )) is the generator of a
substochastic semigroup. By Proposition 3.1, it is enough to show that the generator
of the semigroup {S(t)}t≥0 is an extension of (A1 , D(A0 )). Take f ∈ D(A0 ) and
observe that
1 1
(S(t) f − f ) − A1 f ≤ (P0 (t) f − f ) − A0 f + ϕ f − P0 (t)(ϕ f )
t t
1
+ P0 (t) (Φ(t) − 1) f + ϕ f .
t
We have
1
lim (Φx (t) − 1) = −ϕ(x), x ∈ X,
t↓0 t

which implies, by the dominated convergence theorem, that

1
lim (Φ(t) − 1) f + ϕ f = 0.
t↓0 t

Since {P0 (t)}t≥0 is a stochastic semigroup with generator A0 , we conclude that the
generator of the semigroup {S(t)}t≥0 is an extension of the operator (A1 , D(A0 )).
Let the transition probability P correspond to a stochastic operator P(x, B) on
L 1 = L 1 (X, Σ, m). It follows from Theorem 4.7 that the operator

A f = A0 f − ϕ f + P(ϕ f ), f ∈ D(A0 ),

is the generator of a stochastic semigroup {P(t)}t≥0 . According to the Dyson–Phillips

formula (3.13), we have
4.2 Stochastic Semigroups for PDMPs 103

∞
  t
P(t) f = Sn (t) f, where Sn (t) f = Sn−1 (t − s)P(ϕ S(s) f ) ds (4.19)
n=0 0

for every n ≥ 1, and S0 (t) = S(t) with {S(t)}t≥0 as in (4.15). We now show that the
semigroup {P(t)}t≥0 corresponds to the transition function induced by the PDMP
ξ as constructed in Sect. 2.3.5. To this end it is enough to show (4.9). Recall from
Sect. 2.3.6 that

P(t, x, B) = Px (ξ(t) ∈ B) = lim Tn (t)1 B (x), x ∈ X, B ∈ Σ, t > 0,

n→∞

where Tn (t) is defined in (2.33). Since T0 (t)1 B (x) = 1 B (π(t, x))Φx (t), x ∈ X ,
Eq. (4.16) implies that
 
T0 (t)1 B (x) f (x) m(d x) = 1 B (x)S(t) f (x) m(d x)
X X

for all f ∈ L 1+ . It follows from (4.19) that

 t
T0 (s)(ϕT (T0 (t − s)1 B ))(x) f (x) m(d x) ds
0 X
 t 
= 1 B (x)S0 (t − s)P(ϕ S(s) f )(x) m(d x) ds = 1 B (x)S1 (t) f (x) m(d x)
0 X X

and, by (2.34), we obtain

  
n
Tn (t)1 B (x) f (x) m(d x) = 1 B (x) Sk (t) f (x) m(d x)
X X k=0

for n = 1, which is easy to extend to all n, by induction. The dominated convergence

theorem and the convergence in (4.19) give
 
P(t, x, B) f (x) m(d x) = lim Pn (t, x, B) f (x) m(d x)
X n→∞ X
 
n 
= lim Sk (t) f (x) m(d x) = P f (x) m(d x).
n→∞ B k=0 B

4.2.4 Dynamical Systems with Jumps

We consider a differential equation

x  (t) = b(x(t)) (4.20)

104 4 Stochastic Semigroups

on some Borel subset X ⊆ R d with non-empty interior and with boundary of the
Lebesgue measure zero. We assume that b is a locally Lipschitz function on X and that
for each x0 ∈ X Eq. (4.20) with initial condition x(0) = x0 has a solution x(t) ∈ X
for all t > 0 and denote it by πt (x0 ). We assume that a point moves according to this
flow, i.e. if x0 ∈ X is the initial position of a point, then x = πt (x0 ) is its location at
time t. The point can jump with intensity which is a bounded continuous function
ϕ : X → [0, ∞). Let a transition probability function P(x, B) describe the new
position of the point after a jump from x and let L 1 = L 1 (X, B(X ), m) where m is
the Lebesgue measure on X . We assume that P(x, B) satisfies (2.4) and defines a
stochastic operator P on L 1 , which corresponds to the transition probability P(x, B).
If the random variable ξ(t) is the position of the moving point at time t and ξ(0) has
density u(0), then ξ(t) has density u(t) which satisfies the evolution equation

d
∂
u  (t) = Au(t) = A0 u(t) − ϕu(t) + P(ϕu(t)), A0 u(t) = − (bi u).
i=1
∂ xi
(4.21)
As in Sect. 4.2.1 we define
 
λ = sup{ϕ(x) : x ∈ X }, P̄ f (x) = λ−1 P(ϕ f )(x) + (λ − ϕ(x)) f (x) .

Then P̄ is a stochastic operator and A = A0 − λI + λ P̄. The operator A0 is the

generator of a stochastic semigroup {P0 (t)}t≥0 on L 1 (X, B(X ), m) given by
⎧  d 
⎨
f (π−t x) det π−t x , if x ∈ πt (X ),
P0 (t) f (x) = dx
⎩0, if x ∈
/ πt (X ),

where the mapping x → π−t (x) is the inverse of the one to one mapping πt : X →
πt (X ). The semigroup {P(t)}t≥0 with the generator A can be given by the Dyson–
Phillips expansion (3.15) in Corollary 3.2.

4.2.5 Randomly Switched Dynamical Systems

We consider here the simplest case when the number of different dynamical systems is
finite. Precisely, we consider a family of systems of differential equations x  = bi (x),
i ∈ I = {1, . . . , k}, defined on some open set G ⊆ Rd . We assume that all functions
bi : G → Rd are locally Lipschitz functions on G. Let X ⊆ G be a Borel set with
non-empty interior and with boundary of Lebesgue measure zero. Let x0 ∈ X and
t ≥ 0. We denote by πti (x0 ) the solution x(t) of the system x  = bi (x) with the initial
condition x(0) = x0 provided that such a solution exists in the time interval [0, t]
and x(s) ∈ X for s ∈ [0, t]. The state of the system is a pair (x, i) ∈ X = X × I or
the dead state ∗ which is an extra point outside the space X.
4.2 Stochastic Semigroups for PDMPs 105

If the system is at state (x, i) then we can jump to the state (x, j) with a bounded
and continuous intensity q ji (x). Between two consecutive jumps the trajectory start-
ing from the state (x, i) is deterministic and at time t it reaches the state (πti (x), i)
or the dead state ∗ if πti (x) is not defined and then the system spends the rest of the
time in the dead state.
Consider the differential equation

d d
x(t) = bi(t) (x(t)), i(t) = 0. (4.22)
dt dt
Its solutions are solutions of
d
(x(t), i(t)) = b(x(t), i(t)), (4.23)
dt

where the mapping b defined by b(x, i) = (bi (x), 0) for (x, i) with x ∈ G is locally
Lipschitz continuous. We take the semiflow π : [0, ∞) × X → X ∪ {∗} to be

π(t, x, i) = (πti (x), i), x ∈ X, i ∈ I,

as long as πti (x) exists and belongs to X ; otherwise we set π(t, x, i) = ∗. The jump
rate function ϕ : X → [0, ∞) is given by

ϕ(x, i) = qi (x) = q ji (x),
j=i

and the survival function is

  t
Φ(x,i) (t) = exp − qi (πsi (x)) ds for t < t∗ (x, i),
0

and Φ(x,i) (t) = 0, otherwise. The jump distribution P is described at the end of
Sect. 2.1.6 with the probability of jump from a state (x, i) to the state (x, j) with
j = i defined by 
1, qi (x) = 0, j = i,
p ji (x) = q ji (x)
qi (x)
, qi (x) > 0, j = i,

and the corresponding stochastic operator P on L 1 by


P f (x, i) = pi j (x) f (x, j).
j=i

We can define a family of time homogeneous Markov processes ξ(t) on X ∪ {∗}

setting ξ(t) = (x(t), i(t)) or ξ(t) = ∗ if we have entered the dead state before time
t.
106 4 Stochastic Semigroups

One can also give an analytic definition of the semigroup {P(t)}t≥0 by using a per-
turbation theorem and an analytic formula for a substochastic semigroup generated
by a single semiflow (see e.g. [96]).

4.2.6 Jumps from Boundaries

In this section we show how the perturbation result from Sect. 3.3.4 can be used in
an example with only forced jumps. Consider the Lebowitz–Rubinow model from
Sect. 1.7. We take X = [0, 1) × (0, 1] and X̃ = R2 . The solution of (1.4) with initial
condition (x(0), v(0)) = (x, v) is of the form

π(t, x, v) = (x + tv, v), x, v, t ∈ R.

We have
1−x
Γ = {1} × (0, 1], t∗ (x, v) = , x, v ∈ (0, 1].
v

The survival function is given by Φ(x,v) (t) = 1[0,t∗ (x,v)) (t) for (x, v) ∈ X and the
transition probability satisfies
 
P((1, v), {0} × B) = P(v, dv ) = αδv (B) + β k(v , v) dv , B ∈ B((0, 1]),
B B

where k is an integral probability kernel on (0, 1) and α, β are non-negative constants

with α + β = 1. The extended generator  L of the process ξ is given by

 ∂g(x, v)
Lg(x, v) = v , (x, v) ∈ [0, 1] × (0, 1],
∂x
for all bounded functions which are continuously differentiable in the first variable,
supplemented with the boundary condition
 1
g(1, v) = g(0, v )P(v, dv ). (4.24)
0

We show that the process ξ(t) = (x(t), v(t)) induces a stochastic semigroup
{P(t)}t≥0 on the space L 1 = L 1 (X, Σ, m), where Σ = B(X ), and m(d x × dv) =
d x × dv. The infinitesimal generator of this semigroup is of the form

∂f
A f (x, v) = −v (x, v) (4.25)
∂x
and the functions f from the domain of A satisfy the boundary condition
4.2 Stochastic Semigroups for PDMPs 107
 1
   
v h(0, v ) = αv h(1, v ) + β vk(v , v)h(1, v) dv. (4.26)
0

We apply Theorem 4.6. Consider the maximal operator

∂f
A f (x, v) = −v (x, v), f ∈ D = { f ∈ L 1 : A f ∈ L 1 },
∂x

and the space D with Sobolev norm

 f  A =  f  + A f , f ∈ D.

We have the Green formula

  
A f (x, v) m(d x, dv) = γ0 f (x, v) m 0 (d x, dv) − γ1 f (x, v) m 1 (d x, dv)
X Γ0 Γ1

for f ∈ D, where the boundaries are

Γa = {a} × (0, 1] and m a (d x, dy) = vδa (d x) dv, a = 0, 1,

and the trace operators γa : f → f (a, ·) for a = 0, 1, defined on (D,  ·  A ) and

with values in L a1 = L 1 (Γa , B(Γa ), m a ), are linear and continuous. We define the
boundary operator Ψ : D → L 10 by Ψ f = P(γ1 f ) and the operator P : L 11 → L 10 by
 1
1
P f (0, v ) = α f (1, v ) + β k(v , v) f (1, v) v dv.
v 0

Observe that P preserves the integral on the positive cone so that we have
 
Ph(x, v) m 0 (d x, dv) = h(x, v) m 1 (d x, dv), h ∈ L 11 , h ≥ 0.
Γ0 Γ1

The inverse operator to the boundary operator Ψ0 f = γ0 f , f ∈ D, when restricted

to the nullspace N (λI − A) of the operator (λI − A, D) is given by
λx
Ψ (λ) f 0 (x, v) = f 0 (0, v)e− v , (x, v) ∈ X, f 0 ∈ L 10 .

We have P(γ1 Ψ (λ)) : L 10 → L 10 and

 1
Pγ1 Ψ (λ) f 0  = |Pγ1 Ψ (λ) f 0 (0, v)|v dv ≤ e−λ  f 0 , f 0 ∈ L 10 .
0

The operator
∂f
A0 f (x, v) = −v (x, v)
∂x
108 4 Stochastic Semigroups

with domain
D(A0 ) = { f ∈ D : γ0 f = 0}

is the generator of the substochastic semigroup {S(t)}t≥0 given by

S(t) f (x, v) = f (x − tv, v)1(tv,1] (x), f ∈ L 1.

For each t > 0 the operator S(t) is the restriction to X of the Frobenius–Perron
operator P0 (t) for the transformation πt : R2 → R2 given by πt (x, v) = (x + tv, v)
and defined on L 1 (R2 , B(R2 ), m).
For any λ > 0 the resolvent of (A, D(A)) at λ with

D(A) = { f ∈ D : γ0 f = Pγ1 f }

is a positive operator and for f ∈ D(A)+ we have

  
A f (x, v) d x dv = γ0 f (x, v) m 0 (d x, dv) − γ1 f (x, v) m 1 (d x, dv)
X Γ0 Γ1
 
= P(γ1 f )(x, v) m 0 (d x, dv) − γ1 f (x, v) m 1 (d x, dv) = 0.
Γ0 Γ1

From Theorem 4.6 it follows that A is the generator of a stochastic semigroup.

It remains to show that the semigroup {P(t)}t≥0 corresponds to the transition
function induced by the process ξ . It follows from (4.24) that
 
f (x, v)g(x, v) m 1 (d x, dv) = g(x, v)P f (x, v) m 0 (d x, dv ).
Γ1 Γ0

Using the Green formula, we see that

 
A f (x, v)g(x, y) m(d x, dv) = f (x, v)
L g(x, v) m(d x, dv).
X X

It is enough to show that a restriction L of the operator L̃ is the generator of a positive

contraction semigroup on a closed subspace of B(X ). To this end we make use of
Theorem 4.10 and Lemma 3.4. Let

X = {g ∈ B(X ) : g(·, v) ∈ C[0, 1], v ∈ (0, 1]},

where C[0, 1] is the space of continuous functions on [0, 1]. Consider the operator

L on the maximal domain

 = {g ∈ X : g is continuously differentiable in x and L̃ ∈ X }.

D

0 , Ψ
We define the boundary operators Ψ  : B(X ) → B(Γ ), where Γ = Γ1 , by
4.2 Stochastic Semigroups for PDMPs 109
 1
0 g(1, v) = g(1, v), Ψ
Ψ  g(1, v) = g(0, v )P(v, dv ), v ∈ (0, 1], g ∈ B(X ).
0

(λ) of the operator Ψ

The inverse Ψ 0 restricted to the nullspace of the operator (λI −
  is given by
L, D)
(λ)g(x, v) = g(1, v)e− λv (1−x) .
Ψ

It is a positive operator and the operator I − ΨΨ(λ) is invertible for each λ > 0.
The operator L 0 being the restriction of  0 ) is resolvent positive, since
L to N (Ψ
 1
λx λr
R(λ, L 0 )g(x, v) = e v e− v f (r, v) dr, g ∈ X .
x

Consequently, the restriction L of the extended generator 

L defined on

D(L) = {g ∈ X : g satisfies condition (4.24) and 

Lg ∈ X }

is resolvent positive. Since the boundary condition (4.24) holds for g = 1 X and
L1 X = 0, we conclude from Theorem 4.10 that L generates a positive contraction
semigroup.

4.2.7 Semigroups for the Stein Model

In this section we show how we can generate a stochastic semigroup for a particular
example of a PDMP described in Sect. 1.11 where we considered the Stein model. As
in [91] we introduce an additional 0-phase, which starts at the end of the refractory
period and lasts until depolarization takes place. Thus we consider three phases
0, 1, 2. Recall that in phase 1 the depolarization x(t) decays according to the equation
x  (t) = −αx(t) with α > 0 and phase 2 denotes the refractory phase of duration t R
(see Fig. 4.1). We take X̃ = R × {0, 1, 2}. The flow π : R × X̃ → X̃ is given by

π(t, x, 0) = (x, 0), π(t, x, 1) = (xe−αt , 1), π(t, x, 2) = (x + t, 2), t, x ∈ R.

The process is defined on the space X = {(0, 0)} ∪ (−∞, θ ] × {1} ∪ [0, t R ) × {2}
and the active boundary is Γ = {(t R , 2)} with t∗ (x, 2) = t R − x for x ∈ [0, t R ). We
take
Φ(x,i) (t) = e−(λ E +λ I )t , i = 0, 1,

and Φ(x,2) (t) = 1[0,t∗ (x,2)) (t), so that the jump rate function ϕ is defined by ϕ(x, i) =
λ E + λ I for i = 0, 1 and ϕ(x, 2) = 0. The jump distribution P is given by

P((x, i), B) = p I 1 B (S I (x, 1)) + p E 1 B (S E (x, 1)), (x, i) ∈ X, i = 0, 1,

110 4 Stochastic Semigroups

1-phase
pI pE pI pE
−xI 0 xE

x−xI x x+xE x−xI x x+xE

0-phase (0, 0)

(tR , 2)

2-phase tR

(0, 2)

Fig. 4.1 Phases in Stein’s model

where
λI λE
pI = , pE = , S I (x, i) = (x − x I , 1),
λE + λI λE + λI

and

S E (x, 1) = (x + x E , 1) for x ≤ θ − x E , S E (x, 1) = (0, 2) for x > θ − x E .

We take P((t R , 2), {(0, 0)}) = 1. The extended generator 

L of the process ξ is given
by


Lg(0, 0) = −(λ E + λ I )g(0, 0) + λ I g(−x I , 1) + λ E g(x E , 0),
 ∂g(x, 1)
Lg(x, 1) = −αx − (λ E + λ I )g(x, 1) + λ I g(x − x I , 1)
∂x
+ λ E g(x + x E , 1)1(−∞,θ] (x + x E ) + λ E g(0, 2)1(θ,∞) (x + x E ),
 ∂g(x, 2)
Lg(x, 2) = ,
∂x
for all bounded functions which are differentiable in the first variable, supplemented
with the boundary condition

g(t R , 2) = g(0, 0).

To find the stochastic semigroup on L 1 we apply Theorem 4.6. Let m be a measure

on the σ -algebra B(X ) of the Borel subsets of X such that m({(0, 0)}) = δ(0,0) , the
Dirac measure at (0, 0), m restricted to (−∞, θ ] × {1} is the product of the Lebesgue
measure on the interval (−∞, θ ] and the Dirac measure at {1} and m restricted to
4.2 Stochastic Semigroups for PDMPs 111

[0, t R ] × {2} is the product of the Lebesgue measure on the interval [0, t R ] and the
Dirac measure at {2}. We define the operator P on L 1 = L 1 (X, B(X ), m) by

P f (x, 1) = p E f (x − x E , 1) + p I f (x + x I , 1)1(−∞,θ) (x + x I ),

and P f (x, 0) = P f (x, 2) = 0. We consider the process separately on (−∞, −x I ) ×

{1}, [−x I , x E ] × {1} and (x E , θ ] × {1}. We define the maximal operator

A1 f (0, 0) = −(λ E + λ I ) f (0, 0),

∂(αx f (x, 1))
A1 f (x, 1) = − (λ E + λ I ) f (x, 1),
∂x
∂ f (x, 2)
A1 f (x, 2) = − ,
∂x

with domain D which consists of all functions f ∈ L 1 such that A1 f ∈ L 1 and

f is absolutely continuous on the segments (−∞, −x I ) × {1}, (−x I , x E ) × {1},
(x E , θ ) × {1}, (0, t R ) × {2} and f (θ − , 1) = 0, where f (θ − , 1) is the left limit at θ
of the absolutely continuous function f (x, 1). Note that we have
 θ
A1 f (x, 1) d x = −αx I f (−x − + − +
I , 1) + αx I f (−x I , 1) + αx E f (x E , 1) − αx E f (x E , 1)
−∞
 θ
− (λ E + λ I ) f (x, 1) d x
−∞

and  tR
A1 f (x, 2) d x = f (0+ , 2) − f (t R− , 2).
0

We take

Γ− = {(0, 2), (x E , 1), (−x I , 1)}, m − = δ(0,2) + αx E δ(x E ,1) + αx I δ(−x I ,1) ,

and

Γ+ = {(t R , 2), (x E , 1), (−x I , 1)}, m + = δ(t R ,2) + αx E δ(x E ,1) + αx I δ(−x I ,1) .

Note that Γ+ contains all points through which the flow π can exit the segments
(−∞, −x I ) × {1}, (−x I , x E ) × {1}, (x E , θ ) × {1}, (0, t R ) × {2}, in a finite positive
time, while the set Γ− contains points through which the flow can exit these segments
in a finite negative time. We define the traces γ− : D → L 1 (Γ− ) by

γ− f (0, 2) = f (0+ , 2), γ− f (x E , 1) = f (x E− , 1), γ− f (−x I , 1) = f (−x I+ , 1),

and γ+ : D → L 1 (Γ+ ) by
112 4 Stochastic Semigroups

γ+ f (t R , 2) = f (t R− , 2), γ+ f (x E , 1) = f (x E+ , 1), γ+ f (−x I , 1) = f (−x I− , 1).

The operator

A0 f = A1 f, f ∈ D(A0 ) = { f ∈ D : γ− f = 0},

is the generator of a substochastic semigroup on L 1 . The inverse operator Ψ (λ) to

the boundary operator Ψ0 f = γ− f restricted to the nullspace N (λI − A1 ) is given
by
Ψ (λ) f (0, 0) = 0, Ψ (λ) f (x, 2) = f (0, 2)e−λx , f ∈ L 1 (Γ− ),

and
⎧ r −1
⎪
⎨ f (x E , 1)
⎪ , x ∈ (0, x E )
x
xE
r −1
Ψ (λ) f (x, 1) = f (−x I , 1) x
, x ∈ (−x I , 0)
⎪
⎪ −x I
⎩
0, x ∈ (−∞, −x I ) ∪ (x E , θ ),

where r = (λ + λ E + λ I )/α. The boundary operator Ψ is


Ψ f (x, 1) = γ+ f (x, 1) + a(x, 1) f (0, 0), Ψ f (0, 2) = λ E f (x, 1) d x,
θ−x E

where
λI λE
a(−x I , 1) = , a(x E , 1) = .
αx I αx E

We have Ψ Ψ (λ) f (x E , 1) = Ψ Ψ (λ) f (−x I , 1) = 0 and

 θ
Ψ Ψ (λ) f (0, 2) = λ E Ψ (λ) f (x, 1) d x
θ−x E

for f ∈ L 1 (Γ− ). Thus, if f ≥ 0 then

λE λE
0 ≤ Ψ Ψ (λ) f (0, 2) ≤ αx E f (x E , 1) ≤  f  L 1 (Γ− ) ,
λ + λE + λI λ + λE + λI

which implies that λΨ Ψ (λ) ≤ λ E . From Theorem 4.6 it follows that the operator
(A1 , N (Ψ − Ψ0 )) is the generator of a substochastic semigroup.
The operator

A f (0, 0) = A1 f (0, 0) + f (t R− , 2),

A f (x, i) = A1 f (x, i) + (λ E + λ I )P f (x, i), i = 1, 2,

with domain
4.2 Stochastic Semigroups for PDMPs 113

D(A) = { f ∈ D : Ψ0 ( f ) = Ψ ( f )}

is closed. Note that if f ∈ D(A)+ then we have


A f (x, i) m(d x, di) = 0,
X

since  
θ θ
A1 f (0, 0) + A1 f (x, 1) d x = −(λ E + λ I ) f (x, 1) d x,
−∞ −∞

and  tR 
f (t R− , 2) + A1 f (x, 2) d x = λ E f (x, 1) d x.
0 θ−x E

It follows from Theorem 4.9 that (A, D(A)) is the generator of a stochastic semi-
group.
It remains to show that a restriction of the extended generator 
L of the process ξ
generates a positive semigroup on a sufficiently large subspace of B(X ). We consider
the subspace X ⊂ B(X ) of all functions g such that g restricted to (−∞, θ − x E ) ×
{1} is uniformly continuous function on (−∞, θ − x E ], g restricted to (θ − x E , θ ] ×
{1} belongs to C[θ − x E , θ ] and g(·, 2) belongs to C[0, t R ]. We define 
L 1 on the
domain

L 1 ) = {g ∈ X : g(t R , 2) = g(0, 0), g is differentiable in x, 

D( L1g ∈ X }

by

L 1 g(0, 0) = −(λ E + λ I )g(0, 0),
 ∂g(x, 1)
L 1 g(x, 1) = −αx − (λ E + λ I )g(x, 1),
∂x
 ∂g(x, 2)
L 1 g(x, 2) = .
∂x

It is easy to see that the operator 

L 1 is resolvent positive. In particular, we have
 tR
R(λ, 
L 1 )g(x, 2) = eλ(x−t R ) g(0, 0) + eλx e−λy g(y, 2) dy
x

and
 x
1 −r x 1 ry
R(λ, 
L 1 )g(x, 1) = er (θ−x E −x) g((θ − x E )− , 1) + e e g(y, 1) dy
α θ−x E y

for x > θ − x E , where r = (λ + λ E + λ I )/α. We can write 

L as 
L1 + 
B, where
114 4 Stochastic Semigroups

 

Bg(x, i) = λ I g(x − x I , 1) + λ E g(x + x E , 1)1(−∞,θ−x E ] (x) + g(0, 2)1(θ−x E ,θ] (x)

for (x, i) ∈ X , i = 0, 1, and 

Bg(x, 2) = 0 for (x, 2) ∈ X . We consider the operator

B on the domain
D( B) = {g ∈ X : g is continuous}.

Since B is a positive operator and the operator I − 

B R(λ, 
L 1 ) is invertible for all
sufficiently large λ, the operator L = L with domain D(L) = D( L 1 ) is resolvent
positive. We have 1 X ∈ D(L) and L1 X = 0. It follows from Theorem 4.10 that L
generates a positive contraction semigroup on the closure of D(L).
Chapter 5
Asymptotic Properties of Stochastic
Semigroups—General Results

In the theory of stochastic processes a special role is played by results concerning

the existence of invariant densities and the long-time behaviour of their distribu-
tions. These results can be formulated and proved in terms of stochastic semigroups
induced by these processes. We consider two properties: asymptotic stability and
sweeping. A stochastic semigroup induced by a stochastic process is asymptotically
stable if the densities of one-dimensional distributions of this process converge to
a unique invariant density. Sweeping is an opposite property to asymptotic stability
and it means that the probability that trajectories of the process are in a set Z goes to
zero. The main result presented here shows that under some conditions a substochas-
tic semigroup can be decomposed into asymptotically stable parts and a sweeping
part. This result and some irreducibility conditions allow us to formulate theorems
concerning asymptotic stability, sweeping and the Foguel alternative. This alternative
says that under suitable conditions a stochastic semigroup is either asymptotically
stable or sweeping.

5.1 Asymptotic Stability and Sweeping

5.1.1 Definitions of Asymptotic Stability and Sweeping

Let the triple (X, Σ, m) be a σ-finite measure space. In the whole chapter, we will
consider substochastic operators and semigroups on L 1 = L 1 (X, Σ, m) correspond-
ing to transition kernels. The support of any measurable function f is defined up to
a set of measure zero by the formula

supp f = {x ∈ X : f (x) = 0}.

© The Author(s) 2017 115

R. Rudnicki and M. Tyran-Kamińska, Piecewise Deterministic Processes
in Biological Models, SpringerBriefs in Mathematical Methods,
DOI 10.1007/978-3-319-61295-9_5
116 5 Asymptotic Properties of Stochastic Semigroups—General Results

Recall that a positive semigroup {P(t)}t≥0 is stochastic if and only if P ∗ (t)1 X = 1 X

for t ≥ 0, where P ∗ (t) is the adjoint of P(t). A density f ∗ is called invariant
with respect to a substochastic semigroup if P(t) f ∗ = f ∗ for t ≥ 0. Observe that
if a substochastic semigroup {P(t)}t≥0 has an invariant density f ∗ > 0 a.e. then
{P(t)}t≥0 is a stochastic semigroup. Indeed,
  
P ∗ (t)1 X (x) f ∗ (x) m(d x) = 1 X (x)P(t) f ∗ (x) m(d x) = f ∗ (x) m(d x),
X X X

but since P ∗ (t)1 X ≤ 1 X and f ∗ > 0 a.e., we conclude that P ∗ (t)1 X = 1 X for all
t ≥ 0.
A substochastic semigroup {P(t)}t≥0 is called quasi-asymptotically stable if
there exist an invariant density f ∗ and a positive linear functional α defined on
L 1 (X, Σ, m) such that
lim P(t) f − α( f ) f ∗ = 0 for f ∈ L 1 . (5.1)
t→∞

If the semigroup satisfies (5.1) with α( f ) = 1 for f ∈ D, then it is called asymp-

totically stable [62]. If a substochastic semigroup {P(t)}t≥0 is asymptotically stable
then it is a stochastic semigroup, and the notions of quasi-asymptotic stability and
asymptotic stability are equivalent for stochastic semigroups.
If the semigroup {P(t)}t≥0 is generated by an evolution equation u (t) = Au(t)
then the asymptotic stability of {P(t)}t≥0 means that the stationary solution u(t) =
f ∗ is asymptotically stable in the sense of Lyapunov and this stability is global on
the set D.
A substochastic semigroup {P(t)}t≥0 is called sweeping [58] with respect to a set
Z ∈ Σ if for every f ∈ L 1 we have

lim P(t) f (x) m(d x) = 0. (5.2)
t→∞ Z

We usually consider sweeping from a family of sets, e.g. from sets of finite measure
m or from all compact subsets of X if X is a topological space.
Sweeping is also called zero type or null property [79]. Asymptotic stability is also
called strong mixing or exactness, especially in the context of ergodic theory [62].
We can also introduce quasi-asymptotic stability, asymptotic stability and sweeping
of a substochastic operator P if in (5.1) and (5.2) we replace P(t) with the iterates
P t of the operator P.

5.1.2 Lower Function Theorem

There are several results concerning asymptotic stability [62]. Some of them are based
on assumptions concerning boundedness from below of a stochastic semigroup. One
of them is the lower function theorem of Lasota and Yorke [63]. A function h ∈ L 1 ,
h ≥ 0 and h = 0, is called a lower function for a stochastic semigroup {P(t)}t≥0 if
5.1 Asymptotic Stability and Sweeping 117

lim (P(t) f − h)− = 0 for every f ∈ D. (5.3)

t→∞

Here we use the notation f − (x) = 0 if f (x) ≥ 0 and f − (x) = − f (x) if f (x) < 0.
Condition (5.3) can be equivalently written as: there are functions ε(t) ∈ L 1 such
that limt→∞ ε(t) = 0 and P(t) f ≥ h − ε(t). Observe that if the semigroup is
asymptotically stable then its invariant density f ∗ is a lower function for it. Lasota
and Yorke [63] proved the following converse result.
Theorem 5.1 Let {P(t)}t≥0 be a stochastic semigroup. If there exists a lower func-
tion h for a stochastic semigroup {P(t)}t≥0 then this semigroup is asymptotically
stable.

5.1.3 Partially Integral Semigroups and Asymptotic Stability

A substochastic semigroup {P(t)}t≥0 is called partially integral (or with a kernel

minorant) if there exists a measurable function k : (0, ∞) × X × X → [0, ∞),
called a kernel, such that

P(t) f (x) ≥ k(t, x, y) f (y) m(dy) (5.4)
X

for every density f and

 
k(t, x, y) m(dy) m(d x) > 0
X X

for some t > 0. If equality holds in (5.4) then {P(t)}t≥0 is said to be an integral
semigroup. The following theorem is only valid for continuous-time semigroups.
Theorem 5.2 Let {P(t)}t≥0 be a partially integral stochastic semigroup. Assume
that the semigroup {P(t)}t≥0 has a unique invariant density f ∗ . If f ∗ > 0 a.e., then
the semigroup {P(t)}t≥0 is asymptotically stable.
The proof of Theorem 5.2 is given in [87] and it is based on the theory of Harris
operators [38, 54].
We now formulate corollaries which are often used in applications. A substochas-
tic semigroup {P(t)}t≥0 is called irreducible if
 ∞
P(t) f dt > 0 a.e. for every density f.
0

We say that a substochastic semigroup {P(t)}t≥0 overlaps supports, if for every

f 1 , f 2 ∈ D there exists t > 0 such that

m(supp P(t) f 1 ∩ supp P(t) f 2 ) > 0.

118 5 Asymptotic Properties of Stochastic Semigroups—General Results

Corollary 5.1 A partially integral and irreducible stochastic semigroup which has
an invariant density is asymptotically stable.
Corollary 5.2 A partially integral stochastic semigroup which overlaps supports
and has an invariant density f ∗ > 0 a.e. is asymptotically stable.
Remark 5.1 The assumption in Theorem 5.2 that the invariant density is unique
can be replaced by an equivalent one: there does not exist a set E ∈ Σ such that
m(E) > 0, m(X \ E) > 0 and P(t)E = E for all t > 0. Here P(t) is the operator
acting on the σ-algebra Σ defined by: if f ≥ 0, supp f = A and supp P(t) f = B
then P(t)A = B. The definition of P(t)A does not depend on the choice of f
because if supp f = supp g a.e. then supp P(t) f = supp P(t)g a.e. Irreducibility
is equivalent to the following: the semigroup has no nontrivial invariant subset. We
recall that a measurable set A is invariant with respect to {P(t)}t≥0 if P(t)(A) ⊆ A
for t ≥ 0 (in terms of the transition kernel it means that P(t, x, X \ A) = 0 for
m-a.e. x ∈ A and t ≥ 0).

5.1.4 Sweeping via the Existence of σ-finite Invariant

Function

We now present results concerning the sweeping property which are based on an
assumption about the existence of a σ-finite positive invariant function. These were
proved in [58] and in [95] for discrete-time stochastic semigroups (iterates of a sto-
chastic operator) but similar results also hold for continuous-time stochastic semi-
groups. Before formulating some results we need to extend a substochastic oper-
ator beyond the space L 1 . If P is a substochastic operator and f is an arbitrary
non-negative measurable function, then we define P f as a pointwise limit of the
sequence P f n , where ( f n ) is any monotonic sequence of non-negative functions
from L 1 pointwise convergent to f almost everywhere (see [38, Chap. I]). Observe
that we can also define an extension of a substochastic operator using a transition
kernel. If P(x, B) is the transition kernel then for any non-negative measurable func-
tion f we have P f = dμ/dm, where μ(B) = X f (x)P(x, B) m(d x) and dμ/dm
is the Radon–Nikodym derivative.
The following condition plays a crucial role in results concerning sweeping:
∗ ∗ ∗
∗ ∗
 ∗ f such that: 0 < f < ∞ a.e., P(t) f ≤
(KT) There exists a measurable function
f for t ≥ 0, f ∈ / L and Z f (x) m(d x) < ∞ for some set Z ∈ Σ with
1

m(Z ) > 0.
Theorem 5.3 ([58]) Let {P(t)}t≥0 be an integral substochastic semigroup which
has no invariant density. Assume that the semigroup {P(t)}t≥0 and a set Z ∈ Σ
satisfy condition (KT). Then the semigroup {P(t)}t≥0 is sweeping with respect to Z .
Unfortunately, substochastic semigroups induced by PDMPs are usually not inte-
gral and we need versions of Theorem 5.3 without assumption (KT). The problem
5.1 Asymptotic Stability and Sweeping 119

is discussed in details in [95] and we present here only some conclusions. In order
to formulate the next theorem we need some auxiliary definitions.
We observe that if a substochastic semigroup {P(t)}t≥0 has a subinvariant function
f ∗ > 0 then we can define a new substochastic semigroup { P̄(t)}t≥0 on the space
L 1 (X, Σ, μ) with dμ = f ∗ dm given by

1
P̄(t) f = P(t)( f f ∗ ) (5.5)
f∗

and P̄(t)1 X ≤ 1 X . Thus, all operators P̄(t) and P̄ ∗ (t) are contractions on
L 1 (X, Σ, μ) and L ∞ (X, Σ, μ), respectively, and, consequently, they are positive
contractions on L 2 (X, Σ, μ). Let us define

K = { f ∈ L2 : P̄(t) f L2 = P̄ ∗ (t) f L2 = f L2 for t ≥ 0}

and
Σ1 = {B ∈ Σ : 1 B ∈ K }.

Let {P(t)}t≥0 be a partially integral substochastic semigroup with a kernel

k(t, x, y). If   ∞
k(t, x, y) dt m(dy) > 0 x − a.e.,
X 0

then {P(t)}t≥0 is called a pre-Harris semigroup.

The following result was proved in [95].
Theorem 5.4 Let {P(t)}t≥0 be a substochastic semigroup which has no invariant
density. Assume that the semigroup {P(t)}t≥0 and a set Z ∈ Σ satisfy condition
(K T ). If one of the following conditions holds:
(a) the family Σ1 is atomic,
(b) {P(t)}t≥0 overlaps supports,
(c) {P(t)}t≥0 is a pre-Harris semigroup,
then the semigroup {P(t)}t≥0 is sweeping with respect to Z .
The main problem with application of Theorem 5.4 is to find a positive subinvariant
function. There are some theoretical results concerning this subject. A substochastic
semigroup can have no strictly positive subinvariant function. But under additional
assumptions related to the notions of conservative and dissipative semigroups we
can find a positive subinvariant function. We precede a formulation of some results
with a definition. Let {P(t)}t≥0 be a substochastic semigroup, f > 0 be a density,
and  ∞
C = {x ∈ X : P(t) f (x) dt = ∞}.
0

The definition of C does not depend on the choice of f . The substochastic semigroup
{P(t)}t≥0 is called conservative if C = X a.e. and dissipative if C = ∅ a.e. From this
120 5 Asymptotic Properties of Stochastic Semigroups—General Results

definition it follows immediately that if {P(t)}t≥0 has an invariant density f ∗ > 0 a.e.,
then {P(t)}t≥0 is conservative. We also have P ∗ (t)1C ≥ 1C for t > 0 or, equivalently,
P(t, x, C) = 1 for x ∈ C. In particular, a conservative substochastic semigroup is
a stochastic semigroup. A substochastic semigroup can be neither conservative nor
dissipative, but if it is an irreducible semigroup then it has one of these properties.
If a stochastic
 ∞ semigroup is dissipative then for any positive density f the function
f ∗ (x) = 0 P(t) f (x) dt is a positive and subinvariant function. The function f ∗ is
also not integrable because in the opposite case the semigroup will be conservative.
If {P(t)}t≥0 is a conservative pre-Harris semigroup, then {P(t)}t≥0 has a strictly
positive invariant function. Even if we know that a semigroup has an subinvariant
function f ∗ , a problem still remains because we need to identify sets Z such that
∗
Z f (x) m(d x) < ∞. In order to do this we need to have more information about
f ∗ —not only its existence. We cannot apply directly the formula A f ∗ ≤ 0 to find f ∗
as in the case of a density because f ∗ ∈ / D(A), where A is the infinitesimal generator
of the semigroup and D(A) is its domain.

5.2 Asymptotic Decomposition of Stochastic Semigroups

The main problem with applying results of Sects. 5.1.3 and 5.1.4 to study asymptotic
stability is that we need to have an invariant density and a positive subinvariant
function to check asymptotic stability and sweeping, respectively. Now we present
a general result concerning asymptotic decomposition of substochastic semigroups
and some corollaries which do not have any assumption on invariant functions. We
hope that these results will prove useful to study asymptotic stability and sweeping
of semigroups induced by PDMPs.

5.2.1 Theorem on Asymptotic Decomposition

From now on, we assume that X is a separable metric space and Σ = B(X ) is the
σ-algebra of Borel subsets of X . We will consider partially integral substochastic
semigroups {P(t)}t≥0 which satisfy the following condition:
(K) For every y0 ∈ X there exist an ε > 0, a t > 0 and a measurable function
η ≥ 0 such that η(x) m(d x) > 0 and

k(t, x, y) ≥ η(x)1 B(y0 ,ε) (y) for x ∈ X, (5.6)

where B(y0 , ε) = {y ∈ X : ρ(y, y0 ) < ε}.

In other words condition (K) holds if for each point x0 ∈ X we find its neighbour-
hood U and time t > 0 such that for all points x ∈ U the transition kernel P(t, x, ·)
can be bounded below by the same nontrivial absolutely continuous measure. Con-
dition (K) was proposed in [95] as a tool to study sweeping of stochastic operators
5.2 Asymptotic Decomposition of Stochastic Semigroups 121

from compact sets. Condition (K) is satisfied if, for example, for every point y ∈ X
there exist a t > 0 and an x ∈ X such that the kernel k(t, ·, ·) is continuous in a
neighbourhood of (x, y) and k(t, x, y) > 0.
The following theorem states that a substochastic semigroup can be decomposed
into asymptotically stable and sweeping components.
Theorem 5.5 Let {P(t)}t≥0 be a substochastic semigroup on L 1 (X, Σ, m), where
X is a separable metric space, Σ = B(X ), and m is a σ-finite measure. Assume
that {P(t)}t≥0 satisfies (K). Then there exist an at most countable set J , a family of
invariant densities { f j∗ } j∈J with disjoint supports {A j } j∈J , and a family {α j } j∈J of
positive linear functionals defined on L 1 (X, Σ, m) such that
(i) for every j ∈ J and for every f ∈ L 1 (X, Σ, m) we have

lim 1 A j P(t) f − α j ( f ) f j∗ = 0, (5.7)

t→∞


(ii) if Y = X \ A j , then for every f ∈ L 1 (X, Σ, m) and for every compact set
j∈J
F we have 
lim P(t) f (x) m(d x) = 0. (5.8)
t→∞
F∩Y

Theorem 5.5 was proved in [89] for stochastic semigroups and generalized in [90]
for substochastic semigroups. Theorem 5.5 unifies a part of the theory of substochas-
tic semigroups related to asymptotic stability and sweeping and generalizes some
earlier results [85, 95].
Since f j∗ is an invariant density with supp f j∗ = A j , the restricted semigroup
{PA j (t)}t≥0 is a stochastic semigroup. Hence, condition (i) implies that {PA j (t)}t≥0 is
asymptotically stable and limt→∞ P(t) f − f j∗ = 0 for f ∈ D such that supp f ⊆
Aj.
Remark 5.2 The sets A j , j ∈ J , which occur in the formulation of Theorem 5.5,
are not only disjoint but also their closures are disjoint. Indeed, let m j be the Borel
measures on X given by

m j (B) = m(B ∩ A j ) for B ∈ Σ.

Denote by F j the topological support of the measure m j , j ∈ J . We recall that the

topological support of m j is the smallest closed set F j such that m j (X \ F j ) = 0
or equivalently F j := {x ∈ X : m j (B(x, ε)) > 0 for all ε > 0}. We check that
F j1 ∩ F j2 = ∅ for j1 = j2 . Suppose that, on the contrary, there is y0 ∈ F j1 ∩ F j2 . Let
t > 0, ε > 0, and a function η in inequality (5.6) be related to y0 . Fix j ∈ { j1 , j2 }.
Let f j∗ be the invariant density with the support A j . From (5.6) it follows that

P(t) f j∗ (x) ≥ η(x) f j∗ (y) m(dy) for x ∈ X.
B(y0 ,ε)
122 5 Asymptotic Properties of Stochastic Semigroups—General Results

Since y0 ∈ F j , we have m(B(y0 , ε) ∩ A j ) > 0 and so B(y0 ,ε) f j∗ (y) m(dy) > 0.
This implies that
supp η ⊆ supp P(t) f j∗ = supp f j∗ = A j .

Hence supp η ⊆ A j1 ∩ A j2 = ∅, a contradiction. Thus F j1 ∩ F j2 = ∅ for j1 = j2 . Since

F j is the topological support of m j , we have m(A j ∩ F j ) = m(A j ) and consequently
we can assume that A j ⊆ F j . Since any metric space is a normal topological space,
every two disjoint closed sets of X have disjoint open neighbourhoods. It means that
for j1 , j2 ∈ J and j1 = j2 we find also disjoint open sets U j1 and U j2 such that
A j1 ⊆ F j1 ⊆ U j1 and A j2 ⊆ F j2 ⊆ U j2 .

5.2.2 Sweeping and the Foguel Alternative

Theorem 5.5 has a number of corollaries concerning asymptotic stability and sweep-
ing. We start with a corollary concerning sweeping:
Corollary 5.3 Let {P(t)}t≥0 be a substochastic semigroup on L 1 (X, Σ, m), where
X is a separable metric space, Σ = B(X ), and m is a σ-finite measure. Assume
that {P(t)}t≥0 satisfies (K) and has no invariant density. Then {P(t)}t≥0 is sweeping
from compact sets.
Condition (K) and irreducibility allow us to express asymptotic properties in the
form of the Foguel alternative, i.e. the stochastic semigroup is asymptotically stable
or sweeping [17]. We use the notion of the Foguel alternative in a narrow sense, when
sweeping is from all compact sets.
Corollary 5.4 Let X be a separable metric space, Σ = B(X ) and m be a σ-finite
measure. If {P(t)}t≥0 is a substochastic semigroup on the space L 1 (X, Σ, m), it
satisfies (K) and it is irreducible, then {P(t)}t≥0 is asymptotically stable or sweeping
from compact sets.

Proof If {P(t)}t≥0 has an invariant density f ∗ , then from irreducibility it follows

that supp f ∗ = X . Thus {P(t)}t≥0 is a stochastic semigroup, J is a singleton and
condition (i) of Theorem 5.5 holds with α( f ) = 1 for f ∈ D. If {P(t)}t≥0 has no
invariant density, then it is sweeping from compact sets.

In particular, if {P(t)}t≥0 is a substochastic semigroup with a continuous kernel k

and k(t, x, y) > 0 for some t > 0 and for all x, y ∈ X , then the Foguel alternative
holds.
An advantage of the formulation of Corollary 5.4 in the form of an alternative
is that: in order to show asymptotic stability of a substochastic semigroup we do
not need to prove the existence of an invariant density. It is enough to check that
the semigroup is non-sweeping. In particular, if the space X is compact and the
substochastic semigroup satisfies (K) then it is asymptotically stable. We can also
eliminate the sweeping by constructing Hasminskiı̆ function (see Sect. 5.2.4).
5.2 Asymptotic Decomposition of Stochastic Semigroups 123

Since in many applications substochastic semigroups are not irreducible, we are

going to formulate another corollary, similar in spirit to Corollary 5.4, but based on
a weaker assumption than irreducibility.
We say that a substochastic semigroup {P(t)}t≥0 is weakly irreducible if:
(WI) There exists a point x0 ∈ X such that for each ε > 0 and for each density f we
have 
P(t) f (x) m(d x) > 0 for some t = t (ε, f ) > 0. (5.9)
B(x0 ,ε)

If a substochastic semigroup {P(t)}t≥0 satisfies conditions (WI) and (K) then the set
J defined as in Theorem 5.5 is an empty set or a singleton.

Corollary 5.5 Let {P(t)}t≥0 be a substochastic semigroup on L 1 (X, Σ, m), where

X is a separable metric space, Σ = B(X ), and m is a σ-finite measure. Assume that
{P(t)}t≥0 satisfies (WI) and (K). Then the semigroup {P(t)}t≥0 is sweeping from
compact sets or there exist an invariant density f ∗ with the support A and a positive
linear functional α defined on L 1 (X, Σ, m) such that
(i) for every f ∈ L 1 (X, Σ, m) we have

lim 1 A P(t) f − α( f ) f ∗ = 0, (5.10)

t→∞

(ii) if Y = X \ A, then for every f ∈ L 1 (X, Σ, m) and for every compact set F
we have 
lim P(t) f (x) m(d x) = 0. (5.11)
t→∞
F∩Y

As a simple corollary we have

Corollary 5.6 If X is a compact space and a substochastic semigroup {P(t)}t≥0

satisfies (WI) and (K), then it is quasi-asymptotically stable or

lim P(t) f (x) m(d x) = 0. (5.12)
t→∞
X

In particular, if X is a compact space, {P(t)}t≥0 is a stochastic semigroup which

satisfies (WI) and (K), then it is asymptotically stable.

We now study asymptotic properties of a substochastic semigroup in the case

when X is not compact. We consider the case when X is a union of a countable
family of compact sets invariant with respect to {P(t)}t≥0 . The following corollary
is a simple consequence of the Foguel alternative for semigroups defined on compact
spaces but it is very useful in applications.
124 5 Asymptotic Properties of Stochastic Semigroups—General Results

Corollary 5.7 Let {P(t)}t≥0 be a substochastic semigroup which satisfies all

assumptions of Corollary 5.5 and X be a union of an increasing family (X n ) of
compact sets. We assume that each set X n is invariant with respect to {P(t)}t≥0
and that the set X 1 includes some neighbourhood of the point x0 from condition
(WI). Then {P(t)}t≥0 is quasi-asymptotically stable or condition (5.12) holds. In
particular, if the semigroup {P(t)}t≥0 is stochastic then it is asymptotically stable.

5.2.3 Asymptotic Stability

We now formulate a theorem concerning asymptotic stability, which is a conse-

quence of Theorem 5.5. We need an auxiliary condition. We say that a substochastic
semigroup {P(t)}t≥0 is weakly tight if:
(WT) There exists κ > 0 such that

sup lim sup P(t) f (x) m(d x) ≥ κ (5.13)
F∈F t→∞ F

for f ∈ D0 , where D0 is a dense subset of D and F is the family of all compact

subsets of X .
The following theorem is a combination of Theorem 2 [91] and Corollary 5 [90].
The proof is similar to that of Corollary 5 [90], but we give it for the sake of com-
pleteness.
Theorem 5.6 Let {P(t)}t≥0 be a substochastic semigroup on L 1 (X, Σ, m), where
X is a separable metric space, Σ = B(X ), and m is a σ-finite measure. Assume that
{P(t)}t≥0 satisfies conditions (WI), (K), and (WT). Then the semigroup {P(t)}t≥0 is
quasi-asymptotically stable. If additionally {P(t)}t≥0 is a stochastic semigroup then
it is asymptotically stable.
Proof From condition (WT) it follows that the semigroup {P(t)}t≥0 is non-sweeping
from all compact sets. Thus, conditions (i) and (ii) of Corollary 5.5 hold. Let A and
Y be the sets defined in Corollary 5.5. We claim that

lim 1Y P(t) f = 0 for f ∈ L 1 (X, Σ, m). (5.14)

t→∞


It is enough to check (5.14) for densities. Then the function β(t) = P(t) f (x) m(d x)
Y
is decreasing. Suppose that, contrary to our claim,

lim P(t) f 1 (x) m(d x) = c > 0 (5.15)
t→∞
Y

for some density f 1 . We show that there exists a density 

f ∈ D0 such that condition
(5.13) does not hold for  f . Indeed, let ε = cκ/4. Then for sufficiently large t0 we
5.2 Asymptotic Decomposition of Stochastic Semigroups 125

have 
P(t0 ) f 1 (x) m(d x) < c + ε.
Y

Let f 2 = 1Y P(t0 ) f 1 . Then


P(t) f 2 (x) m(d x) < ε
A

for t ≥ 0. We define f 3 = f 2 / f 2 . Then f 3 is a density and

 
1
P(t) f 3 (x) m(d x) = P(t) f 2 (x) m(d x) < ε/c = κ/4 (5.16)
A f2 A

for t ≥ 0. Condition (ii) of Corollary 5.5 implies that for each compact set F we
have 
P(t) f 3 (x) m(d x) < κ/4 (5.17)
F∩Y

for sufficiently large t. Let us take  f ∈ D0 such that  f − f 3 ≤ κ/4. From

inequalities (5.16) and (5.17) it follows that for each compact set F we have

3κ
P(t) 
f (x) m(d x) < (5.18)
F 4

for sufficiently large t. Thus (WT) does not hold for 

f , and, consequently, condition
(5.14) is fulfilled. Conditions (5.10) and (5.14) imply that the semigroup {P(t)}t≥0
is quasi-asymptotically stable. If the semigroup {P(t)}t≥0 is stochastic, then quasi-
asymptotic stability is equivalent to asymptotic stability.

5.2.4 Hasminskiı̆ Function

If we know that a stochastic semigroup satisfies the Foguel alternative then we can
eliminate sweeping by constructing a Hasminskiı̆ function [86]. Now we show how
a slight modification of this method can be applied to study weak tightness. It is
clear that a weakly tight stochastic semigroup is non-sweeping from compact sets.
We precede the definition of a Hasminskiı̆ function by some observations.
Let A be a generator of a substochastic semigroup {P(t)}t≥0 and let
 ∞
R := R(1, A) = (I − A)−1 = e−t P(t) dt (5.19)
0

be the resolvent of the operator A. It is clear that R is a substochastic operator and

if {P(t)}t≥0 is a stochastic semigroup, then R is also a stochastic operator.
126 5 Asymptotic Properties of Stochastic Semigroups—General Results

Lemma 5.1 Let {P(t)}t≥0 be a substochastic semigroup,

 f ∈ D and Z ∈ Σ. If
lim supn→∞ Z R n f (x) m(d x) ≥ ε then lim supt→∞ Z P(t) f (x) m(d x) ≥ ε.

Proof Let f ∈ D. From (5.19) it follows that

 ∞
R n+1
f = rn (t)P(t) f dt,
0

t n e−t
where rn (t) = . Suppose that, on the contrary, there exist t0 > 0 and ε < ε
n!
such that 
P(t) f (x) m(d x) ≤ ε for t ≥ t0 .
Z

∞  t0
Since 0 rn (t) dt = 1, 0 rn (t) dt → 0 as n → ∞, and
  t0  ∞ 
R f (x) m(d x) ≤
n
rn (t) dt + rn (t) P(t) f (x) m(d x) dt,
Z 0 t0 Z


we obtain lim supn→∞ Z R n f (x) m(d x) ≤ ε , a contradiction.

Let P be a stochastic operator, Z ∈ Σ and V : X → [0, ∞) be a measurable

function. By DV we denote the set

DV = { f ∈ D : f (x)V (x) m(d x) < ∞}.
X

Since m is a σ-finite measure, the set DV is dense in D. Assume that there exist
M > 0 and ε > 0 such that
  
V (x)P f (x) m(d x) ≤ −ε + V (x) f (x) m(d x) + M P f (x) m(d x) (5.20)
X X Z

for f ∈ DV . Then the function V will be called a Hasminskiı̆ function for the operator
P and the set Z .

Lemma 5.2 Let P be a stochastic operator and Z ∈ Σ. Assume that there exists a
Hasminskiı̆ function for the operator P and the set Z . Then for f ∈ D we have

lim sup P n f (x) m(d x) ≥ ε/2M, (5.21)
n→∞ Z

where ε and M are the constants from formula (5.20).

Proof Let f ∈ DV . Then from (5.20) it follows that P n f ∈ DV for every integer
n ≥ 1. Thus
5.2 Asymptotic Decomposition of Stochastic Semigroups 127
  
V (x)P n+1 f (x) m(d x) ≤ V (x)P n f (x) m(d x) − ε + M P n+1 f (x) m(d x)
X X Z
(5.22)
for f ∈ DV and n ≥ 1. Suppose, contrary to our claim,  that (5.21) does not hold. Then
there exists a positive integer n 0 = n 0 ( f ) such that Z P n+1 f (x) m(d x) < ε/(2M)
for n ≥ n 0 . From (5.22) it follows that
 
ε
V (x)P n+1 f (x) m(d x) ≤ V (x)P n f (x) m(d x) − (5.23)
X X 2

for n ≥ n 0 , and consequently


lim V (x)P n f (x) m(d x) = −∞,
n→∞ X

which is impossible. Thus (5.21) holds for f ∈ DV . Since the set DV is dense in D,
(5.21) is satisfied for all f ∈ D.
Consider a stochastic semigroup {P(t)}t≥0 with the generator A and let R =
(I − A)−1 . A measurable function V : X → [0, ∞) is called a Hasminskiı̆ function
for the semigroup {P(t)}t≥0 and a set Z ∈ Σ if (5.20) holds for P = R. Lemmas 5.1
and 5.2 imply the following
Corollary 5.8 Let {P(t)} be a stochastic semigroup generated by the equation

u (t) = Au(t).

Assume that there exists a Hasminskiı̆ function for this semigroup and a set Z . Then

lim sup P(t) f (x) m(d x) ≥ ε/2M for f ∈ D. (5.24)
t→∞ Z

In particular, the semigroup {P(t)} is non-sweeping with respect to the set Z . If Z

is a compact set then {P(t)} is weakly tight.
Let P = R. Using the formula R = I + A R we can write (5.20) in the following
form
 
V (x)A R f (x) m(d x) ≤ −ε + (M + ε) R f (x) m(d x) (5.25)
X Z

for f ∈ DV , where M = M − ε. Now we assume that there exists a measurable

function formally denoted by A∗ V such that
 
A∗ V (x)g(x) m(d x) = V (x)Ag(x) m(d x) for g ∈ D(A) ∩ DV , (5.26)
X X
128 5 Asymptotic Properties of Stochastic Semigroups—General Results

A∗ V (x) ≤ M for x ∈ Z and A∗ V (x) ≤ −ε for x ∈ X \ Z . (5.27)

Then condition (5.24) holds and the stochastic semigroup {P(t)}t≥0 is non-sweeping
with respect to the set Z . If Z is a compact set then {P(t)}t≥0 is weakly tight.
The function V is called the Hasminskiı̆ function because he has showed [47]
that the semigroup generated by a non-degenerate Fokker–Planck equation has an
invariant density if and only if there exists a positive function V such that A∗ V (x) ≤
−c < 0 for x ≥ r . The main difficulty in using this method is to define in a
proper way A∗ V because usually V does not belong to the domain of the operator
A∗ . The method of Hasminskiı̆ function has been applied to continuous-time Markov
chains [92], multi-state diffusion processes [86] and diffusion with jumps [88], where
inequality (5.20) was proved by using some generalization of the maximum principle.
This method was also applied in [84] to flows with jumps (see the end of Sect. 1.5)
but the proof of inequality (5.20) is based on an approximation of V by a sequence
of elements from the domain of the operator A∗ .
Chapter 6
Asymptotic Properties of Stochastic
Semigroups—Applications

Most of time-homogeneous piecewise deterministic Markov processes induce sto-

chastic semigroups. Therefore, general analytic tools used in the description of long-
time behaviour of stochastic semigroups from Chap. 5 can be useful to study asymp-
totic and ergodic properties of PDMPs. In this chapter, we give applications of general
results concerning stochastic semigroups to PDMPs considered in Chap. 1.

6.1 Continuous-Time Markov Chains

6.1.1 Foguel Alternative

We now study stochastic semigroups on the space l 1 = L 1 (N, 2N , m), where m is

∞measure, and the elements of l are real-valued sequences u = (u i )i∈N
1
the counting
such that i=0 |u i | < ∞. Thus, the density u is a sequence (u i ) such that u i ≥ 0 for
∞
i ∈ N and i=0 u i = 1.
Let Q = [qi j ] be a sub-Kolmogorov matrix, as introduced in Sect. 4.2.1. Consider
a minimal substochastic semigroup {P(t)}t≥0 induced by the equation

u  (t) = Qu(t), (6.1)

i.e. the semigroup generated by an extension of the operator Q. Since X = N is

a discrete space, the semigroup {P(t)}t≥0 is integral with a continuous kernel k.
If u i (0) > 0 then u i (t) > 0 for sufficiently small t > 0 because {P(t)}t≥0 is a C0 -
semigroup. Thus k(t, i, i) > 0, which means that the kernel k satisfies condition (K)
and all assumptions of Theorem 5.5 are fulfilled.

© The Author(s) 2017 129

R. Rudnicki and M. Tyran-Kamińska, Piecewise Deterministic Processes
in Biological Models, SpringerBriefs in Mathematical Methods,
DOI 10.1007/978-3-319-61295-9_6
130 6 Asymptotic Properties of Stochastic Semigroups—Applications

We consider the following irreducibility condition:

(I) for all i, j ∈ N there exists a sequence of non-negative integers i 0 , i 1 , . . . , ir such
that i 0 = j, ir = i and
qir ir −1 . . . qi2 i1 qi1 i0 > 0. (6.2)

Condition (I) implies that the semigroup {P(t)}t≥0 is irreducible and Corollary 5.4
gives the following theorem.

Theorem 6.1 Let {P(t)}t≥0 be a substochastic semigroup on l 1 induced by (6.1). If

(I) holds, then the semigroup satisfies the Foguel alternative:
(a) if {P(t)}t≥0 has an invariant density, then it is asymptotically stable,
(b) if {P(t)}t≥0 has no invariant density, then for every u ∈ l 1 and i ∈ N we have

lim (P(t)u)i = 0. (6.3)

t→∞

6.1.2 Non-explosive Markov Chains and Asymptotic Stability

We now present a theorem on asymptotic stability for stochastic semigroups induced

by Markov chains. If the minimal semigroup {P(t)}t≥0 related to a Kolmogorov
matrix Q is a stochastic semigroup, then the matrix Q, the Markov chain and the
minimal semigroup are called non-explosive.

Theorem 6.2 Let Q be a non-explosive Kolmogorov matrix. We assume that there

exist a sequence v = (vi ) of non-negative numbers and positive constants ε, k, M
such that
∞

 M for j ≤ k,
qi j vi ≤ (6.4)
i=0
−ε for j > k.

Then the stochastic semigroup {P(t)}t≥0 related to Q is non-sweeping from the set
{0, 1, . . . , k}. In particular, if the matrix Q satisfies conditions (I) and (6.4), then the
semigroup {P(t)}t≥0 is asymptotically stable.

The proof of Theorem 6.2 based on the idea of Hasminskiı̆ function is given in [92,
Theorem 10].
As an example of application we consider a birth–death process (see Sect. 1.2). The
evolution of densities of this process is given by the following system of equations:

u i (t) = −ai u i (t) + bi−1 u i−1 (t) + di+1 u i+1 (t) (6.5)

for i ≥ 0, where b−1 = d0 = 0, bi ≥ 0, di+1 ≥ 0, ai = bi + di for i ≥ 0. The

matrix Q corresponding to Eq. (6.5) is a Kolmogorov matrix.
Assume that di ≥ bi for i ≥ k, where k is a given positive integer. We check using
Theorem 4.11 that the matrix Q is non-explosive. We can choose λ > 0 such that
6.1 Continuous-Time Markov Chains 131

di + λ ≥ bi for i ∈ N. Consider a sequence w = (wi )i≥0 of non-negative numbers

satisfying Q ∗ w = λw. Then

di wi−1 − (bi + di )wi + bi wi+1 = λwi (6.6)

for all i ≥ 1, where w−1 := 0. Then from (6.6) we obtain

 
di + λ di
wi+1 − wi = wi − wi−1 ≥ wi − wi−1
bi bi

for all i ∈ N. If w is a non-zero sequence then from the last inequality it follows that
limi→∞ wi = ∞. It means that w ∈ / l ∞ and according to Theorem 4.11 the matrix Q
is non-explosive and, consequently, it generates a stochastic semigroup.
We now consider again a birth–death process with bi > 0 and di+1 > 0 for all
i ≥ 0. Let us assume that there exists ε > 0 such that bi ≤ di − ε for i ≥ k. Then
the system (6.5) generates a stochastic semigroup and condition (I) holds. Let vi = i
for i ≥ 0, then
∞

vi qi j = ( j − 1)d j − j (b j + d j ) + ( j + 1)b j = b j − d j ≤ −ε
i=0

for j ≥ k, which implies condition (6.4). Hence, the stochastic semigroup generated
by the system (6.5) is asymptotically stable.

6.1.3 Markov Chain with an Absorbing State

We now consider a non-explosive Markov chain such that condition (I) holds for all
i, j except of j = 0. It means that states i, j ≥ 1 communicate, qi0 = 0 for all i ≥ 0,
and for some j ≥ 1 we have q0 j > 0. Then 0 is an absorbing state, u 0 = (1, 0, 0, . . . )
is an invariant density, and condition (WI) holds for u 0 . According to Corollary 5.5
there exists w ∈ l ∞ with wi ≥ 0 for all i such that

∞ each u ∈ l we have limt→∞ (P(t)u)0 − α(u) = 0, where α(u) =

1
(i) for
i=0 u i wi ,
(ii) limt→∞ (P(t)u)i = 0 for i ≥ 0.
Generally, it is not easy to find α(u) but in some cases such formulae are known.
As an example, we consider a birth–death process with bi > 0, di > 0 for i ≥ 1 and
b0 = 0. The assumption b0 = 0 is highly natural because it means that the zero-size
 d
population cannot develop. Let ρi = ij=1 b jj for i ≥ 1. Then w0 = 1,

∞ ∞

i=k ρi
wk =  ∞ if ρi < ∞,
1 + i=1 ρi i=1
132 6 Asymptotic Properties of Stochastic Semigroups—Applications

and wk = 1 in the opposite case. For example if bk = bk a , dk = dk a , where b > d,

a ∈ [0, 1], then ρi = (d/b)i and we find that wk = (d/b)k for k = 0, 1, 2 . . . .

6.1.4 Asymptotics of Paralog Families

In Sect. 1.18 we considered a PDMP which describes the evolution of paralog families
in a genome. We recall that if sk (t) is the number of k-element families at time t then
∞

s1 (t) = −(d + r )s1 (t) + 2(2m + r )s2 (t) + m js j (t), (6.7)
j=3

sk (t) = d(k − 1)sk−1 (t) − (d + r + m)ksk (t) + (r + m)(k + 1)sk+1 (t) (6.8)

for k ≥ 2. The following result [98] characterizes the long-time behaviour of the
sequences (sk (t)).
Theorem
∞ 6.3 Let X be the space of sequences (sk ) which satisfy the condition
k=1 k|s k | < ∞. There exists a sequence (sk∗ ) ∈ X such that for every solution
(sk (t)) of (1.35) and (1.36) with (sk (0)) ∈ X we have

lim e(r −d)t sk (t) = Csk∗ (6.9)

t→∞

for every k = 1, 2, . . . and C dependent only on the sequence (sk (0)). Moreover if
d = r then
βi
lim sk (t) = C , (6.10)
t→∞ k
r
where β = .
r +m
The idea of the proof of Theorem 6.3 is to replace system (6.7)–(6.8) by a new
one which generates a stochastic semigroup. Let

u i (t) = e(r −d)t isi (t).

Then
∞

u 1 = −(2d + m)u 1 + (m + r )u 2 + mu k , (6.11)
k=1
u k = −(d + r + m + d−r
k
)ku k + dku k−1 + (r + m)ku k+1 for i ≥ 2. (6.12)

This system is of the form u  (t) = Qu(t). It is easy to check that Q is a Kolmogorov
matrix. We apply Theorem 4.11 to prove that Q is non-explosive. The case d = 0 is
trivial, so we can assume that d > 0. The sequence w = (wk )k≥1 satisfies equation
6.1 Continuous-Time Markov Chains 133

Q ∗ w = λw if and only if
 λ (n − 1)(r + m) m
wn+1 = 1 + wn + (wn − wn−1 ) + (wn − w1 )
(n + 1)d (n + 1)d (n + 1)d

for n ≥ 1. Hence, the sequence (wn ) is increasing. Thus

 λ
wn+1 ≥ 1 + wn ,
(n + 1)d

and consequently
n  λ
wn ≥ w1 1+ for n ≥ 2.
i=2
di

∞
Since the product (1 + λd −1 k −1 ) diverges, we have w ∈
/ l ∞ and according to
k=1
Theorem 4.11 the matrix Q generates a stochastic semigroup.
We next apply Theorem 5.1 to prove that the semigroup {P(t)}t≥0 generated by the
system (6.11)–(6.12) is asymptotically stable. From Eq. (6.11) applied to densities
we obtain
u 1 (t) ≥ −(2d + m)u 1 (t) + m.

This implies that

m
lim inf u 1 (t) ≥ . (6.13)
t→∞ 2d + m
m
Then h = ( , 0, 0, . . . ) is a lower function and the semigroup is asymptoti-
2d + m
cally stable. Let u ∗ = (u ∗k ) be an invariant density. Then (6.9) holds with sk∗ = u ∗k /k.
m r k
If r = d, then the invariant density is of the form u ∗k = , which gives
r r +m
(6.10).
Another proof of Theorem 6.3 can be carried out with the help of Theorem 6.1 by
checking that this semigroup is non-sweeping from the set {1}. A generalization of
this result to the case when the mutation rate depends on the family size was made
in [99].

6.1.5 Applications to Pure Jump-Type Markov Processes

We now study asymptotic properties of the semigroup {P(t)}t≥0 related to a pure

jump-type Markov process ξ(t) as described in Sect. 4.2.1 with a bounded measur-
able jump rate function ϕ and the jump distribution P corresponding to a stochastic
operator P. We will apply Theorem 5.6, and therefore we require that X is a sepa-
134 6 Asymptotic Properties of Stochastic Semigroups—Applications

rable metric space and Σ = B(X ). Let us assume that inf{ϕ(x) : x ∈ X } > 0. Let
Q(x, B) = ∞ n=0 P (x, B), where the sequence of transition probablities P (x, B)
n n

is given by the induction formula:

P 0 (x, B) = δx (B), P n+1 (x, B) = P(y, B)P n (x, dy).

X

We assume that there exists a point x0 ∈ X such that for each neighbourhood U
of x0 we have Q(x, U ) > 0 for x-a.e. Then from formula (4.13) it follows that the
semigroup {P(t)}t≥0 is weakly irreducible, i.e. it satisfies condition (WI). Moreover,
we suppose that for every y0 ∈ X there exist an ε > 0, an n ∈ N, and a measurable
function η ≥ 0 such that η(x) m(d x) > 0 and

P n (y, B) ≥ η(x) m(d x) for y ∈ B(y0 , ε) and B ∈ Σ. (6.14)

B

Again from (4.13) the semigroup {P(t)}t≥0 satisfies condition (K). In order to give a
condition which guarantees asymptotic stability, we can use a Hasminskiı̆ function.
Let V : X → [0, ∞) be a measurable function. We define

A∗ V (x) = −ϕ(x)V (x) + ϕ(x) V (y) P(x, dy).

X

This definition is formally correct (though it can happen that A∗ V (x) = ∞ on a set
of positive measure m), and formula (5.26) holds. If for some compact set F we
have A∗ V (x) ≤ M for x ∈ F and A∗ V (x) ≤ −ε for x ∈ X \ F, then there exists
a Hasminskiı̆ function for the semigroup {P(t)}t≥0 and the set F, and consequently
{P(t)}t≥0 is weakly tight. Since the semigroup {P(t)}t≥0 satisfies conditions (WI),
(K) and (WT) of Theorem 5.6, it is asymptotically stable.
Example 6.1 Consider a pure jump-type Markov process on X = [0, ∞) such that
we jump from a point x to the point ϑx , where the random variable ϑx has values in
X . We assume that there exist r > 0 and δ > 0 such that

sup Eϑx < ∞ for x < r and Eϑx ≤ x − δ for x ≥ r. (6.15)

x≤r

We also assume that for each x ≥ 0 the random variable ϑx has a continuous density
f x and that there exists a point x0 ∈ X such that f x (x0 ) > 0 for x ≤ r . Since the
densities f x are continuous, condition (K) is fulfilled. From (6.15) we deduce that
for each x ≥ r there is a point y < x − ε such that y ∈ supp f x . Using induction
argument we check that P n (x, [0, r ]) > 0 for sufficiently large n, and since f x (x0 ) >
0 for x ≤ r we finally have P n+1 (x, U ) > 0 for every neighbourhood of x0 , i.e. (WI)
holds. We check that V (x) = x is a Hasminskiı̆ function. We have

A∗ V (x) = −ϕ(x)V (x) + ϕ(x) V (y) P(x, dy) = ϕ(x)(Eϑx − x).

X
6.1 Continuous-Time Markov Chains 135

From (6.15) we see that A∗ V is bounded from above and A∗ V (x) ≤ −ε for x ≥ r ,
where ε = δ inf ϕ(x). Since the semigroup {P(t)}t≥0 satisfies conditions (WI), (K)
x∈X
and (WT), it is asymptotically stable.

6.2 Dynamical Systems with Random Switching

6.2.1 General Formulation

Based on the paper [90] we present here a general method of studying asymptotic
properties of semigroups induced by dynamical systems with random switching
introduced in Sect. 1.9 and described in Sect. 4.2.5. The family of processes ξ(t)
induces a substochastic semigroup {P(t)}t≥0 on the space L 1 (X, Σ, m), where Σ is
the σ -algebra of the Borel subsets of X = X × I and m is the product measure of the
Lebesgue measure on X and the counting measure on I . The semigroup {P(t)}t≥0
corresponds to the transition function P given by

P(t, (x, i), B) = Prob(ξx,i (t) ∈ B),

where ξx,i (t) is a Markov process from the family ξ(t) starting from the state (x, i).
We now recall how to check condition (K) for the semigroup {P(t)}t≥0 . Let n ∈ N,
i = (i 1 , . . . , i n ) ∈ I n , i p = i p−1 for p = 2, . . . , n, and t = (t1 , . . . , tn ) be such that
t p > 0 for p = 1, . . . , n. Take y ∈ X and assume that the sequence y0 , . . . , yn given
i
by the recurrent formula y0 = y, y p = πt pp (y p−1 ) for p = 1, . . . , n, is well defined
and qi p+1 i p (y p ) > 0 for p = 1, . . . , n − 1. The function

π it (y) = πtinn ◦ · · · ◦ πti11 (y)

is called a cumulative flow along the trajectories of the flows π i1 , . . . , π in which joins
the state (y, i 1 ) with the state (yn , i n ). For i ∈ I d+1 , y ∈ X , and t > 0 we define the
function ψ y,t on the set Δt = {τ = (τ1 , . . . , τd ) : τi > 0, τ1 + · · · + τd ≤ t} by

ψ y,t (τ1 , . . . , τd ) = π i(τ1 ,...,τd ,t−τ1 −τ2 −···−τd ) (y).

Assume that for some y0 ∈ X and τ 0 ∈ Δt we have


dψ y0 ,t (τ 0 )
det = 0. (6.16)
dτ

Then there exists a continuous and positive function κ defined in a neighbourhood V

of the point (x0 , y0 ), such that k(t, (x, i d+1 ), (y, i 1 )) ≥ κ(x, y) for (x, y) ∈ V , where
136 6 Asymptotic Properties of Stochastic Semigroups—Applications

k(t, ·, ·) is the kernel of P(t) and x0 = ψ y,t (τ10 , . . . , τd0 ). It means that condition (K)
holds for the state (y0 , i 1 ).
Condition (6.16) can be checked by using Lie brackets. Let f (x) and g(x) be two
vector fields on Rd . The Lie bracket [ f, g] is a vector field given by

d 
 
∂g j ∂fj
[ f, g] j (x) = f k (x) (x) − gk (x) (x) .
k=1
∂ xk ∂ xk

Assume that the vector fields b1 , . . . , bk are sufficiently smooth in a neighbourhood

of a point x. We say that Hörmander’s condition holds at x if vectors fields

b2 (x) − b1 (x), . . . , bk (x) − b1 (x), [bi , b j ](x)1≤i, j≤k , [bi , [b j , bl ]](x)1≤i, j,l≤k , . . .

span the space Rd . Let (x0 , i), (y0 , j) ∈ X and assume that the cumulative flow joins
(y0 , j) with (x0 , i). If Hörmander’s condition holds at x0 and ql  l (x0 ) > 0 for l = l  ,
then condition (K) is satisfied at the state (y0 , j) (see [8, Theorem 4]).
We now show how to check condition (WI). Let (x0 , i) be given. We assume that
for each ε > 0 and each state (y0 , j) we can find x ∈ B(x0 , ε/2) such that (y0 , j)
can be joined with (x, i) by a cumulative flow π it0 (y0 ), where i 1 = j, i n = i and
t0 = (t10 , . . . , tn0 ). From continuous dependence of solutions on initial conditions
and continuity of intensity functions qi p+1 ,i p it follows that there exists δ > 0 such
that each state (y, j) with y ∈ B(y0 , δ) can be joined with a state (x  , i) with x  ∈
B(x0 , ε/2), i.e. πti0 (y) ∈ B(x0 , ε/2) for y ∈ B(y0 , δ). We now can find γ > 0 such
that πti (y) ∈ B(x0 , ε) if t − t0 < γ . It means that
 
Prob πti (y) ∈ B(x0 , ε) and t1 + · · · + tn = t > 0

for y ∈ B(y0 , δ) and t = t10 + · · · + tn0 . Therefore

P(t, (y, j), B(x0 , ε) × {i}) > 0 for y ∈ B(y0 , δ). (6.17)

Inequality (5.9) in condition (WI) can be written in the following way:

f (y, j)P(t, (y, j), B(x0 , ε) × {i}) m(dy × d j) > 0. (6.18)

X

Let (y0 , j) be a point such that

  
m B(y0 , δ) × { j} ∩ supp f > 0 for each δ > 0. (6.19)

Then inequality (6.18) follows from (6.17) and (6.19). Thus, condition (WI) holds if
there exists a state (x0 , i) such that starting from any state we are able to go arbitrarily
close to (x0 , i) by a cumulative flow. For example, if x0 is an asymptotically stable
stationary point of a semiflow πti with the basin of attraction U ⊆ X and each state
6.2 Dynamical Systems with Random Switching 137

(y, j) can be joined by a cumulative flow with some state (x, i), where x ∈ U , then
condition (WI) holds.
We now explain how to apply Corollary 5.7 to dynamical systems with ran-
dom switching. We assume that we have an increasing sequence
 of bounded open
sets G n ⊆ G with smooth boundaries ∂G n , such that X ⊆ ∞ n=1 c1G n , where c1G n
denotes the closure of G n . Let n(x) be an outward pointing normal field of the
boundary ∂G n . We assume that

n(x) · bi (x) < 0 for x ∈ ∂G n , i ∈ I, and n ∈ N. (6.20)

Then the solutions of each system x  = bi (x) starting from points in c1G n remain in
the set c1G n . In particular the sets X n = X ∩ c1G n are invariant with respect to the
semigroup {P(t)}t≥0 .
We summarize the above discussion. Consider a family of systems of differential
equations x  = bi (x), i ∈ I , such that for each point x̄ ∈ X the initial problem x  =
bi (x), x(0) = x̄, has a unique solution x(t) ∈ X for all t ≥ 0. Then a dynamical
system with random switching created by this family of systems and some continuous
and bounded intensity functions q ji (x) generates a stochastic semigroup {P(t)}t≥0
on L 1 (X, B(X), m). Assume that there exist x0 ∈ X and i 0 ∈ I such that starting
from any state (x, i) ∈ X we are able to go arbitrarily close to (x0 , i 0 ) by a cumulative
flow and that Hörmander’s condition holds at x0 and ql  l (x0 ) > 0 for all l, l  , l = l  .
We also assume ∞that there is an increasing family of bounded open sets G n ⊆ Rd
such that X ⊆ n=1 c1G n and condition (6.20) holds. Then the semigroup {P(t)}t≥0
is asymptotically stable.

6.2.2 Applications to Stochastic Gene Expression Models

We now examine asymptotic properties of the stochastic models of gene expression

(see Sects. 1.8 and 1.9). In order to find sufficient conditions for asymptotic stability
of the semigroups related to these models we can apply the procedure   described in
the last section. It is easy to see that if we substitute x(t) = μP ξ μt and i(t) = A μt
to (1.14) then we obtain
x  (t) = i(t) − x(t). (6.21)
     
Similarly, substituting x1 (t) = μRR ξ1 μtR , x2 (t) = μPP μR R ξ2 μtR , i(t) = A μtR , α =
μP
to (1.15) gives
μR 
x1 (t) = i(t) − x1 (t),
(6.22)
x2 (t) = α(x1 (t) − x2 (t)).

After a simple linear substitution we can also replace (1.16) by

138 6 Asymptotic Properties of Stochastic Semigroups—Applications
⎧ 
⎨ x1 (t) = i(t) − x1 (t),
⎪
x2 (t) = α(x2 (t) − x1 (t)), (6.23)
⎪
⎩ 
x3 (t) = β(x3 (t) − x2 (t)).

From now on we consider only models given by (6.21), (6.22) and (6.23). In all
these models we have two dynamical systems, which correspond, respectively, to
i = 0 and i = 1. Each system has a unique stationary point, either 0 = (0, 0, 0), or
1 = (1, 1, 1), which is asymptotically stable. Assume that intensities q0 and q1 of
transformations of a gene into an active state and into inactive state, respectively, are
non-negative continuous functions of x. We assume that q0 (0) > 0 and q1 (1) > 0.
This assumption implies two properties of the relative PDMP. The first one is that
if the process starts from any point x ∈ Rd then it enters the invariant set X =
[0, 1]d × {0, 1} and the process visits any neighbourhood of the point (0, 0) and the
point (1, 1) for infinitely many times. It means that condition (WI) holds for x 0 = 0
and for x 0 = 1. Moreover, if we assume additionally that q0 (1) > 0 or q1 (0) > 0
then we can check that condition (K) holds by applying Hörmander’s condition. We
consider here only the most advanced three-dimensional model. Let v = (1, 0, 0).
Then b2 (x) = v + b1 (x). Hence

b2 − b1 = (1, 0, 0),
[b1 , b2 ] = [b1 , v] = [(−x1 , α(x2 − x1 ), β(x3 − x2 )), (1, 0, 0)] = (1, −α, 0),
[b1 , [b1 , b2 ]] = (1, −(α 2 + α), αβ).

The vectors b2 − b1 , [b1 , b2 ], [b1 , [b1 , b2 ]] span the space R3 , and consequently
Hörmander’s condition holds. Since the set X is compact, the semigroup {P(t)}t≥0
is asymptotically stable according to Corollary 5.6.
The one-dimensional model from Sect. 1.8 is simple enough to find the invariant
density, which allows us to give more information on asymptotic behaviour of distri-
butions of this process. In this case the functions u i (t, x) = P(t) f (x, i), i ∈ {0, 1},
satisfy the following Fokker–Planck system:

∂u 0 ∂
+ (−xu 0 ) = q1 u 1 − q0 u 0 ,
∂t ∂x
∂u 1 ∂
+ ((1 − x)u 1 ) = q0 u 0 − q1 u 1 .
∂t ∂x
If u i , i ∈ {0, 1}, do not depend on t, then

(−xu 0 (x)) = q1 u 1 − q0 u 0 ,
(6.24)
((1 − x)u 1 (x)) = q0 u 0 − q1 u 1 .

Hence
−xu 0 (x) + (1 − x)u 1 (x) = c for some c ∈ R.
6.2 Dynamical Systems with Random Switching 139

c x
From the last equation we find u 1 (x) = + u 0 (x). If we suppose that
1−x 1−x
u(x, i) is a density then both functions u 0 and u 1 are non-negative and integrable,
but this case is possible only if c = 0. Indeed, if c < 0 then
x
su 0 (s) x
u 0 (s) |c|x
ds ≤ x ds <
0 1−s 0 1−s 2

for sufficiently small x > 0, and consequently u 1 cannot be a non-negative function.

x
If c > 0, then u 1 is not integrable in 1. Substituting u 1 (x) = u 0 (x) to (6.24)
1−x
we get
q0 (x) − 1 q1 (x)
u 0 (x) = a(x)u 0 (x), a(x) = − .
x 1−x

Thus  
q0 (s) − 1 q1 (s)
x
u 0 (x) = C exp − ds,
0 s 1−s
x  (6.25)
Cx q0 (s) − 1 q1 (s)
u 1 (x) = exp − ds,
1−x 0 s 1−s

1
where we choose C such that 0 (u 0 (x) + u 1 (x)) d x = 1.

Example 6.2 Consider the case when q0 and q1 are constants. Then

u 0 (x) = C x q0 −1 (1 − x)q1 and u 1 (x) = C x q0 (1 − x)q1 −1 .

This pair is a density if

C = (B(q0 + 1, q1 ) + B(q0 , q1 + 1))−1 ,

where B is the Beta function.

Remark 6.1 Observe that if q0 (0) > 0 and q1 (1) > 0 then the functions u 0 and u 1
given by (6.25) are integrable. That means that there exists a unique density for this
semigroup and since the semigroup is partially integral, it is asymptotically stable
according to Theorem 5.2. Also in two- and three-dimensional cases the semigroup
remains asymptotically stable when q0 (1) = 0 and q1 (0) = 0, but the proof is more
advanced because we need to check directly condition (6.16).

Remark 6.2 It is rather difficult to find a general formula for an invariant density
in two- and three-dimensional cases. But we can find a formula for the support of
invariant density if q0 and q1 are strictly positive functions (see [100]). Also in this
paper we can find a discussion concerning adiabatic limit of the system, i.e. the
behaviour of our model, when both rates q0 and q1 tend to infinity, unlike their ratio.
In this case, our process “tends” to a deterministic model, which can exhibit two
specific types of behaviour: bistability and the existence of the limit cycle.
140 6 Asymptotic Properties of Stochastic Semigroups—Applications

Remark 6.3 If q0 (0) = 0 or q1 (1) = 0 then the system can loose asymptotic stability
even in the case when qi (x) are positive for all other x. Consider one-dimensional
example with q0 (x) = x r , r > 0 and q1 (x) = 1. −1Then from (6.25) it follows that
−1 r
u 0 (x) = C 1−x
x
exp r x and u 1 (x) = C exp r x r . In this case u 0 is not inte-
grable in 0, and therefore an invariant density does not exist. In this example the
distributions of the process converge weakly to δ(0,0) . It means that the average time
when the gene is inactive tends to 1. If q0 (0) = 0 and q1 (1) = 0, then the distribution
of the process can converge to c1 δ(0,0) + c2 δ(1,1) , where the constants c1 and c2 can
depend on the initial density. Such properties can be checked by using Theorem 5.3.

6.3 Cell Maturation Models

6.3.1 Introductory Remarks

In this section we consider size-structured (or maturity-structured) cellular mathe-

matical models. Examples of such models were presented in Sects. 1.5, 1.6, 1.7 and
1.13. Generally, two types of such models are considered. The first group consists
of discrete-time models, which show us how the size or maturation is inherited from
a mother to a daughter cell. Such models are popular in the description of the cell
cycle. The second type includes continuous-time models, when we observe the evo-
lution of size distribution. We should underline that there are a lot of papers and
books concerning such models [76]. Some of them are very advanced. For example,
a continuous-time two-phase model [68] is given by a system of nonlinear partial
differential equations with time and space retarded arguments. We present only the
simplest models which can be easily treated using PDMPs.
The models presented in Sects. 1.5 and 1.6 belong to the category of dynamical
systems with random jumps, i.e. we have one dynamical system and a point moves
along the trajectories of this system and with some intensity jumps to a new place
and resumes movement. In a single-phase model of cell cycle, we have a jump at
the moment of cellular division, but in two or more phase models we can have also
jumps when a cell enter the next phase. Such models are based on the assumption
that in each generation and each phase the size or maturation changes according
to the same law. Moreover, since we consider a single line of cells we do not take
into account birth and death rates which usually depends on cell parameters and the
size of population. At the end of this section we give some remarks how methods
developed for stochastic semigroups can be applied to positive semigroups related
to more general size-structured models.
6.3 Cell Maturation Models 141

6.3.2 Flows with Jumps

We now generalize the method considered for pure jump-type Markov processes to
flows with jumps described in Sect. 4.2.4. We show how we can apply Theorem 5.6 to
prove asymptotic stability of the stochastic semigroup {P(t)}t≥0 . Define R(t, x, B) =
ϕ(πt x)P(πt x, B) for t ≥ 0, x ∈ X , B ∈ Σ and by induction

R(t1 , . . . , tn , tn+1 , x, B) = R(tn+1 , y, B)R(t1 , . . . , tn , x, dy).

X

Let Q(t1 , t2 , . . . , x, B) = ∞ n=0 R(t1 , . . . , tn , x, B) for any sequence of non-negative
numbers (tn ). We fix a point x0 ∈ X and assume that for a.e. point x ∈ X and each
neighbourhood U of x0 we find a sequence (tn ) such that Q(t1 , t2 , . . . , x, U ) > 0.
Then from formulae (3.15) and (3.14) it follows that the semigroup {P(t)}t≥0 satisfies
condition (WI).
We now discuss the problem how to check condition (K). If for example the
transition function P has a continuous kernel minorant k ≥ 0, i.e. P(x, B) ≥
B k(x, y) dy, and for every point x ∈ X we find t ≥ 0 and y ∈ X such that
k(πt x, y) > 0, then (K) holds.
But in many applications the transition function can have no kernel part. Consider
an example when the jump is deterministic from x to S(x), i.e. P(x, {S(x)}) = 1. We
assume that the transformation S is continuously differentiable and the Frobenius–
Perron operator PS exists (see Sect. 2.1.5). Let x0 ∈ X and define a sequence x j =
S(x j−1 ) for j = 1, . . . , d. Set

v j = S  (xd−1 ) . . . S  (x j−1 )b(x j−1 ) − b(xd ) (6.26)

for j = 1, . . . , d. We say that the point x0 satisfies Hörmander’s condition if

ϕ(x j−1 ) > 0 for all j = 1, . . . , d and the vectors v1 , . . . , vd are linearly inde-
pendent. We say that the point y0 ∈ X can be joined with x̃ ∈ X if there is a
sequence t0 , t1 , . . . , tn of positive numbers such that yi+1 := S(πti yi ), ϕ(πti yi ) > 0
for i = 0, . . . , n − 1, and x̃ = πtn yn . Assume that each point y0 ∈ X can be joined
with a point x̃ which satisfies Hörmander’s condition. Then condition (K) holds (see
[87] the proof of Proposition 1). Moreover, if we find one point x0 such that each
point y0 can be joined with x0 then condition (WI) holds. If for any x, y ∈ X we can
join x with y, then the semigroup {P(t)}t≥0 is irreducible.
In order to give a condition which guarantees asymptotic stability, we can use a
Hasminskiı̆ function. Let V : X → [0, ∞) be a continuously differentiable function.
We define

A∗ V (x) = A∗0 V (x) − ϕ(x)V (x) + ϕ(x) V (y) P(x, dy),

X
142 6 Asymptotic Properties of Stochastic Semigroups—Applications


d ∂V
where A∗0 V (x) = bi (x) (x). If for some compact set F we have A∗ V (x) ≤ M
i=1 ∂ xi
for x ∈ F and A∗ V (x) ≤ −ε for x ∈ X \ F, then V is a Hasminskiı̆ function for
the semigroup {P(t)}t≥0 and the set F. The proof of this fact can be found in [84]
and it is based on an approximation of V by a sequence of elements from the domain
of the operator A∗ . Thus {P(t)}t≥0 is weakly tight. Since the semigroup {P(t)}t≥0
satisfies conditions (WI), (K) and (WT) of Theorem 5.6, it is asymptotically stable.

6.3.3 Size-Structured Model

We now investigate a size-structured model from Sect. 1.5. We keep the notation from
Sect. 1.5, i.e. x is the size of a cell, it changes according to the equation x  = g(x);
ϕ(x) is the intensity of division; if the mother cell has size x at the moment of division,
then the new born daughter cell has size x/2. Here the jump is deterministic and given
by S(x) = x/2. Let xmin be the minimum cell size. We assume that ϕ is a continuous
function and that ϕ(x) = 0 for x ≤ 2xmin and ϕ(x) > 0 for x > 2xmin , because 2xmin
is the minimum cell size when it can divide. Thus we can consider our process in the
phase space X = [xmin , ∞). We also assume that the function g : X → (0, ∞) has
a bounded derivative. Moreover, we assume that

g(2 x̄) = 2g(x̄) for some x̄ > xmin . (6.27)

Then taking x0 = 2 x̄ and x1 = S(x0 ) = x̄ condition (6.26) takes the form

v1 = S  (x0 )g(x0 ) − g(x1 ) = 21 g(2 x̄) − g(x̄) = 0.

We can join any point y0 ∈ X with x0 by the action of the dynamical system and
transformation S. Indeed, if y0 < x0 then there exists t > 0 such that x0 = πt (y0 ).
If y0 > x0 then S n (y0 ) < x0 for some positive integer n and, therefore, it can be
joined with x0 . Hence condition (K) holds. The semigroup {P(t)}t≥0 induced by this
process is irreducible because we can join any two points from the set (xmin , ∞).
From Corollary 5.4 it follows that the semigroup {P(t)}t≥0 is asymptotically stable
or sweeping from compact sets. Therefore, if {P(t)}t≥0 has an invariant density f ∗ ,
then {P(t)}t≥0 is asymptotically stable and f ∗ > 0 a.e.; if {P(t)}t≥0 has no invariant
density, then for all c > xmin and f ∈ L 1 we have
c
lim P(t) f (x) d x = 0.
t→∞ xmin

The evolution equation is of the form

∂u ∂
= − (g(x)u(t, x)) − ϕ(x)u(t, x) + 2ϕ(2x)u(t, 2x). (6.28)
∂t ∂x
6.3 Cell Maturation Models 143

Thus, if there exists an invariant density f ∗ , then it satisfies the following equation:

g(x) f ∗ (x) = −(ϕ(x) + g  (x)) f ∗ (x) + 2ϕ(2x) f ∗ (2x). (6.29)

Since (6.29) is an ordinary differential equations with advanced argument, it is rather

difficult to find its solution or to check when a density is one of its solutions (see [70]
for sufficient conditions and particular examples).
We now apply the Hasminskiı̆ function technique to find a sufficient condition for
asymptotic stability. Assume that

g(x)
lim sup ϕ(x) > 0, and κ = lim sup < log 2. (6.30)
x→∞ x→∞ xϕ(x)

Then we check that there exists a Hasminskiı̆ function. We have

A∗ V (x) = g(x)V  (x) − ϕ(x)V (x) + ϕ(x)V ( 21 x).

Let V (x) = x r , where r > 0. Then

A∗ V (x) = x r [r x −1 g(x) − (1 − 2−r )ϕ(x)]. (6.31)

Let h(r ) = (1 − 2−r )/r . Since limr →0+ h(r ) = log 2, we find r > 0 such that h(r ) >
κ. From (6.30) and (6.31) it follows that there exists ε > 0 such that A∗ V (x) ≤ −ε
for sufficiently large x. Thus if (6.27) and (6.30) hold, then the semigroup {P(t)}t≥0
is asymptotically stable.

Remark 6.4 Age- and size-structured models play an important role in population
dynamics. The first one was introduced in [74, 105] and the second in [12]. Both
models were intensively studied and generalized [31, 46, 76, 97, 118]. In [11] it
is considered an evolution equation u  = Au which can be used to study both types
of models. This equation generates a semigroup of positive operators {T (t)}t≥0 on
some space L 1 (X, Σ, μ). The main result of this paper is asynchronous exponential
growth of the population. Precisely, under suitable assumptions there exist λ ∈ R,
an integrable function f ∗ and a positive linear functional α such that

lim e−λt T (t) f = α( f ) f ∗ . (6.32)

t→∞

We can use the technique developed for stochastic semigroup in Chap. 5 to prove
(6.32). In order to do it we first prove that there exist λ ∈ R and function w
such that A∗ w = λw and 0 < c1 ≤ w(x) ≤ c2 for x ∈ X . Then we introduce a sto-
chastic semigroup {P(t)}t≥0 on the space L 1 (X, Σ, m) with m(d x) = w(x) μ(d x)
given by P(t) = e−λt T (t). We check that {P(t)}t≥0 is asymptotically stable, which
gives (6.32).
144 6 Asymptotic Properties of Stochastic Semigroups—Applications

6.3.4 Two-Phase Cell Cycle Model

We now consider the two-phase cell cycle model presented in Sect. 1.6. This is a
simplified version of the model introduced in [68]. We assume that the size x of
a cell changes according to the equation x  = g(x), where the function g has a
bounded derivative; the intensity of division ϕ(x) is a continuous function, ϕ(x) = 0
for x ≤ x B and ϕ(x) > 0 for x > x B , where x B is the minimum cell size when
it can enter the phase B; the constant t B > 0 is the duration of the phase B. The
minimum cell size is xmin = 21 πt B x B and it is clear that we need to assume that
πt B x B < 2x B to have xmin < x B . Thus we can consider our process on the space
X = [xmin , ∞) × {1} ∪ [x B , ∞) × [0, t B ] × {2}. As in Sect. 6.3.3 we assume that

g(2 x̄) = 2g(x̄) for some x̄ > xmin . (6.33)

Since this model is not exactly a special case of the general model considered in
Sect. 6.3.2, we explain how to check condition (K). Let z̄ = π−t B (2 x̄). Then z̄ > x B
and ϕ(z̄) > 0. First, we check that (K) holds for y0 = (z̄, 1). Precisely, we show that
there exist t > t B , ε > 0, x0 ∈ (xmin , ∞) × {1} and neighbourhoods U of y0 and V
of x0 , such that the operator P(t) is partially integral with the kernel

k(x, y) ≥ ε1V (x)1U (y). (6.34)

Let z > x B and t > t B . We introduce a function ψz,t defined on the interval [t B , t]
by  
ψz,t (τ ) = πt−τ 21 πτ z .

We have 
dψz,t   ∂πt−τ  1 
(τ ) = −g πt−τ 21 πτ z + πτ z · 21 g(πτ z),
dτ ∂x 2
dψz,t        
lim+ lim (t B ) = −g 21 πt B z̄ + 21 g πt B z̄ = −g x̄ + 21 g 2 x̄ = 0.
t→t B z→z̄ dτ

Therefore we find δ > 0 and M > 0 such that

 
 dψz,t 
δ <  (τ ) < M (6.35)
dτ

for z from some neighbourhood U1 of z̄, for some t > t B and τ ∈ [t B , t]. Now we
consider our stochastic process starting from the point (z, 1). Let the random variable
T be the first time when the process jumps to the phase B. Since ϕ(z̄) > 0, T has a
density distribution h such that h(τ )≥ β for  τ ∈[0,t − t B ], where β is a positive
constant. Let τ0 = 21 (t B + t), x0 = πt−τ0 21 πτ0 z̄ , 1 and V = (x0 − r, x0 + r ) ×
{1} with r = 21 δ(t − t B ). Then (6.34) holds with ε = β/M. Since every point x ∈ X
can be joined with (z̄, 1), condition (K) is fulfilled. The semigroup {P(t)}t≥0 induced
6.3 Cell Maturation Models 145

by this process also satisfies condition (WI) because we can join any point from X
with y0 . The infinitesimal generator of {P(t)}t≥0 is of the following form:

∂
A f (x, 1) = − (g(x) f (x, 1)) − ϕ(x) f (x, 1) + 2 f (2x, t B , 2),
∂x (6.36)
∂ ∂
A f (x, τ, 2) = − (g(x) f (x, τ, 2)) − ( f (x, τ, 2)),
∂x ∂τ
and the functions from the domain of A satisfy the condition

ϕ(x) f (x, 1) = f (x, 0, 2).

We find that
∂f
A∗ f (x, 1) = g(x) (x, 1) − ϕ(x) f (x, 1) + ϕ(x) f (x, 0, 2),
∂x (6.37)
∂f ∂f
A∗ f (x, τ, 2) = g(x) (x, τ, 2) + (x, τ, 2),
∂x ∂τ

and the functions from the domain of A∗ satisfy

f (x, 1) = f (2x, t B , 2). (6.38)

We take V (x, 1) = x r and V (x, τ, 2) = Ce−aτ x r , where r , a and C are some positive
constants. Then

A∗ V (x, 1) = x r [rg(x)x −1 + ϕ(x)(C − 1)],

A∗ V (x, τ, 2) = Ce−aτ x r [rg(x)x −1 − a].

From (6.38) we find that C = 2−r eat B . We assume that

g(x) log 2 g(x)

lim sup <κ< , lim sup < log 2 − κt B .
x→∞ x tB x→∞ xϕ(x)

Then taking a = κr , we check that there exist ε > 0 and a sufficiently small r > 0
such that A∗ V (x, 1) ≤ −ε and A∗ V (x, τ, 1) ≤ −ε for τ ∈ [0, t B ] and sufficiently
large x, which means that the semigroup {P(t)}t≥0 is weakly tight. Summarizing, if
g(2 x̄) = 2g(x̄) for some x̄ > xmin and

g(x) g(x)
lim sup < log 2 − t B lim sup ,
x→∞ xϕ(x) x→∞ x

then, according to Theorem 5.6, the semigroup {P(t)}t≥0 is asymptotically sta-

ble. Moreover, if f ∗ is the invariant density, then f ∗ (x, 1) > 0 ⇐⇒ x > xmin and
f ∗ (x, τ, 2) > 0 ⇐⇒ x > πτ x B .
146 6 Asymptotic Properties of Stochastic Semigroups—Applications

6.3.5 Lebowitz–Rubinow’s Model

We recall that in this model a maturity of a cell is a real variable x ∈ [0, 1]. A new born
cell has maturity 0 and a cell splits at maturity 1. The maturity x grows according to the
equation x  = v, where the maturation velocity v of each individual cell is constant.
The relation between the maturation velocities of mother’s v and daughter’s cells v
is given by a transition probability P(v, dv ). As we have mentioned in Sect. 1.7, this
model is a special case of one-dimensional stochastic billiards investigated in [78] and
based on that paper we briefly present results concerning its asymptotic properties.
We assume that v ∈ (0, 1] and P(v, dv ) = αδv (v ) + βk(v , v) dv , where β ∈ (0, 1],
α + β = 1, and 0 k(v , v) dv = 1 for v ∈ (0, 1]. We showed in Sect. 4.2.6 that the
1

process ξ(t) = (x(t), v(t)) induces a stochastic semigroup {P(t)}t≥0 on the space
L 1 (X, Σ, m), where X = (0, 1]2 , Σ = B(X ) and m(d x × dv) = d x × dv. Since
the boundary condition (4.26) contains a kernel operator, one can check that the semi-
group {P(t)}t≥0 is partially integral. Observe that if this semigroup has an invariant
density f ∗ , then A f ∗ = 0 and from (4.25) it follows that f ∗ does not depend on
x. From (4.26) we deduce that v f ∗ = K (v f ∗ ), where K is a stochastic operator on
L 1 [0, 1] given by
1
K h(v ) = k(v , v)h(v) dv. (6.39)
0

We assume that K is irreducible which is equivalent to say that there does not exist a
set B ⊆ (0, 1] of the Lebesgue measure 0 < |B| < 1 such that k(v , v) = 0 for v ∈ B
and v ∈/ B. From irreducibility it follows that if an invariant density f ∗ exists then
it is unique and f ∗ (x) > 0 for a.e. x. The question of the existence of an invariant
density is nontrivial. If for example we assume that there exist C > 0 and γ > 0
such that
k(v, v ) ≤ C|v|γ for v, v ∈ (0, 1], (6.40)

then an invariant density exists. It means that irreducibility of K and condition

(6.40) implies asymptotic stability of the semigroup. Now we consider the case
when there is no invariant density. Assume that the kernel k is bounded and K
is irreducible. Then the operator K has a unique invariant density h ∗ > 0 and the
function f ∗ (v) = v−1 h ∗ (v) is invariant with respect to the semigroup {P(t)}t≥0 . Since
f ∗ is not integrable, but ε f ∗ (v) dv < ∞ for every ε > 0, from Theorem 5.4(c) it
1

follows that
ε 1
lim P(t) f (x, v) d x dv = 1 (6.41)
t→∞ 0 0

for every density f and every ε > 0. For example if k ≡ 1, then h ∗ ≡ 1 and f ∗ (v) =
v−1 is not integrable. Thus, the semigroup has no invariant density and consequently
(6.41) holds. It is interesting that in this example we have
c
P(t) f (x, v) ∼ (log t)−1 as t → ∞
|v|
6.3 Cell Maturation Models 147

for v ≥ ε and x ∈ [0, 1], where c is some constant.

6.3.6 Stein’s Model

In Sect. 1.11 we considered Stein’s model which describes how the depolarization
V (t) changes in time. Now, based on the paper [91], we present its asymptotic
properties. First, we introduce an extra 0-phase, which begins at the end of the
refractory period and ends when depolarization jumps from 0 for the first time. Thus
the process is defined on the space X = {(0, 0)} ∪ (−∞, θ ] × {1} ∪ [0, t R ] × {2}.
Define a measure m on the σ -algebra B(X ) of the Borel subsets of X by m =
δ(0,0) + m 1 + m 2 , where δ(0,0) is the Dirac measure at (0, 0), m 1 is the Lebesgue
measure on the segment (−∞, θ ] × {1} and m 2 is the Lebesgue measure on the
segment [0, t R ] × {2}. The process induces a stochastic semigroup {P(t)}t≥0 on the
space L 1 (X, B(X ), m).
The stochastic semigroup introduced by Stein’s model is asymptotically stable.
The proof of this result is given in [91] and it is based on Theorem 5.6. Since the
semigroup is asymptotically stable it would be useful to find an invariant density.
Unfortunately, this is rather a difficult task and we only present the fact that the
invariant density f ∗ of the process with the refractory period can be expressed by
the invariant density f¯∗ of the process without the refractory period (i.e. t R = 0) and
the same other parameters. Let
 −1
c = 1 + t R (λ E + λ I ) f¯∗ (0, 0) .

Then
f ∗ (0, 0) = c f¯∗ (0, 0),
f ∗ (x, 1) = c f¯∗ (x, 1) for x ∈ (−∞, θ ],
f ∗ (x, 2) = c(λ E + λ I ) f¯∗ (0, 0) for x ∈ [0, t R ].
Appendix A
Measure and Probability Essentials

Our objective in this appendix is to introduce basic definitions and results from
measure and probability theory required in the text. For the general background and
proofs we refer the reader to [55].

A. 1 Measurable Spaces and Mappings

A measurable space is a pair (X, Σ) consisting of a non-empty set X and a family

Σ of subsets of X , called a σ-algebra, satisfying the following
(1) X ∈ Σ,
(2) if B ∈ Σ then its complement
 B c := X \ B is in Σ,
(3) if B1 , B2 , . . . ∈ Σ then n Bn ∈ Σ.
Let C be a family of subsets of X . Then there exists a smallest σ-algebra containing
all sets from C . It is the intersection of all σ-algebras of subsets of X containing the
family C . We denote it by σ(C ) and we call it the σ-algebra generated by C . Note
that if X is a metric space, then the Borel σ-algebra B(X ) is the σ-algebra generated
by all open subsets of X and its elements are called Borel sets.
Let (X 1 , Σ1 ) and (X 2 , Σ2 ) be measurable spaces. For B1 ∈ Σ1 and B2 ∈ Σ2 we
write B1 × B2 for the set of all pairs (x1 , x2 ) with x1 ∈ B1 and x2 ∈ B2 and call it a
measurable rectangle in X 1 × X 2 . We let Σ1 ⊗ Σ2 to be the σ-algebra generated by
the family of all measurable rectangles, so that

Σ1 ⊗ Σ2 = σ(C ), where C = {B1 × B2 : B1 ∈ Σ1 , B2 ∈ Σ2 }.

Note that the intersection of two measurable rectangles is a measurable rectangle. The
σ-algebra Σ1 ⊗ Σ2 is called the product σ-algebra on X 1 × X 2 , and the measurable
space (X 1 × X 2 , Σ1 ⊗ Σ2 ) is called the product of (X 1 , Σ1 ) and (X 2 , Σ2 ).

© The Author(s) 2017 149

R. Rudnicki and M. Tyran-Kamińska, Piecewise Deterministic Processes
in Biological Models, SpringerBriefs in Mathematical Methods,
DOI 10.1007/978-3-319-61295-9
150 Appendix A: Measure and Probability Essentials

A family C of subsets of X is called a π-system if it is closed under intersections,

i.e. if B1 , B2 ∈ C then B1 ∩ B2 ∈ C . A family B of subsets of X is a λ-system if:
(1) X ∈ B.
(2) If B1 , B2 ∈ B and B1 ⊂ B2 then B2 \ B 1 ∈ B.
(3) If B1 , B2 , . . . ∈ B and Bn ⊂ Bn+1 then n Bn ∈ B.
Note that a family of subsets of X is a σ-algebra if and only if it is a π-system and
λ-system. We have the fundamental result for establishing measurability.
Lemma A.1 (π−λ lemma) If C is a π-system and B is a λ-system such that C ⊂ B
then σ(C ) ⊂ B.
A mapping f : X 1 → X 2 is said to be measurable if the counter image of any
B ∈ Σ2 under f is a measurable subset of X 1 , i.e.

f −1 (B) = {x ∈ X 1 : f (x) ∈ B} ∈ Σ1 .

If X 2 = R then we take Σ2 = B(R). If we want to underline the σ-algebra in the

domain X 1 then we say that f is measurable with respect to Σ1 , or Σ1 -measurable.
A measurable function f : X → R is called a simple function if there exist a finite
number of pairwise disjoint sets B1 , . . . , Bk ∈ Σ and constants c1 , . . . , ck such that

f (x) = ci 1 Bi (x), x ∈ X.
i

Observe that every non-negative measurable function f : X → [0, ∞] is a point-

wise limit of a monotone sequence of simple functions, since

f (x) = lim f n (x), x ∈ X,

n→∞

where

(k − 1)2−n , if (k − 1)2−n ≤ f (x) < k2−n , 1 ≤ k ≤ n2n ,
f n (x) =
n, if n ≤ f (x).

Finally, if f is measurable then f = f + − f − , where f + = max{0, f } and f − =

max{0, − f } are, respectively, the positive part and the negative part of f .
Many results concerning measurable functions are usually easily proved for indi-
cator functions and we can extend them to all measurable functions by using the
monotone class argument based on the following result.
Theorem A.1 (Monotone class theorem) Let C be a π-system of subsets of X such
that X ∈ C . Let H be a class of real-valued functions on X such that
(1) If B ∈ C then 1 B ∈ H .
(2) If f, g ∈ H then f + g ∈ H and c f ∈ H for any non-negative (resp. real)
constant c.
Appendix A: Measure and Probability Essentials 151

(3) If f n ∈ H are non-negative and f n ↑ f with a non-negative (and bounded) f

then f ∈ H .
Then H contains all non-negative (bounded) functions measurable with respect to
σ(C ).

A. 2 Measure and Integration

A measure on (X, Σ) is a mapping μ : Σ → [0, ∞] such that μ(∅) = 0 and μ is

countably additive, i.e.  
μ( Bn ) = μ(Bn )
n n

for every sequence of disjoint sets Bn ∈ Σ. A measure μ is called a probability

measure if μ(X ) = 1, a finite measure if μ(X ) < ∞, and a σ-finite measure if there
is a sequence B j ∈ Σ such that X = j B j and μ(B j ) < ∞, for each j. The
triple (X, Σ, μ) is said to be a (probability, finite, σ-finite) measure space if μ is a
(probability, finite, σ-finite) measure on (X, Σ).

Theorem A.2 (Uniqueness) Let μ and ν be two finite measures on (X, Σ). Suppose
that C is a π-system such that X ∈ C and σ(C ) = Σ. If μ(B) = ν(B) for all B ∈ C
then μ = ν.

Let(X, Σ, μ) be a measure space. The Lebesgue integral of a simple function

f = i ci 1 Bi is defined as
 
f (x) μ(d x) = ci μ(Bi )
X i

and that of a non-negative measurable function f as

 
f (x) μ(d x) = lim f n (x) μ(d x),
X n→∞ X

where ( f n ) is a non-decreasing sequence of simple measurable functions converging

pointwise to f . Finally, if f is measurable then we set
  
f (x) μ(d x) = f + (x) μ(d x) − f − (x) μ(d x)
X X X

if at least one of the terms

 
f + (x) μ(d x), f − (x) μ(d x)
X X
152 Appendix A: Measure and Probability Essentials

is finite. If both are finite then f is said to be μ-integrable, or simply integrable.

Lemma A.2 (Fatou) Let (X, Σ, μ) be a measure space and let ( f n ) be a sequence
of non-negative measurable functions. Then
 
lim inf f n (x) μ(d x) ≤ lim inf f n (x) μ(d x).
X n→∞ n→∞ X

Theorem A.3 (Lebesgue) Let (X, Σ, μ) be a measure space and let ( f n ) be a

sequence of measurable functions.
(1) Monotone convergence: If 0 ≤ f n (x) ≤ f n+1 (x) for all n and x then
 
lim f n (x) μ(d x) = lim f n (x) μ(d x). (A.1)
n→∞ X X n→∞

(2) Dominated convergence: If g is integrable, | f n (x)| ≤ g(x) for all n and x, and
the sequence ( f n ) is convergent pointwise, then (A.1) holds.

The next theorem, referred to usually as the Fubini theorem, is about the existence
of a product measure and the change of the order of integration in double integrals.

Theorem A.4 (Fubini) Let (X 1 , Σ1 , μ1 ) and (X 2 , Σ2 , μ2 ) be σ-finite measure

spaces. Then there exists a unique measure μ1 ×μ2 on (X 1 × X 2 , Σ1 ⊗Σ2 ) satisfying

(μ1 × μ2 )(B1 × B2 ) = μ1 (B1 )μ2 (B2 ), B1 ∈ Σ1 , B2 ∈ Σ2 .

Moreover, if f : X 1 × X 2 → [0, ∞] is Σ1 ⊗ Σ2 -measurable then

  
f (x1 , x2 ) μ1 × μ2 (d x1 , d x2 ) = f (x1 , x2 ) μ2 (d x2 ) μ1 (d x1 )
X 1 ×X 2 X1 X2
 
= f (x1 , x2 ) μ1 (d x1 ) μ2 (d x2 ).
X2 X1

The result remains true for any μ1 × μ2 -integrable f . The measure μ1 × μ2 is called
the product measure of μ1 and μ2 .

For a given measure space (X, Σ, μ) a finite signed measure ν on (X, Σ) is called
absolutely continuous with respect to μ if ν(B) = 0 whenever μ(B) = 0. In this
case ν is also said to be dominated by μ, written ν μ.

Theorem A.5 (Radon–Nikodym) Let (X, Σ, μ) be a σ-finite measure space and let
ν be a finite signed measure on (X, Σ) absolutely continuous with respect to μ. Then
there exists f ∈ L 1 (X, Σ, μ) such that

ν(B) = f (x) μ(d x) for all B ∈ Σ.
B
Appendix A: Measure and Probability Essentials 153

The function f is unique as an element of L 1 (X, Σ, μ) and it is called the Radon–

dν
Nikodym derivative of ν with respect to μ.
dμ

A. 3 Random Variables

Let (Ω, F , P) be a probability space and let (X, Σ) be a measurable space. An

X -valued random variable ξ is called discrete, if X is a finite or a countable set.
If X = Rd then we say that a random variable is continuous if its distribution
is absolutely continuous with respect to the Lebesgue measure. Then the Radon–
Nikodym theorem implies that the distribution of ξ has a density, i.e. there exists a
Borel measurable non-negative function f ξ such that

μξ (B) = P(ξ ∈ B) = f ξ (x) d x, B ∈ B(X ).
B

Given a real-valued random variable ξ the function

F(t) = P(ξ ≤ t), t ∈ R,

is called the distribution function of the random variable ξ. It is non-decreasing,

right-continuous, and

lim F(t) = 0, lim F(t) = 1.

t→−∞ t→∞

A function F : R → R is called absolutely continuous if and only if for every

ε > 0 there exists a δ > 0 such that for each finite collection of disjoint intervals
(a1 , b1 ), . . . , (ak , bk )


k 
k
|F(b j ) − F(a j )| < ε if (b j − a j ) < δ.
j=1 j=1

In particular, a distribution function of a real-valued random variable is absolutely

continuous if and only if the random variable has a density.
The expectation of a real-valued random variable ζ is defined as the integral of ζ
with respect to P  
E(ζ) = ζ(ω)P(dω) = ζ dP.
Ω

If ζ = h(ξ) with measurable h and if the X -valued random variable ξ has distribution
μξ then
154 Appendix A: Measure and Probability Essentials

E(h(ξ)) = h(x) μξ (d x).
X

In particular, if the random variable ξ is discrete then


E(h(ξ)) = h(x)P(ξ = x)
x∈X

and if ξ is continuous with density f ξ then


E(h(ξ)) = h(x) f ξ (x) d x.
X

Families F1 , . . . , Fn ⊂ F of events are said to be independent if for any sets

A1 ∈ F1 , . . . , An ∈ Fn we have

P(A1 ∩ . . . ∩ An ) = P(A1 ) . . . P(An ).

We say that ξ1 , . . . , ξn are independent random variables if the corresponding σ-

algebras σ(ξ1 ), . . . , σ(ξn ) are independent. A collection of random variables is said
to be independent if each finite number of them is independent. We have the following
characterization.

Theorem A.6 Let ξ1 , . . . , ξn be random variables with values in some measurable

spaces X 1 , . . . , X n and have distributions μ1 , . . . , μn . Then ξ1 , . . . , ξn are indepen-
dent if and only if the distribution of (ξ1 , . . . , ξn ) is the product measure μ1 ×· · ·×μn .

Theorem A.7 (Strong law of large numbers) Let ξ1 , ξ2 , . . . be i.i.d. random vari-
ables such that E(|ξ1 |) < ∞. Then

1
n
P lim ξk = E(ξ1 ) = 1.
n→∞ n
k=1

A. 4 Conditional Expectations and Distributions

Let (Ω, F , P) be a probability space. Suppose that ζ : Ω → R is an integrable

random variable, i.e. E|ζ| < ∞. The conditional expectation of ζ given an event
A ∈ F with P(A) > 0 is defined as the number
 
1
E(ζ|A) = ζ(ω)P(dω|A) = ζ(ω)P(dω),
Ω P(A) A

where P(·|A) : F → [0, 1] is the probability measure defined by

Appendix A: Measure and Probability Essentials 155

P(A ∩ B)
P(B|A) = , B ∈ F.
P(A)

Given a σ-algebra G ⊂ F the conditional expectation of ζ with respect to G is a

real-valued random variable η such that η is G -measurable and
 
ζ(ω)P(dω) = η(ω)P(dω) for all A ∈ G . (A.2)
A A

It is denoted by E(ζ|G ).

Theorem A.8 (Existence) If ζ is an integrable random variable then the conditional

expectation of ζ with respect to a σ-algebra G exists and is unique, as an element of
L 1 (Ω, G , P).

Theorem A.9 (Properties of conditional expectation) The following properties hold

whenever the corresponding expressions exist for the absolute values:
(1) E(E(ζ|G )) = E(ζ).
(2) E(ζ1 + ζ2 |G ) = E(ζ1 |G ) + E(ζ2 |G ) and E(cζ|G ) = cE(ζ|G ).
(3) If ζ ≥ 0 then E(ζ|G ) ≥ 0.
(4) If ζn ↑ ζ then E(ζn |G ) ↑ E(ζ|G ).
(5) If ξ is G -measurable then E(ξζ|G ) = ξE(ζ|G ).
(6) If G ⊂ H then E(E(ζ|H )|G ) = E(ζ|G ).

The conditional probability of an event A ∈ F , given a σ-algebra G , is defined as

P(A|G ) = E(1 A |G ).

If G = σ(ξ) then we write E(ζ|ξ) to denote E(ζ|σ(ξ)), thus we also have

P(A|ξ) = E(1 A |σ(ξ)), A ∈ F.

In particular, the random variable P(ζ ∈ B|ξ), being σ(ξ) measurable, can be repre-
sented as h B (ξ) for some measurable h B , since we have the following.
Theorem A.10 (Representation theorem) Let ξ be an X -valued random variable.
Then a real-valued random variable η is measurable with respect to σ(ξ) if and only
if there exists a measurable h : X → R such that η = h(ξ).
The next result is useful for the computation of conditional expectations.

Lemma A.3 Let ξ be a G -measurable X -valued random variable and ζ be inde-

pendent of G . If g is a measurable non-negative function and h(x) = E(g(x, ζ)),
x ∈ X , then
E(g(ξ, ζ)|G ) = h(ξ). (A.3)
156 Appendix A: Measure and Probability Essentials

Proof Since ξ is G -measurable, the random variable h(ξ) is G -measurable. It remains

to show that  
g(ξ(ω), ζ(ω)) P(dω) = h(ξ(ω)) P(dω)
A A

for every A ∈ G , or, equivalently, E(ηg(ξ, ζ)) = E(ηh(ξ)) for every positive G -
measurable random variable η. Let μζ and μ(ξ,η) denote the distributions of ζ and
(ξ, η). We have 
h(x) = E(g(x, ζ)) = g(x, y) μζ (dy).

Since (ξ, η) and ζ are independent, we obtain


E(ηg(ξ, ζ)) = zg(x, y) μ(ξ,η) (d x, dz) μζ (dy)
 
= z g(x, y) μζ (dy) μ(ξ,η) (d x, dz),

which implies that


E(ηg(ξ, ζ)) = zh(x) μ(ξ,η) (d x, dz) = E(ηh(ξ))

and completes the proof.

Let K , G , H ⊆ F be σ-algebras such that H ⊆ G ∩ K . We say that G and

K are conditionally independent given H , if

P(A ∩ B|H ) = P(A|H )P(B|H ), A ∈ G, B ∈ K .

Lemma A.4 Let K , G , H ⊆ F be σ-algebras such that H ⊆ G ∩ K . Then the

following are equivalent:
(1) P(A ∩ B|H ) = P(A|H )P(B|H ) for all A ∈ G , B ∈ K .
(2) P(A|K ) = P(A|H ) for all A ∈ G .

Proof We show that (1) implies (2). Since for any A the random variable P(A|H )
is K -measurable, it is enough to show that for every B ∈ K we have

E(1 B E(1 A |K )) = E(1 B E(1 A |H )).

Since 1 B is K -measurable, we obtain

E(1 B E(1 A |K )) = E(E(1 B 1 A |K )) = E(1 B 1 A ) = P(A ∩ B).

On the other hand

Appendix A: Measure and Probability Essentials 157

E(1 B E(1 A |H )) = E(E(1 B E(1 A |H )|H )) = E(E(1 A |H )E(1 B |H ))

which, by assumption, we can rewrite as

E(E(1 B |H )E(1 A |H )) = E(E(1 B∩A |H )) = P(A ∩ B).

Finally, to show that (2) implies (1), we observe that

P(B ∩ A|H ) = E(1 B∩A |H ) = E(E(1 B∩A |K )|H ) = E(1 B E(1 A |K )|H )
= E(1 B E(1 A |H )|H ) = E(1 A |H )E(1 B |H ).
References

1. Allen, L.J.S.: An Introduction to Stochastic Processes with Applications to Biology. Chapman

Hall/CRC Press, Boca Raton (2010)
2. Almeida, C.R., de Abreu, F.V.: Dynamical instabilities lead to sympatric speciation. Evol.
Ecol. Res. 5, 739–757 (2003)
3. Alur R., Pappas, G.J. (eds.): Hybrid Systems: Computation and Control. LNCS, vol. 2993.
Springer, Berlin (2004)
4. Arendt, W.: Resolvent positive operators. Proc. Lond. Math. Soc. 54, 321–349 (1987)
5. Arendt, W., Grabosch, A., Greiner, G., Groh, U., Lotz, H.P., Moustakas, U., Nagel, R.,
Neubrander, F., Schlotterbeck, U.: One-parameter semigroups of positive operators. Lecture
Notes in Mathematics, vol. 1184. Springer, Berlin (1986)
6. Arino, O., Rudnicki, R.: Phytoplankton dynamics. C. R. Biologies 327, 961–969 (2004)
7. Bachar, M., Batzel, J., Ditlevsen, S. (eds.): Stochastic Biomathematical Models with Appli-
cations to Neuronal Modeling. Lecture Notes in Mathematics, vol 2058, Springer, Berlin,
Heidelberg (2013)
8. Bakhtin, Y., Hurth, T.: Invariant densities for dynamical system with random switching. Non-
linearity 25, 2937–2952 (2012)
9. Banasiak, J.: On an extension of the Kato-Voigt perturbation theorem for substochastic semi-
groups and its application. Taiwanese J. Math. 5, 169–191 (2001)
10. Banasiak, J., Arlotti, L.: Perturbations of positive semigroups with applications. London Ltd.,
Springer Monographs in Mathematics. Springer, London (2006)
11. Banasiak, J., Pichór, K., Rudnicki, R.: Asynchronous exponential growth of a general struc-
tured population model. Acta Applicandae Mathematicae 119, 149–166 (2012)
12. Bell, G.I., Anderson, E.C.: Cell growth and division I. A Mathematical model with applications
to cell volume distributions in mammalian suspension cultures. Biophys. J. 7, 329–351 (1967)
13. Benaïm, M., le Borgne, S., Malrieu, F., Zitt, P.A.: Qualitative properties of certain piecewise
deterministic Markov processes. Ann. Inst. H. Poincaré Probab. Stat. 51, 1040–1075 (2015)
14. Bernoulli, D., Blower, S.: An attempt at a new analysis of the mortality caused by smallpox
and of the advantages of inoculation to prevent it. Rev. Med. Virol. 14, 275–288 (2004)
15. Biedrzycka, W., Tyran-Kamińska, M.: Existence of invariant densities for semiflows with
jumps. J. Math. Anal. Appl. 435, 61–84 (2016)
16. Blom, H.A.P., Lygeros, J. (eds.): Stochastic Hybrid Systems: Theory and Safety Critical
Applications. Series: Lecture Notes in Control and Information Sciences, vol. 337. Springer,
Berlin Heidelberg (2006)

© The Author(s) 2017 159

R. Rudnicki and M. Tyran-Kamińska, Piecewise Deterministic Processes
in Biological Models, SpringerBriefs in Mathematical Methods,
DOI 10.1007/978-3-319-61295-9
160 References

17. Bobrowski, A., Lipniacki, T., Pichór, K., Rudnicki, R.: Asymptotic behavior of distributions
of mRNA and protein levels in a model of stochastic gene expression. J. Math. Anal. Appl.
333, 753–769 (2007)
18. Bokes, P., King, J.R., Wood, A.T.A., Loose, M.: Transcriptional bursting diversifies the behav-
iour of a toggle switch: Hybrid simulation of stochastic gene expression. Bull. Math. Biol.
75, 351–371 (2013)
19. Boulanouar, M.: A mathematical study for a Rotenberg model. J. Math. Anal. Appl. 265,
371–394 (2002)
20. Bressloff, P., Newby, J.: Directed intermittent search for hidden targets. New J. Phys. 11,
023033 (2009)
21. Bressloff, P., Newby, J.: Quasi-steady state reduction of molecular motor-based models of
directed intermittent search. Bull. Math. Biol. 72, 1840–1866 (2010) Rev. Mod. Phys. 85,
135–196 (2013)
22. Bressloff, P., Newby, J.: Stochastic models of intracellular transport. Rev. Mod. Phys. 85,
135–196 (2013)
23. Buckwar, E., Riedler, M.G.: An exact stochastic hybrid model of excitable membranes includ-
ing spatio-temporal evolution. J. Math. Biol. 63, 1051–1093 (2011)
24. Burkitt, A.N.: A review of the integrate-and-fire neuron model: I Homogeneous synaptic
input. Biol. Cybern. 95, 1–19 (2006)
25. Cai, L., Friedman, N., Xie, X.S.: Stochastic protein expression in individual cells at the single
molecule level. Nature 440, 358–362 (2006)
26. Capasso, V., Bakstein, D.: An Introduction to Continuous-Time Stochastic Processes. Theory,
Models and Applications to Finance, Biology and Medicine. Birkhäuser, Boston (2005)
27. Cassandras, C.G., Lygeros, J. (eds.): Stochastic Hybrid Systems. Control Engineering Series,
vol. 24, CRC Press, Boca Raton, FL (2007)
28. Davis, M.H.A.: Piecewise-deterministic Markov processes: a general class of nondiffusion
stochastic models. J. R. Stat. Soc. Ser. B 46, 353–388 (1984)
29. Davis, M.H.A.: Markov Models and Optimization. Monographs on Statistics and Applied
Probability, vol. 49, Chapman & Hall, London (1993)
30. Desch, W.: Perturbations of positive semigroups in AL-spaces. Unpublished (1988)
31. Diekmann, O., Heijmans, H.J.A.M., Thieme, H.R.: On the stability of the cell size distribution.
J. Math. Biol. 19, 227–248 (1984)
32. Dieudonne, J.: Sur le théorème de Radon-Nikodym. Ann. Univ. Grenoble 23, 25–53 (1948)
33. Dunford, N.: On one parameter groups of linear transformations. Ann. Math. 39, 569–573
(1938)
34. Engel, K.J., Nagel, R.: One-parameter semigroups for linear evolution equations. Graduate
Texts in Mathematics, vol. 194. Springer, New York (2000)
35. Ethier, S.N., Kurtz, T.G.: Markov Processes. Characterization and Convergence. Wiley, New
York (1986)
36. Evans, S.N.: Stochastic billiards on general tables. Ann. Appl. Probab. 11, 419–437 (2001)
37. Feller, W.: An introduction to Probability Theory and Applications. Wiley, New York (1966)
38. Foguel, S.R.: The Ergodic Theory of Markov Processes. Van Nostrand Reinhold Comp, New
York (1969)
39. Fox, R.F.: Stochastic versions of the Hodgkin-Huxley equations. Biophys. J. 72, 2068–2074
(1997)
40. Friedman, N., Cai, L., Xie, X.S.: Linking stochastic dynamics to population distribution: an
analytical framework of gene expression. Phys. Rev. Lett. 97, 168302 (2006)
41. Gavrilets, S., Boake, C.R.B.: On the evolution of premating isolation after a founder event.
Am. Nat. 152, 706–716 (1998)
42. Genadot, A., Thieullen, M.: Averaging for a fully coupled piecewise-deterministic Markov
process in infinite dimensions. Adv. Appl. Probab. 44, 749–773 (2012)
43. Golding, I., Paulsson, J., Zawilski, S., Cox, E.: Real-time kinetics of gene activity in individual
bacteria. Cell 123, 1025–1036 (2005)
References 161

44. Greiner, G.: Perturbing the boundary conditions of a generator. Houston J. Math. 13, 213–229
(1987)
45. Gwiżdż, P., Tyran-Kamińska, M.: Densities for piecewise deterministic Markov processes
with boundary. Preprint
46. Gyllenberg, M., Heijmans, H.J.A.M.: An abstract delay-differential equation modelling size
dependent cell growth and division. SIAM J. Math. Anal. 18, 74–88 (1987)
47. Hasminskiı̆, R.Z.: Ergodic properties of recurrent diffusion processes and stabilization of
the solutions of the Cauchy problem for parabolic equations (in Russian). Teor. Verojatn.
Primenen. 5, 196–214 (1960)
48. Hennequin, P., Tortrat, A.: Théorie des probabilités et quelques applications. Masson et Cie,
Paris (1965)
49. Hille, E.: Functional Analysis and Semi-Groups. American Mathematical Society Colloquium
Publications, vol. 31. American Mathematical Society, New York (1948)
50. Hillen, T., Hadeler, K.P.: Hyperbolic systems and transport equations in mathematical biology.
In: Warnecke, G. (ed.) Analysis and Numerics for Conservation Laws, pp. 257–279. Springer,
Berlin, Heidelberg (2005)
51. Hu, J., Lygeros, J. Sastry, S.: Towards a theory of stochastic hybrid systems. In: Lynch, N.,
Krogh, B,H. (eds.) Hybrid Systems: Computation and Control. LNCS vol. 1790, pp. 160–173.
Springer, Berlin (2000)
52. Hu, J., Wu, W.C., Sastry, S.S.: Modeling subtilin production in bacillus subtilis using stochastic
hybrid systems. In: Alur, R., Pappas, G.J. (eds.) Hybrid Systems: Computation and Control.
LNCS, vol. 2993, pp. 417–431. Springer, Berlin (2004)
53. Jacod, J., Skorokhod, A.V.: Jumping Markov processes. Ann. Inst. H. Poincaré Probab. Statist.
32, 11–67 (1996)
54. Jamison, B., Orey, S.: Markov chains recurrent in the sense of Harris Z. Wahrsch. Verw.
Gebiete 8, 41–48 (1967)
55. Kallenberg, O.: Foundations of Modern Probability, 2nd edn. Springer, New York (2002)
56. Kato, T.: On the semi-groups generated by Kolmogoroff’s differential equations. J. Math.
Soc. Jpn 6, 1–15 (1954)
57. Kokko, H., Jennions, M.D., Brooks, R.: Unifying and testing models of sexual selection.
Annu. Rev. Ecol. Evol. Syst. 37, 43–66 (2006)
58. Komorowski T., Tyrcha, J.: Asymptotic properties of some Markov operators. Bull. Pol. Ac.:
Math. 37, 221–228 (1989)
59. Kouretas, P., Koutroumpas, K., Lygeros, J., Lygerou, Z.: Stochastic hybrid modeling of bio-
chemical processes. In: Cassandras, C.G., Lygeros, J. (eds.) Stochastic Hybrid Systems. Con-
trol Engineering Series, vol. 24, pp. 221–248. CRC Press, Boca Raton, FL (2007)
60. Kuno, E.: Simple mathematical models to describe the rate of mating in insect populations.
Res. Popul. Ecol. 20, 50–60 (1978)
61. Lasota, A., Mackey, M.C.: Globally asymptotic properties of proliferating cell populations.
J. Math. Biol. 19, 43–62 (1984)
62. Lasota, A., Mackey, M.C.: Chaos, Fractals, and Noise, Applied Mathematical Sciences, vol.
97. Springer, New York (1994)
63. Lasota, A., Yorke, J.A.: Exact dynamical systems and the Frobenius-Perron operator. Trans.
AMS 273, 375–384 (1982)
64. Lebowitz, J.L., Rubinow, S.L.: A theory for the age and generation time distribution of micro-
bial population. J. Math. Biol. 1, 17–36 (1974)
65. Lipniacki, T., Paszek, P., Marciniak-Czochra, A., Brasier, A.R., Kimmel, M.: Transcriptional
stochasticity in gene expression. J. Theoret. Biol. 238, 348–367 (2006)
66. Lotz, H.P.: Uniform convergence of operators on L ∞ and similar spaces. Math Z 190, 207–220
(1985)
67. Lumer, G., Phillips, R.S.: Dissipative operators in a Banach space. Pacific J. Math. 11, 679–698
(1961)
68. Mackey, M.C., Rudnicki, R.: Global stability in a delayed partial differential equation describ-
ing cellular replication. J. Math. Biol. 33, 89–109 (1994)
162 References

69. Mackey, M.C., Santillán, M., Tyran-Kamińska, M., Zeron, E.S.: Simple Mathematical Models
of Gene Regulatory Dynamics. Lecture Notes on Mathematical Modelling in the Life Sciences.
Springer, Cham (2016)
70. Mackey, M.C., Tyran-Kamińska, M.: Dynamics and density evolution in piecewise determin-
istic growth processes. Ann. Polon. Math. 94, 111–129 (2008)
71. Mackey, M.C., Tyran-Kamińska, M., Yvinec, R.: Molecular distributions in gene regulatory
dynamics. J. Theoret. Biol. 274, 84–96 (2011)
72. Mackey, M.C., Tyran-Kamińska, M., Yvinec, R.: Dynamic behavior of stochastic gene expres-
sion models in the presence of bursting. SIAM J. Appl. Math. 73, 1830–1852 (2013)
73. Mackey, M.C., Tyran-Kamińska, M.: The limiting dynamics of a bistable molecular switch
with and without noise. J. Math. Biol. 73, 367–395 (2016)
74. McKendrick, A.G.: Application of mathematics to medical problems. Proc. Edinb. Math. Soc.
14, 98–130 (1926)
75. Méndez, V., Campos, D., Pagonabarraga, I., Fedotov, S.: Density-dependent dispersal and
population aggregation patterns. J. Theoret. Biol. 309, 113–120 (2012)
76. Metz, J.A.J., Diekmann, O. (eds.): The Dynamics of Physiologically Structured Populations.
Springer Lecture Notes in Biomathematics, vol. 68. Springer, New York (1986)
77. Miȩkisz, J., Szymańska, P.: Gene expression in self-repressing system with multiple gene
copies. Bull. Math. Biol. 75, 317–330 (2013)
78. Mokhtar-Kharroubi, M., Rudnicki, R.: On asymptotic stability and sweeping of collisionless
kinetic equations. Acta Appl. Math. 147, 19–38 (2017)
79. Nummelin, E.: General Irreducible Markov Chains and Non-negative Operators, Cambridge
Tracts in Mathematics, vol. 83. Cambridge University Press, Cambridge (1984)
80. Ochab-Marcinek, A., Tabaka, M.: Transcriptional leakage versus noise: a simple mechanism
of conversion between binary and graded response in autoregulated genes. Phys. Rev. E 91,
012704 (2015)
81. Othmer, H.G., Dunbar, S.R., Alt, W.: Models of dispersal in biological systems. J. Math. Biol.
26, 263–298 (1988)
82. Othmer, H.G., Hillen, T.: The diffusion limit of transport equations II: Chemotaxis equations.
SIAM J. Appl. Math. 62, 1222–1250 (2002)
83. Phillips, R.S.: Semi-groups of positive contraction operators. Czechoslovak Math. J. 12, 294–
313 (1962)
84. Pichór, K.: Asymptotic stability of a partial differential equation with an integral perturbation.
Ann. Polon. Math. 68, 83–96 (1998)
85. Pichór, K.: Asymptotic stability and sweeping of substochastic semigroups. Ann. Polon. Math.
103, 123–134 (2012)
86. Pichór, K., Rudnicki, R.: Stability of Markov semigroups and applications to parabolic sys-
tems. J. Math. Anal. Appl. 215, 56–74 (1997)
87. Pichór, K., Rudnicki, R.: Continuous Markov semigroups and stability of transport equations.
J. Math. Anal. Appl. 249, 668–685 (2000)
88. Pichór, K., Rudnicki, R.: Asymptotic behaviour of Markov semigroups and applications to
transport equations. Bull. Pol. Ac.: Math. 45, 379–397 (1997)
89. Pichór, K., Rudnicki, R.: Asymptotic decomposition of substochastic operators and semi-
groups. J. Math. Anal. Appl. 436, 305–321 (2016)
90. Pichór, K., Rudnicki, R.: Asymptotic decomposition of substochastic semigroups and appli-
cations. Stochastics and Dynamics, (accepted)
91. Pichór, K., Rudnicki, R.: Stability of stochastic semigroups and applications to Stein’s neu-
ronal model. Discrete Contin. Dyn. Syst. B, (accepted)
92. Pichór, K., Rudnicki, R., Tyran-Kamińska, M.: Stochastic semigroups and their applications
to biological models. Demonstratio Mathematica 45, 463–495 (2012)
93. Rotenberg, M.: Transport theory for growing cell populations. J. Theoret. Biol. 103, 181–199
(1983)
94. Rubinow, S.I.: A maturity time representation for cell populations. Biophys. J. 8, 1055–1073
(1968)
References 163

95. Rudnicki, R.: On asymptotic stability and sweeping for Markov operators. Bull. Pol. Ac.:
Math. 43, 245–262 (1995)
96. Rudnicki, R.: Stochastic operators and semigroups and their applications in physics and biol-
ogy. In: Banasiak, J., Mokhtar-Kharroubi, M. (eds.) Evolutionary Equations with Applications
in Natural Sciences. Lecture Notes in Mathematics, vol. 2126, pp. 255–318. Springer, Hei-
delberg (2015)
97. Rudnicki, R., Pichór, K.: Markov semigroups and stability of the cell maturation distribution.
J. Biol. Syst. 8, 69–94 (2000)
98. Rudnicki, R., Tiuryn, J., Wójtowicz, D.: A model for the evolution of paralog families in
genomes. J. Math. Biol. 53, 759–770 (2006)
99. Rudnicki, R., Tiuryn, J.: Size distribution of gene families in a genome. Math. Models Methods
Appl. Sci. 24, 697–717 (2014)
100. Rudnicki, R., Tomski, A.: On a stochastic gene expression with pre-mRNA, mRNA and
protein contribution. J. Theoret. Biol. 387, 54–67 (2015)
101. Rudnicki, R., Tyran-Kamińska, M.: Piecewise deterministic Markov processes in biological
models. Semigroups of Operators Theory And Applications. Springer Proceedings in Math-
ematics & Statistics 113, pp. 235–255. Springer, Cham (2015)
102. Rudnicki, R., Wieczorek, R.: Mathematical models of phytoplankton dynamics. In: Russo R.
(ed.) Aquaculture I. Dynamic Biochemistry, Process Biotechnology and Molecular Biology,
vol. 2 (Special Issue 1), pp. 55–63 (2008)
103. Rudnicki, R., Wieczorek, R.: Phytoplankton dynamics: from the behaviour of cells to a trans-
port equation. Math. Model. Nat. Phenom. 1, 83–100 (2006)
104. Rudnicki, R., Zwoleński, P.: Model of phenotypic evolution in hermaphroditic populations.
J. Math. Biol. 70, 1295–1321 (2015)
105. Sharpe, F.R., Lotka, A.J.: A problem in age-distributions. Philos. Mag. 21, 435–438 (1911)
106. Singh, A., Hespanha, J.P.: Stochastic hybrid systems for studying biochemical processes.
Philos. Trans. R. Soc. A 368, 4995–5011 (2010)
107. Stein, R.B.: A theoretical analysis of neuronal variability. Biophys. J. 5, 173–194 (1965)
108. Stein, R.B.: Some models of neuronal variability. Biophys. J. 7, 37–68 (1967)
109. Stroock, D.W.: Some stochastic processes which arise from a model of the motion of a
bacterium. Z. Wahrscheinlichkeitstheorie verw. Gebiete 28, 305–315 (1974)
110. Tuckwell, H.C.: Stochastic Processes in the Neurosciences. SIAM, Philadelphia (1989)
111. Tuckwell, H.C.: Synaptic transmission in a model for stochastic neural activity. J. Theoret.
Biol. 77, 65–81 (1979)
112. Tyson, J.J., Hannsgen, K.B.: Cell growth and division: a deterministic/probabilistic model of
the cell cycle. J. Math. Biol. 23, 231–246 (1986)
113. Tyran-Kamińska, M.: Substochastic semigroups and densities of piecewise deterministic
Markov processes. J. Math. Anal. Appl. 357, 385–402 (2009)
114. Tyran-Kamińska, M.: Ergodic theorems and perturbations of contraction semigroups. Studia
Math. 195, 147–155 (2009)
115. Tyrcha, J.: Asymptotic stability in a generalized probabilistic/deterministic model of the cell
cycle. J. Math. Biol. 26, 465–475 (1988)
116. Voigt, J.: On substochastic C0 -semigroups and their generators. Transp. Theory Statist. Phys.
16, 453–466 (1987)
117. Voigt, J.: On resolvent positive operators and positive C0 -semigroups on AL-spaces. Semi-
group Forum 38, 263–266 (1989)
118. Webb, G.W.: Theory of Nonlinear Age-Dependent Population Dynamics. Marcel Dekker,
New York (1985)
119. Yin, G.G., Zhu, C.: Hybrid Switching Diffusions: Properties and Applications. Stochastic
Modelling and Applied Probability, vol. 63. Springer, New York (2010)
120. Yosida, K.: On the differentiability and the representation of one-parameter semi-group of
linear operators. J. Math. Soc. Jpn 1, 15–21 (1948)
121. Yu, J., Xiao, J., Ren, X., Lao, K., Xie, X.: Probing gene expression in live cells, one protein
molecule at a time. Science 311, 1600–1603 (2006)
164 References

122. Zeiser, S., Franz, U., Müller, J., Liebscher, V.: Hybrid modeling of noise reduction by a
negatively autoregulated system. Bull. Math. Biol. 71, 1006–1024 (2009)
123. Zeiser, S., Franz, U., Liebscher, V.: Autocatalytic genetic networks modeled by piecewise-
deterministic Markov processes. J. Math. Biol. 60, 207–246 (2010)
Index

A Characteristics of PDMP, 32
Absolutely continuous Closable operator, 76
function, 153 Closed operator, 64
measure, 154 Closure of a linear operator, 76
Active boundary, 31 Coagulation-fragmentation process, 28
Adiabatic limit, 139 Compact operator, 93
Adjoint Complete metric space, 35
operator, 38, 39, 42, 64, 76, 93, 94 Composition operator, 42
semigroup, 95 Compound Poisson process, 3, 53
Age- structured population, 24 Condition
Asymptotic stability, 116 (I), 130
Asynchronous exponential growth, 143 (K), 120, 129, 134, 135, 138, 141, 144
(WI), 123, 131, 134, 136, 137, 141, 145
(WT), 124
B Conditional
Banach space, 63 distribution, 35
Birth expectation, 34, 154, 155
pure process, 3 independence, 156
rate, 2 probability, 155
Birth–death process, 2, 130, 131 Cone, 37
Borel Conservative semigroup, 119
σ-algebra, 149 Continuous operator, 64
set, 149 Continuous-time
space, 35 Markov chain, 2, 96, 129
Bounded operator, 63 stochastic process, 49
Contraction
operator, 64
C semigroup, 65
Càdlàg, 10, 30, 49 Core, 77
Canonical space, 48 C0 -semigroup, 65
Cell cycle model, 7, 144
Chapman–Kolmogorov equation, 45, 49
D
Death
pure process, 3
rate, 2
© The Author(s) 2017 165
R. Rudnicki and M. Tyran-Kamińska, Piecewise Deterministic Processes
in Biological Models, SpringerBriefs in Mathematical Methods,
DOI 10.1007/978-3-319-61295-9
166 Index

Densely defined operator, 64 Hille–Yosida

Density, 38, 39, 153 operator, 76
invariant, 116 theorem, 73
Depolarization, 17 History, 45, 49
Dirac measure, 22 Homogeneous Markov process, 45, 49
Dispersive operator, 85 Hörmander’s condition, 136, 141
Dissipative Hybrid systems, 19
operator, 74, 75
semigroup, 119
Distribution I
conditional, 35 Identity operator, 65
function, 153 Independent
initial, 45 σ-algebras, 34
of a random variable, 34 increments, 51
of the process, 46 random variables, 154
Division rate, 5 Infinitesimal generator, 65, 106, 145
Dominated convergence theorem, 152 matrix, 2
Dual space, 64 of Markov process, 55
Dynamical system, 31 Initial distribution, 45
Dynamical systems with random switching, Integral
14, 135 kernel, 40
Dynkin formula, 55 Lebesgue, 151
Dyson–Phillips expansion, 79, 92, 102 operator, 40
Riemann, 64
semigroup, 117
E Invariant
Essential supremum, 38 density, 116
Exponential distribution, 6 set, 118
Extended generator, 56, 61, 62, 106, 110 Invertible operator, 64
Ionescu–Tulcea theorem, 47
Irreducibility, 129
F Irreducible semigroup, 117
Filtration, 49
natural, 45
Finite-dimensional distributions, 46 J
Flows with jumps, 7, 141 Jumps on a subspace, 19
Foguel alternative, 122
Frobenius–Perron operator, 41, 99, 141
Fubini theorem, 152 K
Kato theorem, 98
Kernel, 117
G integral, 40
Gamma distribution, 53 transition, 33
Gene expression, 11, 13, 20 Kolmogorov
Generator equation, 60
extended, 56, 61, 62, 106, 110 matrix, 97, 98, 130, 132
infinitesimal, of a semigroup, 65 non-explosive, 98, 130
infinitesimal, of Markov process, 55 Koopman operator, 42
Growth rate, 5

L
H Law of large numbers, 154
Hahn–Banach theorem, 64 Lebesgue
Hasminskiı̆ function, 125–127 integral, 151
Index 167

theorem, 152 Non-singular, 41

Lebowitz–Rubinow’s model, 10, 106, 146 Nullspace, 64
Lévy process, 51
Lie bracket, 136
Life-span, 5 O
Lower function, 116 One to one operator, 64
theorem of Lasota and Yorke, 116 Operator
Lumer–Phillips theorem, 76 adjoint, 38, 39, 42, 64, 76, 93, 94
bounded, 63
closable, 76
M closed, 64
Markov compact, 93
canonical process, 48 continuous, 64
chain, 129 contraction, 64
family, 51 densely defined, 64
process, 45, 49 dispersive, 85
strong, 50 dissipative, 74, 75
property, 44 extension of, 66
strong, 50 Frobenius–Perron, 99
Martingale, 55 Hille–Yosida, 76
Matrix integral, 40
Kolmogorov, 97, 98, 130, 132 invertible, 64
non-explosive, 98, 130 nullspace of, 64
stochastic, 41 one to one, 64
sub-Kolmogorov, 98, 129 positive, 38, 83
Maturation velocity, 9 quasi-compact, 93
Maturity, 9 range of, 64
Measurable space, 149 resolvent positive, 86
Measure, 151 stochastic, 38
on product space, 35 substochastic, 38
space, 151 the part of, 76
Mild solution, 66 Overlapping supports, 117
Minimal PDMP, 32, 57
Minimal semigroup, 92
Model P
cell cycle, 7 Paralog, 29
Lebowitz–Rubinow, 10, 106, 146 families, 29
Rotenberg, 11 Partially integral semigroup, 117
Rubinow, 9 Part of operator, 76
Stein, 17, 147 Perturbation theorem, 78
Monotone Phenotype structured population, 25, 26
class theorem, 150 Phillips
convergence theorem, 152 perturbation theorem, 78
Multiset, 22 theorem, 85
Physiological age, 9
Phytoplankton, 28
N Point process, 22
Negative part, 83 Poisson
Neumann expansion, 67 distribution, 52
Neural activity, 17 process, 3, 53
Non-explosive compound, 3, 53
Kolmogorov matrix, 98, 130 Polish space, 35
Markov chain, 130 Positive
168 Index

operator, 38, 83 birth, 2

part, 83 death, 2
semigroup, 84, 87, 90 growth, 5
Pre-Harris semigroup, 119 Resolvent
Probability identity, 67
distribution, 34 operator, 67
law, 34 positive operator, 86
measure, 151 set, 67
space, 151 Restriction
transition, 33 of a σ-algebra, 86
Process of a substochastic operator, 86
with independent increments, 51 of a substochastic semigroup, 86
birth-death, 2 Riemann integral, 64
coagulation-fragmentation, 28 Rotenberg, 11
continuous in probability, 51 Rotenberg’s model, 11
Lévy, 51 Rubinow’s model, 9
Markov, 45, 49 Ruelle operator, 41
piecewise deterministic, 32
Poisson, 3, 53
compound, 3, 53 S
pure Semiflow, 31
birth, 3
Semigroup, 50, 65
death, 3
adjoint, 95
pure jump-type Markov, 52
bounded, 67
right-continuous, 49
conservative, 119
stochastic, 49
contraction, 65
with jumps on a subspace, 19
dissipative, 119
with stationary increments, 51
generator of, 65
Product
integral, 117
σ-algebra, 46, 50, 149
minimal, 92
measure, 152
partially integral, 117
space, 35, 45, 47
positive, 84, 87, 90
Production of subtilin, 19
pre-Harris, 119
Pure
stochastic, 84
birth process, 3
death process, 3 strongly continuous, 65
jump-type Markov process, 3, 4, 25, 26, substochastic, 84
29, 52, 96 uniformly continuous, 69
with a kernel minorant, 117
Separable, 35
Q σ-algebra generated by
Quasi-asymptotic stability, 116 a family of sets, 149
Quasi-compact operator, 93 a random variable, 34
rectangles, 35, 149
σ-finite measure, 151
R space, 151
Radon–Nikodym Size- structured population, 21
derivative, 39, 153 Space
theorem, 38, 152 Banach, 63
Random variable Borel, 35
independent of a σ-algebra, 34 B(X ), 36
Random walk, 47 canonical, 48
Range of operator, 64 discrete, 40
Rate of dual, 64
Index 169

L ∞ (X, Σ, m), 38 Hahn–Banach, 64

L 1 (X, Σ, m), 38 Hille–Yosida, 73
M(X ), 56 Ionescu–Tulcea, 47
Polish, 35 Kato, 98
Spectral bound, 86 Lebesgue, 152
State space, 44 Lumer–Phillips, 76
Stationary increments, 51 monotone class, 150
Stein, 17, 147 monotone convergence, 152
Stein’s model, 17, 19, 147 Phillips, 85
Stochastic Radon–Nikodym, 152
hybrid systems, 19 Topological support, 121
matrix, 41 Total variation, 37
operator, 38 norm, 37
process, 44, 49 Transfer operator, 41
semigroup, 84 Transition
Stopping time, 50 function, 49
Strongly continuous semigroup, 65 kernel, 33
Strong Markov matrix, 41
process, 50 probability, 2, 4, 35, 45
property, 50 rate, 2
Structured population w.r.t. rate matrix, 2
age, 24 semigroup, 50
phenotype, 25, 26
size, 21
Sub-Kolmogorov matrix, 98, 129 U
Substochastic Uniform distribution, 34
operator, 38 Uniformly continuous semigroup, 69
semigroup, 84
Subtilin, 19
Sun dual, 95 V
semigroup, 95 Velocity jump process, 4, 20
Support, 115
Survival function, 5, 31
Sweeping, 116 W
Weakly irreducible semigroup, 123, 134
Weak tightness, 124, 125
T
Theorem
dominated convergence, 152 X
Fubini, 152 X -valued random variable, 33