Last Time
• Continuous time Markov processes
• Transition probability functions
• Functions of Markov processes
• Strong Markov property
Today’s lecture: Section 6.2
MATH136/STAT219 Lecture 25, November 21, 2008 – p. 1/12
Exponential Random Variables
• A RV X has an exponential distribution with parameter
λ > 0 (X ∼ Exp(λ)) if for all x ≥ 0,
IP (X > x) = e−λx
• Memoryless property : If X has an exponential distribution
then
IP (X > x + y|X > y) = IP (X > x) for all x, y ≥ 0
• If a continuous RV has the memoryless property, then the
RV has an exponential distribution
• If X1 , X2 , . . . are i.i.d. Exp(λ) then T = X1 + · · · + Xn has a
Gamma(n, λ) distribution with density
λe−λt (λt)n−1
fT (t) = , t≥0
(n − 1)!
MATH136/STAT219 Lecture 25, November 21, 2008 – p. 2/12
Poisson Process
• Let X1 , X2 , . . . be i.i.d. Exp(λ) random variables
• Let T0 = 0 and Tn = X1 + · · · + Xn , n = 1, 2, . . .
• A Poisson process with rate (or intensity) λ is the
continuous time stochastic process {N (t), t ≥ 0}, where
N (t) = sup{n ≥ 0 : Tn ≤ t} for t ≥ 0
• Interpretation:
◦ Xi are the times between occurrences of some event
◦ Tn is the time of the nth occurrence of the event
◦ N (t) counts the number of occurrences of the event in
the time interval [0, t]
MATH136/STAT219 Lecture 25, November 21, 2008 – p. 3/12
Illustration: Poisson Process Sample Path
3
N(t)
0
0
T1 T2 T3 T4
MATH136/STAT219 Lecture 25, November 21, 2008 – p. 4/12
Counting Process
A continuous time stochastic process {N (t), t ≥ 0} is a counting
process if
• N (0) = 0
• Sample paths of {N (t), t ≥ 0} are piecewise constant
• Sample paths of {N (t), t ≥ 0} are nondecreasing
• Sample paths of {N (t), t ≥ 0} are right-continuous
• All jump discontinuities are of size one and there are
infinitely many of them
• A counting process has state space S = {0, 1, 2, . . .}
MATH136/STAT219 Lecture 25, November 21, 2008 – p. 5/12
Counting Process: Jump Times
Associated with each sample path of a counting process
{N (t), t ≥ 0} are the jump times 0 = T0 < T1 < T2 < · · · , which
satisfy
Tn = inf{t ≥ 0 : N (t) ≥ n}, n = 0, 1, 2, . . . ,
or equivalently
N (t) = sup{n ≥ 0 : Tn ≤ t}, t ≥ 0
In particular, for any t ≥ 0, n = 0, 1, 2, . . .
N (t) = n if and only if Tn ≤ t < Tn+1
MATH136/STAT219 Lecture 25, November 21, 2008 – p. 6/12
Basic Properties of Poisson Processes
Let {N (t), t ≥ 0} be a Poisson process. Then:
• {N (t), t ≥ 0} is a counting process
• N (t) < ∞ a.s. for all t ≥ 0 and N (t) → ∞ as t → ∞ a.s.
• For each t ≥ 0, N (t) has a Poisson distribution with
parameter λt, i.e.
e−λt (λt)n
IP (N (t) = n) = , n = 0, 1, 2, . . .
n!
• For each t > s ≥ 0, N (t) − N (s) has a Poisson distribution
with parameter λ(t − s)
MATH136/STAT219 Lecture 25, November 21, 2008 – p. 7/12
Poisson Process: Equivalent Definition 1
A counting process {N (t), t ≥ 0} is a Poisson process if and
only if
• {N (t), t ≥ 0} has independent increments, and
• {N (t), t ≥ 0} has stationary increments
Furthermore, there exists λ > 0 such that for all t > s ≥ 0,
N (t) − N (s) has a Poisson distribution with parameter λ(t − s)
MATH136/STAT219 Lecture 25, November 21, 2008 – p. 8/12
Poisson Process: Equivalent Definition 2
• Let {N (t), t ≥ 0} be a counting process and let X1 , X2 , . . .
be the RV’s representing the times between jumps
• {N (t), t ≥ 0} is a Poisson process with rate λ if and only if
X1 , X2 , . . . are i.i.d. Exp(λ) random variables
• In particular, if Tn = X1 + · · · Xn then Tn has a Gamma
distribution with parameters n and λ
MATH136/STAT219 Lecture 25, November 21, 2008 – p. 9/12
Poisson Process: Equivalent Definition 3
A counting process {N (t), t ≥ 0} is a Poisson process with rate
λ if and only if
• {N (t), t ≥ 0} is continuous in probability
• For any positive integer k , 0 < t1 < · · · < tk , and
nonnegative integers n1 , n2 , . . . nk , and h > 0, both
IP (N (tk + h) − N (tk ) = 1|N (tj ) = nj , j ≤ k) = λh + o(h), and
IP (N (tk + h) − N (tk ) ≥ 2|N (tj ) = nj , j ≤ k) = o(h)
Notation: o(h) denotes a function g(h) which satisfies
g(h)
→ 0 as h ↓ 0
h
MATH136/STAT219 Lecture 25, November 21, 2008 – p. 10/12
Markov Property of Poisson Process
• Let {N (t), t ≥ 0} be a Poisson process with rate λ
• Then {N (t), t ≥ 0} is a homogenous Markov process with
• State space: S = {0, 1, 2, . . .}
• Initial distribution: π({0}) = 1
• Stationary transition probability function:
e−λt (λt)k
pt (n + k|n) = , t ≥ 0, n, k = 0, 1, 2, . . .
k!
• A Poisson process is a strong Markov process
MATH136/STAT219 Lecture 25, November 21, 2008 – p. 11/12
Compensated Poisson Process
• Let {N (t), t ≥ 0} be a Poisson process with rate λ
• Let {Ft } be the canonical filtration of N
• Define M (t) = N (t) − λt
• The process {M (t), t ≥ 0} is called a compensated Poisson
process
• {M (t), Ft } is a martingale
• {M 2 (t) − λt, Ft } is a martingale
MATH136/STAT219 Lecture 25, November 21, 2008 – p. 12/12
