ISyE 3232C

YL. Chang

Stochastic Manufacturing and Service Systems

Fall 2015

Exponential Distribution and Poisson Process

Exponential Distribution

Definition: A random variable X is said to be exponentially distributed with rate (with mean 1/) if it
has CDF of
(
1 ex x 0
F (x) =
0
otherwise
and pdf of
(
ex
f (x) =
0

x0
otherwise

Since exponential r.v. is continuous and nonnegative, it is often used to describe random time.

1.1

What is ?

represents rate as an inverse value of the time in such cases. For example, in post office, you expect one
customer arrives in 3 minutes in average, and the time X follows exponential distribution, we say
(1) X follows an exponential distribution with mean 3 minutes, or
(2) X follows exponential distribution with rate 1/3 per minute
and we denote it as X exp(1/3).

1.2

Memoryless Property

Definition A random variable X is memoryless, if

P (X > s + t|X > t) = P (X > s)
for all s, t 0.
It means regardless of how long you have already waited, the distribution of the time you still have to wait
until the next ambulance comes is exact same.
Exercise: Show that exponential random variable is memoryless.
P (X > s + t|X > t) =

P (X > s + t, X > t)
P (X > s + t)
e(s+t)
=
=
= es = P (X > s).
P (X > t)
P (X > t)
t

Remark:
For any positive random variable S that is independent of X, memoryless property still holds:
P (X > t + S|X > S) = P (X > t).
If we are saying, if the light is on at noon today, its remaining lifetime distribution is as if it is new, we
are referring to s which is fixed. If we are saying, if the light is on when IBM stock price hits 100 dollars,
its remaining lifetime is as if it is new, we are referring to S which is random. The important thing to
note is that S should be independent of X. The memoryless property does not hold if X and S are not
independent.

1.3

Properties:

E[X] = 1/
V ar[X] = 1/2
CV [X] = 1
Memoryless property: P (X > s + t|X > t) = P (X > s) for all s, t > 0.
Interpretation: An exponential random variable forgets its age.
If X1 exp(1 ), X2 exp(2 ) and they are independent, then
(1) P (X1 < X2 ) =

1
1 +2 .

(2) P (X1 > X2 ) =

2
1 +2 .

(3) P (X1 = X2 ) = 0.
(4) min(X1 , X2 ) exp(1 + 2 ).
(5) X1 + X2 = min(X1 , X2 ) + max(X1 , X2 ), hence,
E[X1 + X2 ] = E[min(X1 , X2 )] + E[max(X1 , X2 )],
1
.
E[max(X1 , X2 )] = 11 + 12 1 +
2

1.4

Exercises:

1. You are waiting for a bus, and you have been waiting for 3 minutes, what is the probability that next
bus will not come in 5 minutes from now?
P (X > 3 + 5|X > 3) = P (X > 5).
2. Smith and Jones came to post office together and they are served by two clerks, A and B, respectively.
Severer A has service time following exp(1/3) and server B has service time following exp(1/5). What
is the probability that Smith will leave the post office first? What is the distribution of the first person
leaves the post office?
1/3
P (Smith leave first) = P (X1 < X2 ) = 1/3+1/5
= 5/8.
The first person leaves the post office = min(X1 , X2 ) exp(1/3 + 1/5)

3. Let X1 and X2 follow exponential distribution with means 2 hours and 6 hours, respectively. What is
the mean of min(X1 , X2 )? What is the expectation of max(X1 , X2 )?
1 = 1/2, 2 = 1/6

Y = min(X1 , X2 ) exp(1 + 2 )
E(Y ) =

1
1
=
= 6/4 = 1.5
1 + 2
1/2 + 1/6

E[max(X1 , X2 )] = E[X1 + X2 ] E[min(X1 , X2 )] = 1/1 + 1/2 1/(1 + 2 ) = 2 + 6 1.5 = 6.5

4. A bank has two tellers, John and Mary. Johns processing times are iid exponential distributions
X1 with mean 6 minutes. Marys processing times are iid exponential distributions X2 with mean 4
minutes. A car with three customers A, B, C shows up at 12:00 noon and two tellers are both free at
that time.
(1) What is the expected time the car leave the bank?
Suppose A, B first start to get service and A goes to server of John and B goes to server of
Mary. Once one server completes, C will occupy either A or Bs position depending on which
server finishes first. If C occupies As server after A is completed, B has already been in service
while A was getting served. Let Y1 , Y2 denote the remaining processing time for John and Mary
respectively. The expected time when the car leaves is
E[W ] = E[min(X1 , X2 )] + E[max(Y1 , Y 2)]
= E[min(X1 , X2 )] + E[max(X1 , X2 )]
because of the memoryless property of exponential distribution. Even if B has gone through service
for a while, due to MEMORYLESS PROPERTY of exponential distribution, Bs service time is as
if B just started the service. Thus,
E[W ] = E[min(X1 , X2 ) + max(X1 , X2 )] = E[X1 + X2 ] = 10.
(2) What is the probability that C finishes service before A?
1/4
)2 = (6/10)2
P(C before A) = P(C before A|B before A)P(B before A) = ( 1/4+1/6
This is because we can think of A just starting to get service when C started service.
1/6
Similarly, P(C before B) = P(C before B|A before B)P(A before B) = ( 1/4+1/6
)2 = (4/10)2

(3) What is the probability that C finishes last?

6 2
4 2
) ( 10
) =
P(C finishes last) = 1 - P(C is not last) = 1-P(C before A) - P(C before B) = 1 ( 10
12/25

Math Review: Poisson Distribution

Definition: A discrete random variable X is said to follow Poisson distribution with parameter and write
X poi(), if it has pmf of
k e
P (X = k) =
for k = 0, 1, 2, ...
k!
It is the probability of a given number of events occurring in a fixed interval of time and/or space, if these
events occur with a known average rate and independently of the time since the last event.
Examples:
(1) the number of phone calls received by a call center per hour,
(2) the number of decay events per second from a radioactive source,
(3) or the number of taxis passing a particular street corner per hour
Properties:
E[X] = Var(X) = .
If X1 poi(1 ) and X2 poi(2 ), then X1 + X2 poi(1 + 2 ).

Poisson Process

Definition of a counting process: N = {N (t), t 0} is said to be a counting process if N (0) = 0, and

the integer-valued function N (t) is non-decreasing.
Definition of a Homogeneous Poisson Process: A counting process N = {N (t), t 0} is said to be a
Poisson process with constant rate and write P P () if
(1) (Stationary increments)
N (t2 ) N (t1 ) poi((t2 t1 )) for any 0 t1 t2 , i.e.,
P (N (t2 ) N (t1 ) = k) =

e(t2 t1 ) [(t2 t1 )]k

k!

for k = 0, 1, 2, ....

(2) (Independent increments)

For t1 t2 t3 t4 , N (t2 ) N (t1 ) is independent of N (t4 ) N (t3 )
(3) N(0)=0
Interpretation:
(1) (Stationary increments)Amount of increment follows Poisson distribution; And the increment counted
in any time interval only depends on the length of the interval
(2) (Independent increments)Increase in non-overlapping intervals are independent .
Remark: A sequence of customers having iid exponential inter-arrival times is specially called a Poisson
process.

Example 1: Assume N is a Poisson process with rate = 2/minute.

(1) Find the probability that there are exactly 4 arrivals in first 3 minutes.
P (N (3) N (0) = 4) = P (poi((3 0)) = 4) = P (poi(6) = 4) =

e23 [2 3]4
4!

(2) What is the probability that exactly two arrivals in [0, 2] and at least 3 arrivals in [1, 3]?
Trick: decompose the intervals into non-overlapping ones.
P (N (2) N (0) = 2, N (3) N (1) 3)
= P (N (1) N (0) = 0, N (2) N (1) = 2, N (3) N (2) 1)
+ P (N (1) N (0) = 1, N (2) N (1) = 1, N (3) N (2) 2)
+ P (N (1) N (0) = 2, N (2) N (1) = 0, N (3) N (2) 3)
= P (N (1) N (0) = 0)P (N (2) N (1) = 2)P (N (3) N (2) 1)
+ P (N (1) N (0) = 1)P (N (2) N (1) = 1)P (N (3) N (2) 2)
+ P (N (1) N (0) = 2)P (N (2) N (1) = 0)P (N (3) N (2) 3)
2 2

P (N (1) N (0) = 0) = e2 , P (N (2) N (1) = 2) = e 2!2 ,

P (N (3) N (2) 1) = 1 P (N (3) N (2) = 0) = 1 e2 .
The other probabilities are on your own.
(3) What is the probability that there is no arrival in [0, 4]?
P (N (4) N (0) = 0) = P (poi(2 4) = 0) =

e24 [2 4]0
= e8 .
0!

(4) The probability that the first arrival takes at least 4 minutes?
P (T1 4) = P (T1 > 4) = P (N (4) = 0) = e8
The first arrival takes at least 4 minutes is equivalent to there is no arrival for the first 4 minutes
(duality).
(5) The probability that the first arrival takes at least t minutes?
P (T1 > t) = P (N (t) = 0) = e2t = et
This is exponential distribution with arrival rate .

(6) What is the distribution of T1 (Time to the first arrival)?

P (T1 t) = 1 P (T1 > t) = 1 P (N (t) N (0) = 0) = 1 P (poi(2t) = 0) = 1

(2t)0 e2t
= 1 e2t .
0!

T1 exp(2).
(7) Prove that the time to the first arrival in P P () follows exp().
Let T1 be the time to the first arrival.
P (T1 t) = 1 P (T1 > t) = 1 P (N (t) N (0) = 0) = 1 P (poi(t) = 0) = 1

et (t)0
= 1 et
0!

T1 exp().

Merging Poisson Process

Theorem 1: Assume NA is a Poisson process with rate A . Assume NB is a Poisson process with rate B .
Assume further that NA and NB are independent. Then, N = {N (t), t 0} is a Poisson process with rate
= A + B where N (t) = NA (t) + NB (t).

Example 2: Consider the north confluence area of I75 and I85. Suppose, for South bound, NI75S is
P P (100/min) and NI85S is P P (200/min). Then, at the north confluence point, we have NI75S + NI85S ,
which is PP(100+200/min) traffic heading South.

Thinning Poisson Process

Theorem 2: Suppose N is a Poisson process with rate . Assume each arrival will choose subset A or B,
with probability p and q = 1 p, respectively. Then NA = {NA (t), t 0} is a Poisson process with rate p
and NB = {NB (t), t 0} is a Poisson process with rate q.

Example 3: Consider the north confluence area of I75 and I85. Suppose, for North bound, we have
PP(400/min) heading North. Each car heading north chooses I75N with probability 0.4 and I85N with
probability 0.6. Then we have NI75N , which is P P (400 0.4/min) and NI85N , which is P P (400 0.6/min).

Non-homogeneous Poisson Process

Criticism: Arrival process is not time homogeneous for systems such as:
Call center traffic in the morning vs afternoon;
Aircraft arrival rate in ATL airport during Thanksgiving weekend vs today
Remedy: Making rate to be dependent on time!
This is called Non-homogeneous Poisson Process.
Key function: (t) as a function for arrival rate that changes over time.
We can still use very similar mathematical technique of homogeneous Poisson process.
Definition of Non-homogeneous Poisson process:
A counting process N = {N (t), t 0} is said to be a Non-homogeneous Poisson process with rate function
(t), t 0, and write N HP P ((t)) if
6

(1) N(0) = 0;
(2) N(t) has independent increment
R t+s
(3) N (t + s) N (s) P oisson( s (u)du)
Remark: PP is just a special case of NHPP where rate function of NHPP is a constant.
Example 4: Assume that call arrival to a call center follows a NHPP. The call center opens from 9am to
5pm. During the first hour, the arrival rate increases linearly from 0 at 9am to 60 calls per hour at 10am.
After 10am, the arrival rate is constant at 60 calls per hour.

(a) Plot the arrival rate function (t) as a function of time t; indicate clearly the time unit used.

(b) Find the probability that exactly 5 calls have arrived by 9:10am.
=

1/6

1/6

(t)dt =
0

60tdt = 5/6
0

P (N (1/6) N (0) = 5) = P (poi(5/6) = 5) =

e5/6 (5/6)5
.
5!

(c) Find the probability that exactly k calls will arrive between 9:50am to 10:10am.
=

7/6

(t)dt =
5/6

60tdt +
5/6

7/6

60dt = 115/6
1

115/6
(115/6)k
= k) = e
P (N (7/6) N (5/6) = k) = P (poi()
.
k!

Review
Exponential Distribution and its properties
Memoryless property
Poisson process vs. Poisson distribution
Homogeneous Poisson process two properties: stationary and independent
Merging and Thinning Poisson Processes
Non-Homogeneous Poisson process: independent