ISYE8012 - System Simulation and

Modeling
Week 3 - Session 3
Statistical Models in Simulation
Chapter 5
Statistical Models in
Simulation

Banks, Carson, Nelson & Nicol

Discrete-Event System Simulation
Purpose & Overview
 The world the model-builder sees is probabilistic rather
than deterministic.
 Some statistical model might well describe the variations.

 An appropriate model can be developed by sampling the

phenomenon of interest:
 Select a known distribution through educated guesses
 Make estimate of the parameter(s)
 Test for goodness of fit

 In this chapter:
 Review several important probability distributions
 Present some typical application of these models

3
Review of Terminology and Concepts
 In this section, we will review the following
concepts:
 Discreterandom variables
 Continuous random variables
 Cumulative distribution function
 Expectation

4
Discrete Random Variables [Probability Review]

 X is a discrete random variable if the number of possible

values of X is finite, or countably infinite.
 Example: Consider jobs arriving at a job shop.
 Let X be the number of jobs arriving each week at a job shop.
 Rx = possible values of X (range space of X) = {0,1,2,…}
 p(xi) = probability the random variable is xi = P(X = xi)
 p(xi), i = 1,2, … must satisfy:
1. p ( xi )  0, for all i



2. i 1
p( xi )  1

 The collection of pairs [xi, p(xi)], i = 1,2,…, is called the probability

distribution of X, and p(xi) is called the probability mass function
(pmf) of X.

5
Continuous Random Variables [Probability Review]

 X is a continuous random variable if its range space Rx is an interval

or a collection of intervals.
 The probability that X lies in the interval [a,b] is given by:
b
P(a  X  b)   f ( x)dx
a

 f(x), denoted as the probability density function (pdf) of X, satisfies:

1. f ( x)  0 , for all x in R X
2.  f ( x)dx  1
RX

3. f ( x)  0, if x is not in R X

 Properties x0
1. P( X  x0 )  0, because x0
f ( x) dx  0
2. P(a  X  b)  P (a  X  b)  P(a  X  b)  P (a  X  b)
6
Continuous Random Variables [Probability Review]

 Example: Life of an inspection device is given by X, a

continuous random variable with pdf:

 1  x / 2
e , x0
f ( x)   2
0, otherwise

 X has an exponential distribution with mean 2 years

 Probability that the device’s life is between 2 and 3 years is:
1 3 x/ 2
P (2  x  3)   e dx  0.14
2 2

7
Cumulative Distribution Function [Probability Review]

 Cumulative Distribution Function (cdf) is denoted by F(x),

where F(x) = P(X ≤ x)
 If X is discrete, then F ( x)   p ( xi )
all
xi  x
 If X is continuous, then x
F ( x )   f (t )dt


 Properties
1. F is nondecreasing function. If a  b, then F (a)  F (b)
2. lim x  F ( x)  1
3. lim x  F ( x)  0

 All probability questions about X can be answered in terms of the

cdf, e.g.:
P (a  X  b)  F (b)  F (a ), for all a  b

8
Cumulative Distribution Function [Probability Review]

 Example: An inspection device has cdf:

1 x t / 2
F ( x)   e dt  1  e  x / 2
2 0

 The probability that the device lasts for less than 2 years:
P(0  X  2)  F (2)  F (0)  F (2)  1  e 1  0.632

 The probability that it lasts between 2 and 3 years:

P(2  X  3)  F (3)  F (2)  (1  e  ( 3 / 2) )  (1  e 1 )  0.145

9
Expectation [Probability Review]

 The expected value of X is denoted by E(X)

 If X is discrete E ( x)   xi p ( xi )
all i

 If X is continuous E ( x)   xf ( x)dx


 a.k.a the mean, , or the 1st moment of X

 A measure of the central tendency
 The variance of X is denoted by V(X) or var(X) or 2
 Definition: V(X) = E[(X – E[X]2]
 Also, V(X) = E(X2) – [E(X)]2
 A measure of the spread or variation of the possible values of X around
the mean
 The standard deviation of X is denoted by 
 Definition: Square root of V(X)
 Expressed in the same units as the mean

10
Expectations [Probability Review]

 Example: The mean of life of the previous inspection device

is:
1  x/2  x / 2
 
E ( X )   xe dx   xe   e  x / 2 dx  2
2 0 0 0

 To compute variance of X, we first compute E(X2):


1  2 x/2 2  x / 2 
E ( X )   x e dx   x e
2
  e  x / 2 dx  8
2 0 0
0

 Hence, the variance and standard deviation of the device’s life

are: 2
V (X )  8  2  4
  V (X )  2
11
Useful Statistical Models
 In this section, statistical models appropriate to
some application areas are presented. The
areas include:
 Queueing systems
 Inventory and supply-chain systems
 Reliability and maintainability
 Limited data

12
Queueing Systems [Useful Models]

 In a queueing system, interarrival and service-time

patterns can be probablistic (for more queueing examples, see
Chapter 2).
 Sample statistical models for interarrival or service time
distribution:
 Exponential distribution: if service times are completely random
 Normal distribution: fairly constant but with some random
variability (either positive or negative)
 Truncated normal distribution: similar to normal distribution but
with restricted value.
 Gamma and Weibull distribution: more general than exponential
(involving location of the modes of pdf’s and the shapes of tails.)

13
Inventory and supply chain [Useful Models]

 In realistic inventory and supply-chain systems, there are

at least three random variables:
 The number of units demanded per order or per time period
 The time between demands
 The lead time
 Sample statistical models for lead time distribution:
 Gamma
 Sample statistical models for demand distribution:
 Poisson: simple and extensively tabulated.
 Negative binomial distribution: longer tail than Poisson (more
large demands).
 Geometric: special case of negative binomial given at least one
demand has occurred.

14
Reliability and maintainability [Useful Models]

 Time to failure (TTF)

 Exponential: failures are random
 Gamma: for standby redundancy where each
component has an exponential TTF
 Weibull: failure is due to the most serious of a large
number of defects in a system of components
 Normal: failures are due to wear

15
Other areas [Useful Models]

 For cases with limited data, some useful

distributions are:
 Uniform, triangular and beta
 Other distribution: Bernoulli, binomial and
hyperexponential.

16
Discrete Distributions
 Discrete random variables are used to describe
random phenomena in which only integer values
can occur.
 In this section, we will learn about:
 Bernoullitrials and Bernoulli distribution
 Binomial distribution
 Geometric and negative binomial distribution
 Poisson distribution

17
Bernoulli Trials and Bernoulli Distribution
[Discrete Dist’n]
 Bernoulli Trials:
 Consider an experiment consisting of n trials, each can be a
success or a failure.
 Let Xj = 1 if the jth experiment is a success
 and Xj = 0 if the jth experiment is a failure
 The Bernoulli distribution (one trial):
 p, x j  1, j  1,2,..., n

p j ( x j )  p( x j )  1  p  q, x j  0,j  1,2,...,n
0, otherwise

 where E(Xj) = p and V(Xj) = p (1 – p) = p q
 Bernoulli process:
 The n Bernoulli trials where trails are independent:
p(x1,x2,…, xn) = p1(x1) p2(x2) … pn(xn)
18
Binomial Distribution [Discrete Dist’n]

 The number of successes in n Bernoulli trials, X, has a

binomial distribution.
 n  x n  x
  p q , x  0,1,2,..., n
p( x)   x 
0, otherwise

The number of
outcomes having Probability that
the required there are
number of x successes and
successes and (n-x) failures
failures

 The mean, E(x) = p + p + … + p = n*p

 The variance, V(X) = pq + pq + … + pq = n*pq

19
Geometric & Negative
Binomial Distribution [Discrete Dist’n]

 Geometric distribution
 The number of Bernoulli trials, X, to achieve the 1st success:
q x 1 p, x  0,1, 2,..., n
p ( x)  
0, otherwise
 E(X) = 1/p, and V(X) = q/p2

 Negative binomial distribution

 The number of Bernoulli trials, X, until the kth success
 If X is a negative binomial distribution with parameters p and k,
then:  x  1 x k k
  q p , x  k , k  1, k  2,...
p ( x)   k  1
0, otherwise

 E(X) = k/p, and V(X) = kq/p2

20
Poisson Distribution [Discrete Dist’n]

 Poisson distribution describes many random processes

quite well and is mathematically quite simple.
 where  > 0, pdf and cdf are:
 e   x x
e   i

p ( x)   x! , x  0,1,... F ( x)  
i 0 i!
0, otherwise

 E(X) =  = V(X)

21
Poisson Distribution [Discrete Dist’n]

 Example: A computer repair person is “beeped” each

time there is a call for service. The number of beeps per
hour ~ Poisson( = 2 per hour).

 The probability of three beeps in the next hour:

p(3) = e–223/3! = 0.18
also, p(3) = F(3) – F(2) = 0.857 – 0.677 = 0.18

 The probability of two or more beeps in a 1-hour period:

p(2 or more) = 1 – p(0) – p(1)
= 1 – F(1)
= 0.594

22
Continuous Distributions
 Continuous random variables can be used to
describe random phenomena in which the
variable can take on any value in some interval.
 In this section, the distributions studied are:
 Uniform
 Exponential
 Normal
 Weibull
 Lognormal

23
Uniform Distribution [Continuous Dist’n]

 A random variable X is uniformly distributed on the

interval (a,b), denoted U(a,b), if its pdf and cdf are:
0, xa
 1 xa
 , a xb 
f ( x)   b  a F ( x)   , a xb
 b  a
0, otherwise
1, xb

 Properties
 P(x1 < X < x2) is proportional to the length of the interval:
F(x2) – F(x1) = (x2 – x1)/(b – a)
 E(X) = (a+b)/2 V(X) = (b – a)2/12
 U(0,1) provides the means to generate random numbers,
from which random variates can be generated.
24
Exponential Distribution [Continuous Dist’n]

 A random variable X is exponentially distributed with

parameter  > 0 if its pdf and cdf are:
e  x , x  0 0, x0
f ( x)   F ( x)   x  t
 0  e dt  1  e , x  0
x
0, elsewhere

 E(X) = 1/V(X) = 1/2

 Used to model interarrival times
when arrivals are completely
random, and to model service
times that are highly variable
 For several different exponential
pdf’s (see figure), the value of
intercept on the vertical axis is ,
and all pdf’s eventually intersect.
25
Exponential Distribution [Continuous Dist’n]

 Memoryless property
 For all s and t greater or equal to 0:
P(X > s + t | X > s) = P(X > t)

 Example: A lamp ~ exp( = 1/3 per hour), hence, on

average, 1 failure per 3 hours.
 The probability that the lamp lasts longer than its mean life is:
P(X > 3) = 1-(1-e-3/3) = e-1 = 0.368
 The probability that the lamp lasts between 2 to 3 hours is:
P(2 <= X <= 3) = F(3) – F(2) = 0.145
 The probability that it lasts for another hour given it is
operating for 2.5 hours:
P(X > 3.5 | X > 2.5) = P(X > 1) = e-1/3 = 0.717

26
Normal Distribution [Continuous Dist’n]

 A random variable X is normally distributed if it has the

pdf:  2

1 1 x 
f ( x)  exp     ,   x  
 2 2
    

    
 Mean: 2
 Variance:  0
 Denoted as X ~ N(,2)
 Special properties:
 lim x  f ( x)  0, and lim x f ( x)  0.
 f( – x) = f(( + x); the pdf is symmetric about .
 The maximum value of the pdf occurs at x = ; the mean and
mode are equal.

27
Normal Distribution [Continuous Dist’n]

 Evaluating the distribution:

 Use numerical methods (no closed form)
 Independent of  and  using the standard normal distribution:
Z ~ N(0,1)
 Transformation of variables: let Z = (X – ) / ,
F ( x)  P  X  x 
 x 
 PZ  
  
( x ) / 1  z2 / 2
 e dz

2
( x ) /
  ( z )dz


1 t 2 / 2
 
z
 x
 where  ( z )   e dt

2 28
Normal Distribution [Continuous Dist’n]

 Example: The time required to load an oceangoing

vessel, X, is distributed as N(12,4)
 The probability that the vessel is loaded in less than 10 hours:
 10  12 
F (10)      (1)  0.1587
 2 
 Using the symmetry property, (1) is the complement of  (-1)

29
Weibull Distribution [Continuous Dist’n]

 A random variable X has a Weibull distribution if its pdf has the form:
   x    1   x    

f ( x)     
exp    , x  
    
0, otherwise

 3 parameters:
 Location parameter:  (    )
 Scale parameter: 
 Shape parameter. 
 Example:  = 0 and = 1:

When  = 1,
X ~ exp( = 1/)

30
Lognormal Distribution [Continuous Dist’n]

 A random variable X has a lognormal distribution if its pdf

has the form:
 1   ln x  μ  2 
 exp   , x  0 =1,
2
f ( x)   2πσx  2 σ  2=0.5,1,2.

0, otherwise
2
 Mean E(X) = e+ /2
2 2
 Variance V(X) = e+ (e - 1)

 Relationship with normal distribution

 When Y ~ N(, 2), then X = eY ~ lognormal(, 2)
 Parameters  and 2 are not the mean and variance of the
lognormal
31
Poisson Distribution
 Definition: N(t) is a counting function that represents
the number of events occurred in [0,t].
 A counting process {N(t), t ≥ 0} is a Poisson process
with mean rate  if:
 Arrivals occur one at a time
 {N(t), t ≥ 0} has stationary increments
 {N(t), t ≥ 0} has independent increments
 Properties
e  t ( t ) n
P[ N (t )  n]  , for t  0 and n  0,1,2,...
n!
 Equal mean and variance: E[N(t)] = V[N(t)] = t
 Stationary increment: The number of arrivals between time s
and time t > s is also Poisson-distributed with mean (t – s)

32
Interarrival Times [Poisson Dist’n]

 Consider the interarrival times of a Possion process (A1, A2, …),

where Ai is the elapsed time between arrival i and arrival i+1

 The 1st arrival occurs after time t iff there are no arrivals in the interval
[0,t], hence:
P{A1 > t} = P{N(t) = 0} = e-t
P{A1 ≤ t} = 1 – e-t [cdf of exp()]
 Interarrival times, A1, A2, …, are exponentially distributed and
independent with mean 1/

Arrival counts Interarrival time

~ Poisson() ~ Exp(1/)
Stationary & Memoryless
Independent
33
Splitting and Pooling [Poisson Dist’n]

 Splitting:
 Suppose each event of a Poisson process can be classified as
Type I, with probability p and Type II, with probability 1-p.
 N(t) = N1(t) + N2(t), where N1(t) and N2(t) are both Poisson
processes with rates  p and  (1-p)
 

p N1(t) ~ Poi[p]
N(t) ~ Poi() 

(1-p) N2(t) ~ Poi[(1-p)]

 Pooling:
 Suppose two Poisson processes are pooled together
 N1(t) + N2(t) = N(t), where N(t) is a Poisson processes with
rates 1 + 2N1(t) ~ Poi[ ] 1
 
N(t) ~ Poi(12)
N2(t) ~ Poi[] 2
34
Nonstationary Poisson
Process (NSPP) [Poisson Dist’n]

 Poisson Process without the stationary increments, characterized by

(t), the arrival rate at time t.
 The expected number of arrivals by time t, (t):
t
Λ(t)   λ(s)ds
0

 Relating stationary Poisson process n(t) with rate  and NSPP N(t)
with rate (t):
 Let arrival times of a stationary process with rate  = 1 be t1, t2, …,
and arrival times of a NSPP with rate (t) be T1, T2, …, we know:
ti = (Ti)
Ti = (ti)

35
Nonstationary Poisson
Process (NSPP) [Poisson Dist’n]
 Example: Suppose arrivals to a Post Office have rates 2 per minute
from 8 am until 12 pm, and then 0.5 per minute until 4 pm.
 Let t = 0 correspond to 8 am, NSPP N(t) has rate function:
2, 0t 4
 (t )  
0.5, 4  t  8
Expected number of arrivals by time t:
2t , 0t 4

 (t )   4 t t
 0 2 ds  4 0.5 ds 
2
 6, 4  t  8

 Hence, the probability distribution of the number of arrivals between

11 am and 2 pm.
P[N(6) – N(3) = k] = P[N((6)) – N((3)) = k]
= P[N(9) – N(6) = k]
= e(9-6)(9-6)k/k!
= e3(3)k/k!
36
Empirical Distributions [Poisson Dist’n]

 A distribution whose parameters are the observed values

in a sample of data.
 May be used when it is impossible or unnecessary to establish that
a random variable has any particular parametric distribution.
 Advantage: no assumption beyond the observed values in the
sample.
 Disadvantage: sample might not cover the entire range of possible
values.

37
Summary
 The world that the simulation analyst sees is probabilistic,
not deterministic.
 In this chapter:
 Reviewed several important probability distributions.
 Showed applications of the probability distributions in a simulation
context.
 Important task in simulation modeling is the collection and
analysis of input data, e.g., hypothesize a distributional
form for the input data. Reader should know:
 Difference between discrete, continuous, and empirical
distributions.
 Poisson process and its properties.

38