Computer Simulation
Computer Simulation
Lecture 8: Generating
random variables
Marko Boon
http://www.win.tue.nl/courses/2WB05
January 7, 2013
Outline
1. How do we generate random variables?
2. Fitting distributions
2/36
3/36
Continuous distributions
4/36
Continuous distributions
5/36
Examples
X = a + (b a)U is uniform on (a, b);
X = ln(U )/ is exponential with parameter ;
X = ( ln(U ))1/a / is Weibull, parameters a and .
Unfortunately, for many distribution functions we do not have an easy-to-use (closed-form) expression for the
inverse of F.
Continuous distributions
6/36
Composition method
This method applies when the distribution function F can be expressed as a mixture of other distribution functions F1 , F2 , . . .,
X
F(x) =
pi Fi (x),
i=1
where
pi 0,
pi = 1
i=1
i = 1, 2, . . .
Continuous distributions
7/36
Examples
Hyper-exponential distribution:
F(x) = p1 F1 (x) + p2 F2 (x) + + pk Fk (x),
x 0,
1 x
2e ,
1 x
2e ,
x < 0;
x 0,
Continuous distributions
Convolution method
In some case X can be expressed as a sum of independent random variables Y1 , . . . , Yn , so
X = Y1 + Y2 + + Yn .
where the Yi s can be generated more easily than X .
Algorithm:
Generate independent Y1 , . . . , Yn , each with distribution function G;
Return X = Y1 + + Yn .
8/36
Continuous distributions
9/36
Example
If X is Erlang distributed with parameters n and , then X can be expressed as a sum of n independent
exponentials Yi , each with mean 1/.
Algorithm:
Generate n exponentials Y1 , . . . , Yn , each with
mean ;
Set X = Y1 + + Yn .
More efficient algorithm:
Generate n uniform (0, 1) random variables
U1 , . . . , Un ;
Set X = ln(U1U2 Un )/.
Continuous distributions
10/36
Acceptance-Rejection method
Denote the density of X by f . This method requires a function g that majorizes f ,
g(x) f (x)
for all x. Now g will not be a density, since
Z
g(x)d x 1.
c=
Continuous distributions
11/36
P(Y x, Y accepted) =
f (y)
1
h(y)dy =
g(y)
c
f (y)dy,
f (y)dy.
Continuous distributions
12/36
Continuous distributions
13/36
Example
The Beta(4,3) distribution has density
f (x) = 60x 3 (1 x)2 ,
0 x 1.
The maximal value of f occurs at x = 0.6, where f (0.6) = 2.0736. Thus, if we define
g(x) = 2.0736,
0 x 1,
Continuous distributions
Normal distribution
Methods:
Acceptance-Rejection method
Box-Muller method
14/36
Continuous distributions
15/36
Acceptance-Rejection method
If X is N (0, 1), then the density of |X | is given by
2
2
f (x) = ex /2 ,
2
x > 0.
2e/ex
3. If U e(Y 1)
Continuous distributions
Box-Muller method
If U1 and U2 are independent U (0, 1) random variables, then
p
X 1 = 2 ln U1 cos(2U2 )
p
X 2 = 2 ln U1 sin(2U2 )
are independent standard normal random variables.
This method is implemented in the function nextGaussian() in java.util.Random
16/36
Discrete distributions
17/36
i = 0, 1, . . . ,
pi = 1.
i=0
pj U <
i
X
j=0
pj.
Discrete distributions
18/36
pj U <
j=0
I
X
pj
j=0
and return X = x I .
The second step requires a search; for example, starting with I = 0 we keep adding 1 to I until we have found
the (smallest) I such that
I
X
U<
pj
j=0
Note: The algorithm needs exactly one uniform random variable U to generate X ; this is a nice feature if you
use variance reduction techniques.
Discrete distributions
Array method: when X has a finite support
Suppose pi = ki /100, i = 1, . . . , m,
where ki s are integers with 0 ki 100
Construct array A[i], i = 1, . . . , 100 as follows:
set A[i] = x1 for i = 1, . . . , k1
set A[i] = x2 for i = k1 + 1, . . . , k1 + k2 , etc.
Then, first sample a random index I from 1, . . . , 100:
I = 1 + b100U c and set X = A[I ]
19/36
Discrete distributions
Bernoulli
Two possible outcomes of X (success or failure):
P(X = 1) = 1 P(X = 0) = p.
Algorithm:
Generate U from U (0, 1);
If U p, then X = 1; else X = 0.
20/36
Discrete distributions
21/36
Discrete uniform
The possible outcomes of X are m, m + 1, . . . , n and they are all equally likely, so
P(X = i) =
1
,
nm+1
i = m, m + 1, . . . , n.
Algorithm:
Generate U from U (0, 1);
Set X = m + b(n m + 1)U c.
Note: No search is required, and compute (n m + 1) ahead.
Discrete distributions
22/36
Geometric
A random variable X has a geometric distribution with parameter p if
P(X = i) = p(1 p)i ,
i = 0, 1, 2, . . . ;
X is the number of failures till the first success in a sequence of Bernoulli trials with success probability p.
Algorithm:
Generate independent Bernoulli( p) random variables Y1 , Y2 , . . .; let I be the index of the first successful
one, so Y I = 1;
Set X = I 1.
Alternative algorithm:
Generate U from U (0, 1);
Set X = bln(U )/ ln(1 p)c.
Discrete distributions
Binomial
A random variable X has a binomial distribution with parameters n and p if
n i
p (1 p)ni ,
i = 0, 1, . . . , n;
P(X = i) =
i
X is the number of successes in n independent Bernoulli trials, each with success probability p.
Algorithm:
Generate n Bernoulli( p) random variables
Y1 , . . . , Yn ;
Set X = Y1 + Y2 + + Yn .
Alternative algorithms can be derived by using the following results.
23/36
Discrete distributions
24/36
Let Y1 , Y2 , . . . be independent geometric( p) random variables, and I the smallest index such that
I +1
X
(Yi + 1) > n.
i=1
Yi
> ln(1 p).
ni +1
Discrete distributions
25/36
Negative Binomial
A random variable X has a negative-binomial distribution with parameters n and p if
n+i 1 n
p (1 p)i ,
i = 0, 1, 2, . . . ;
P(X = i) =
i
X is the number of failures before the n-th success in a sequence of independent Bernoulli trials with success
probability p.
Algorithm:
Generate n geometric( p) random variables
Y1 , . . . , Yn ;
Set X = Y1 + Y2 + + Yn .
Discrete distributions
26/36
Poisson
A random variable X has a Poisson distribution with parameter if
i
P(X = i) = e ,
i!
i = 0, 1, 2, . . . ;
X is the number of events in a time interval of length 1 if the inter-event times are independent and exponentially distributed with parameter .
Algorithm:
Generate exponential inter-event times Y1 , Y2 , . . . with mean 1; let I be the smallest index such that
I +1
X
i=1
Set X = I .
Yi > ;
Discrete distributions
27/36
Poisson (alternative)
Generate U(0,1) random variables U1 , U2 , . . .;
let I be the smallest index such that
IY
+1
i=1
Set X = I .
Ui < e ;
Fitting distributions
28/36
Input of a simulation
Specifying distributions of random variables (e.g., interarrival times, processing times) and assigning parameter values can be based on:
Historical numerical data
Expert opinion
In practice, there is sometimes real data available, but often the only information of random variables that is
available is their mean and standard deviation.
Fitting distributions
Empirical data can be used to:
construct empirical distribution functions and generate samples from them during the simulation;
fit theoretical distributions and then generate samples from the fitted distributions.
29/36
Fitting distributions
30/36
Moment-fitting
Obtain an approximating distribution by fitting a phase-type distribution on the mean, E(X ), and the coefficient
of variation,
X
cX =
,
E(X )
of a given positive random variable X , by using the following simple approach.
Moment-fitting
31/36
k(1
+
c
,
p=
=
X
X
X
E(X )
1 + c2X
the E k1,k distribution matches E(X ) and c X .
Moment-fitting
32/36
pk
k
k
Moment-fitting
33/36
But the H2 distribution is not uniquely determined by its first two moments. In applications, the H2 distribution
with balanced means is often used. This means that the normalization
p1
p2
=
1
2
is used. The parameters of the H2 distribution with balanced means and fitting E(X ) and c X ( 1) are given by
s
!
2 1
c
1
X
1+
,
p2 = 1 p1 ,
p1 =
2
2
cX + 1
1 =
2 p1
,
E(X )
2 =
2 p2
.
E(X )
Moment-fitting
34/36
In case c2X 0.5 one can also use a Coxian-2 distribution for a two-moment fit.
Phase diagram for the Coxian-k distribution:
1
1
2
p1
1 p1
k
p2
pk 1
1 p2
1 pk 1
p1 = 0.5/c2X ,
2 = 1 p1 .
Moment-fitting
35/36
Fitting distributions
Adequacy of fit
Graphical comparison of fitted and empirical curves;
Statistical tests (goodness-of-fit tests).
36/36