Simulation Modeling and

Analysis
Dr. Md. Nasim Adnan
 Random Variables and their Properties
• An experiment is a process whose outcome is not
known with certainty. The set of all possible outcomes
of an experiment is called the sample space and is
denoted by S.
• The outcomes themselves are called the sample points
in the sample space.
• If the experiment consists of flipping a coin, then S = {H,
T}
– where the symbol { } means the “set consisting of,” and “H”
and “T” mean that the outcome is a head and a tail,
respectively.
• If the experiment consists of tossing a die,
then S = {1, 2, . . . , 6}
– where the outcome i means that i appeared on
the die, i = 1, 2, . . . , 6.
– A random variable is a function (or rule) that
assigns a real number (any number greater than -
ꚙ and less than ꚙ) to each point in the sample
space S.
• In general, we denote random variables by
capital letters such as X, Y, Z and the values
that random variables take on by lowercase
letters such as x, y, z.
• The distribution function (sometimes called
the cumulative distribution function) F(x) of
the random variable X is defined for each real
number x as follows:
• F(x) = P(X <= x) for -ꚙ < x < ꚙ
• where P(X <= x) means the probability
associated with the event {X <= x}.
• Thus, F(x) is the probability that, when the
experiment is done, the random variable X
will take on a value no larger than the
number x.
• A random variable X is said to be discrete if it can
take on at most a countable number of values,
say, x1, x2, . . . An example of an uncountable set
is all real numbers between 0 and 1.)
• Thus, a random variable that takes on only a
finite number of values x1, x2, . . . , xn is discrete.
The probability that the discrete random variable
X takes on the value xi is given by:
p(xi) = P(X = xi) for i = 1, 2, . . .
• and we must have
• For the inventory example, the size of the
demand for the product is a discrete random
variable X that takes on the values 1, 2, 3, 4
with respective probabilities 1/6, 1/3, 1/3,
1/6.
• The probability mass function and the
distribution function for X are given in the
following figures.
• We now consider random variables that can
take on an uncountably infinite number of
different values (e.g., all nonnegative real
numbers).
• A random variable X is said to be continuous if
there exists a nonnegative function f(x) such
that for any set of real numbers B:
• Thus, the total area under f(x) is 1.
• Also, if X is a nonnegative random variable, as
is often the case in simulation applications,
the second range of integration is from 0 to
ꝏ.
• All probability statements about X can (in
principle) be computed from f(x), which is
called the probability density function for the
continuous random variable X.
 Random-Number Generators
• A simulation of any system or process in which there are
inherently random components (Single-Server Queueing
System) requires a method of generating or obtaining
numbers that are random.
• We now study some methods of generating random
variates from the uniform distribution on the interval [0,
1]; this distribution is denoted by U(0, 1).
• We agree that arithmetic generators, if designed carefully,
can produce numbers that appear to be independent
draws from the U(0, 1) distribution, in that they pass a
series of statistical tests.
• A “good” arithmetic random-number generator should
possess several properties:
– Above all, the numbers produced should appear to be
distributed uniformly on [0, 1] and should not exhibit any
correlation with each other; otherwise, the simulation’s
results may be completely invalid.
– From a practical standpoint, we would naturally like the
generator to be fast and avoid the need for a lot of storage.
– We would like to be able to reproduce a given stream of
random numbers exactly, for at least two reasons.
• First, this can sometimes make debugging or verification of the
computer program easier.
 • More importantly, we might want to use identical
random numbers in simulating different systems in order
to obtain a more precise comparison.
– There should be provision in the generator for easily
producing separate “streams” of random numbers.
We can think of the different streams as being
separate and independent generators.
• We would like the generator to be portable, i.e.,
to produce the same sequence of random
numbers for all standard compilers and
computers.
 LINEAR CONGRUENTIAL GENERATORS
• Many random-number generators in use today
are linear congruential generators (LCGs),
introduced by Lehmer (1951).
• A sequence of integers Z1, Z2, . . . is defined by the
recursive formula
Zi = (aZi-1+c)(mod m)
• where m (the modulus), a (the multiplier), c (the
increment), and Z0 (the seed) are nonnegative
integers.
• Therefore, 0 <= Zi <= m - 1, and to obtain the desired
random numbers Ui (for i = 1, 2, . . . ) in [0, 1], we let Ui =
Zi/m.
• We shall concentrate our attention for the most part on
the Zi’s, although the precise nature of the division of Zi
by m should be paid attention to due to differences in
the way various computers and compilers handle
floating-point arithmetic.
• In addition to non-negativity, the integers m, a, c, and Z0
should satisfy 0 < m, a < m, c < m, and Z0 < m.
• In fact, one can show by mathematical
induction that that every Zi is completely
determined by m, a, c, and Z0.
• However, by careful choice of these four
parameters we try to induce behavior in the Zi’s
that makes the corresponding Ui’s appear to be
independent and identically distributed IID U(0,
1) random variates when subjected to a variety
of tests.
• One objection to LCGs might be that the Ui’s can take on only the
rational values 0, 1/m, 2/m, . . . , (m-1)/m; in fact, the Ui’s might
actually take on (only a fraction of) these values, depending on
the specification of the constants m, a, c, and Z0, as well as on the
nature of the floating-point division by m.
• Thus there is no possibility of getting a value of Ui between, say,
0.1/m and 0.9/m, whereas this should occur with probability
0.8/m>0.
• As we shall see, the modulus m is usually chosen to be very large,
say 109 or more, so that the points in [0, 1] where the Ui’s can fall
are very dense; for m >= 109, there are at least a billion possible
values.
• Even full-period LCGs, however, can exhibit
nonuniform behavior in segments within a cycle.
• For example, if we generate only m/2 consecutive
Zi’s, they may leave large gaps in the sequence 0,
1, . . . , m - 1 of possible values.
• Thus, it is useful to know how to choose m, a, and c
so that the corresponding LCG will have full period.
• The following theorem, proved by Hull and Dobell
(1962), gives such a characterization:
• The LCG defined previously has full period if
and only if the following three conditions hold:
• (a) The only positive integer that (exactly)
divides both m and c is 1.
• (b) If q is a prime number (divisible by only
itself and 1) that divides m, then q divides a - 1.
• (c) If 4 divides m, then 4 divides a - 1.
• Obtaining a full (or at least a long) period is just one desirable
property for a good LCG.
• We also want good statistical properties (such as apparent
independence), computational and storage efficiency,
reproducibility, facilities for separate streams, and portability.
• Reproducibility is simple, for we must only remember the initial
seed used, Z0, and initiate the generator with this value again to
obtain the same sequence of Ui’s exactly.
• Also, we can easily resume generating the Zi’s at any point in the
sequence by saving the final Zi obtained previously and using it
as the new seed; this is a common way to obtain
nonoverlapping, “independent” sequences of random numbers.
 LINEAR CONGRUENTIAL GENERATORS
• Many random-number generators in use today
are linear congruential generators (LCGs),
introduced by Lehmer (1951).
• A sequence of integers Z1, Z2, . . . is defined by the
recursive formula
Zi = (aZi-1+c)(mod m)
• where m (the modulus), a (the multiplier), c (the
increment), and Z0 (the seed) are nonnegative
integers.
• The “looping” behavior of the LCG in the previous figure is
inevitable. By the definition (Zi = (aZi-1+c)(mod m)), whenever
Zi takes on a value it has had previously, exactly the same
sequence of values is generated, and this cycle repeats itself
endlessly.
• The length of a cycle is called the period of a generator. For
LCGs, Zi depends only on the previous integer Zi-1, and since 0
<= Zi <= m - 1, it is clear that the period is at most m; if it is in
fact m, the LCG is said to have full period.
• Clearly, if a generator is full-period, any choice of the initial
seed Z0 from {0, 1, . . . , m - 1} will produce the entire cycle in
some order. (25 slide)
• Therefore, 0 <= Zi <= m - 1, and to obtain the desired
random numbers Ui (for i = 1, 2, . . . ) in [0, 1], we let Ui =
Zi/m.
• We shall concentrate our attention for the most part on
the Zi’s, although the precise nature of the division of Zi
by m should be paid attention to due to differences in
the way various computers and compilers handle
floating-point arithmetic.
• In addition to non-negativity, the integers m, a, c, and Z0
should satisfy 0 < m, a < m, c < m, and Z0 < m.
• Streams are typically set up in a LCG by simply
specifying the initial seed for each stream.
• For example, if we want streams of length 1,000,000
each, we set Z0 for the first stream to some value,
then use Z1,000,000 as the seed for the second stream,
Z2,000,000 as the seed for the third stream, and so on.
• Because of condition (a) in the previously
demonstrated Theorem, LCGs tend to behave
differently for c > 0 (called mixed LCGs) than for c = 0
(called multiplicative LCGs).
• The LCG defined previously has full period if
and only if the following three conditions hold:
• (a) The only positive integer that (exactly)
divides both m and c is 1.
• (b) If q is a prime number (divisible by only
itself and 1) that divides m, then q divides a - 1.
• (c) If 4 divides m, then 4 divides a - 1.
 Mixed Generators
• For c > 0, condition (a) in the Theorem is possible, so
we might be able to obtain full period m, as we now
discuss.
• For a large period and high density of the Ui’s on [0,
1], we want m to be large.
• Furthermore, in the early days of computer
simulation when computers were relatively slow,
dividing by m to obtain the remainder was a relatively
slow arithmetic operation, and it was desirable to
avoid having to do this division explicitly…
 Multiplicative Generators
• Multiplicative LCGs are advantageous in that the
addition of c is not needed, but they cannot have full
period since condition (a) of the aforementioned
Theorem cannot be satisfied.
• As we shall see, however, it is possible to obtain
period m - 1 if m and a are chosen carefully.
• It’s still computationally efficient to choose m = 2b
and thus avoid explicit division.
• However, it can be shown that in this case the period
is at most 2b-2, that is, only one-fourth of the integers
0 through m - 1 can be obtained as values for the Zi’s.
 OTHER KINDS OF GENERATORS
• Although LCGs are probably the most widely
used and best understood kind of random-
number generator, there are many alternative
types.
• Most of these other generators have been
developed in an attempt to obtain longer
periods and better statistical properties.
 More General Congruences
• LCGs can be thought of as a special case of
generators defined by:
• Zi = g(Zi-1, Zi-2, . . .)(mod m)
• where g is a fixed deterministic function of
previous Zj’s. As with LCGs, the Zi’s defined by
the aforementioned Equation lie between 0
and m - 1, and the U(0, 1) random numbers
are given by Ui = Zi/m.
• One obvious generalization of LCGs would be to let
g(Zi-1, Zi-2, . . .) = a’Z2i-1 + aZi-1 + c, which produces
a quadratic congruential generator.
• A special case that has received some attention is
when a’ = a = 1, c = 0, and m is a power of 2, has
better statistical properties.
• Since Zi still depends only on Zi-1 (and not on
earlier Zj’s), and since 0 <= Zi <= m - 1, the period of
quadratic congruential generators is at most m, as
for LCGs.
• A different choice of the function g is to maintain linearity
but to use earlier Zj’s; this gives rise to generators called
multiple recursive generators (MRGs) and defined by
– g(Zi-1, Zi-2, . . .) = a1Zi-1 + a2Zi-2 + . . . + aq Zi-q
• where a1, a2, . . . , aq are constants.
• A period as large as mq - 1 then becomes possible if the
parameters are chosen properly.
• L’Ecuyer, Blouin, and Couture (1993) investigated such
generators and identified several specific generators of this
type that perform well, and give portable implementations.
 Composite Generators
• Several researchers have developed methods that
take two or more separate generators and combine
them in some way to generate the final random
numbers.
• It is hoped that this composite generator will exhibit a
longer period and better statistical behavior than any
of the simple generators composing it.
• The disadvantage in using a composite generator is, of
course, that the cost of obtaining each Ui is more than
that of using one of the simple generators alone.
• Perhaps the earliest kind of composite generators used a second
LCG to shuffle the output from the first LCG.
• Initially, a vector V = (V1, V2, . . . , Vk) is filled sequentially with the
first k Ui’s from the first LCG (k = 128 was originally suggested).
• Then the second LCG is used to generate a random integer I
distributed uniformly on the integers 1, 2, . . . , k, and VI is returned
as the first U(0, 1) variate; the first LCG then replaces this Ith
location in V with its next Ui s onward, and the second LCG
randomly chooses the next returned random number from this
updated V…
• Shuffling has a natural intuitive appeal, especially since we would
expect it to break up any correlation and greatly extend the period.
• Despite these apparent advantages of
rearranging a simple generator’s output in
some way, there is not as much known about
shuffled generators; for example, one cannot
jump into an arbitrary point of a shuffled
generator’s output sequence without
generating all the intermediate values,
whereas this is possible for LCGs.
• Another way to combine two generators (in simplified form)
is to let {Z1i} and {Z2i} denote the integer sequences
generated by two different LCGs with different moduli, then
let Zi = (Z1i - Z2i)(mod m) for some integer m, and finally set
Ui = Zi/m.
• This idea could clearly be extended to more than two
generators, and has several advantages; the period is very
long (at least 1018 in one example), the multipliers in each of
the component generators can be small (thus promoting
portability and usability on virtually any computer), the
resulting generator is quite fast, and the statistical
properties of these generators also appear to be very good.
• One very appealing class of generators is obtained by
combining multiple recursive generators (MRGs) in a
particular way.
• Here, J different MRGs are implemented simultaneously
to form the sequences {Z1,i}, {Z2,i}, . . . , {ZJ,i}.
• Letting m1 be the modulus used in the first of these
MRGs, and letting d1, d2, . . . , dJ be specified constants,
we define
– Yi = (d1Z1,i + d2Z2,i + . . . + dJZJ,i)(mod m1)
• and finally return Ui = Yi/m1 as the ith random number
for use in the simulation.
• Although the parameters for these kinds of generators must be
chosen very carefully, extremely long periods and exceptionally
good statistical properties are possible.
• L’Ecuyer (1999) carried out an extensive search for good parameter
settings and the simplest of them, where J = 2 MRGs are combined,
has a period of approximately 2191 as well as excellent statistical
properties.
Feedback Shift Register Generators