Industrial Systems Engineering - International University – VNU-HCM SIMULATION

Random Numbers _ Random Variates

_Input Modeling

Discrete-Event System Simulation

1
Steps in a Simulation Study

- Why do we need to do
simulation?

- What do we need to
create a simulation
model?

- What are we to do with

a simulation model?

2
2
Inputs Outputs

Random Statistics

3
 Input modelling

Probability Distributions
Random Variates
EXPO(..)
Data TRIA(..)
NORM(..)
WEIB(..)
….???

• What are Random Variates and Probability

Distributions ?
• How does the computer generate Random Variates
from Probability Distributions?
4
 Random Variates

▪ Customers arrive at random times from

1 to 8 minutes apart, with equal ▪ The service times vary from 1 to 6
probability of occurrence: minutes,
with probabilities:

5
 Random Variates
Example of Arrival process

Time Period: 10 minutes

 Histogram of Random Variate
Example 9.5 page 342
The number of vehicles at the northwest corner of an intersection in a 5 minute
period between 7:00 AM and 7:05 AM was monitored for five workday over a 20-
week period.

Table 9.1 Number of Arrivals in a 5-Minute Period

Fig 9.4 Histogram of number of arrivals per period

7
Theoretical Probability Distributions
Poisson
Binomial

Triangular
Uniform
Normal Exponential

Gamma Weibull
How does the computer generate Random Variates from
Probability Distributions?

… Starting with Random Numbers

10
1.What are properties of Random Numbers?

2. How to generate Random Numbers?

3. How to make sure that a series of numbers is

Random Numbers?
1. Properties of Random numbers

f(x)
1
Figure 1. pdf for
random numbers

x
0 1

12
 2. Generation of Pseudo-random numbers
Goal: to produce a sequence of numbers between 0 and 1 that
simulates the ideal properties: uniformity and independence.
Considerations for the method:
-Fast.
-Portable to different computers, different program languages.
-Long cycle. Cycle length (maximal period): length of RN
sequence before previous number repeated.
-RN should be repeatable.
-Generated RN should be uniform and independent.

13
3. Techniques for Generating RNs

14
3. Techniques for Generating RNs (cont.)

Example

● Use X0 = 27, a = 17, c = 43, and m = 100.

● The Xi and Ri values are:
X1 = (17*27+43) mod 100 = 502 mod 100 = 2, R1 =
0.02;
X2 = (17*2+43) mod 100 = 77, R2 =
0.77;
Example: compareX3 = (17*77+43) mod 100 = 52, R3 =
0.52;
m =9; a=2;… c=0, Xo = 1
m =9; a=2; c=0, Xo = 3
m =9; a=4; c=1, Xo = 0

15
 3. Techniques for Generating RNs (cont.)

Characteristics of a Good Generator

● Maximum Density
– Such that values assumed by Ri, i = 1,2,…, leave no large gaps on [0,1]
– Problem: Instead of continuous, each Ri is discrete
– Solution: a very large integer for modulus m (e.g., 231-1, 248)
● Approximation appears to be of little consequence

● Maximum Period
– To achieve maximum density and avoid cycling.
– Achieve by: proper choice of a, c, m, and X0.

● Most digital computers use a binary representation of numbers

– Speed and efficiency are aided by a modulus, m, to be (or close to) a power of 2.
16
17
 3. Techniques for Generating RNs (cont.)
3.2. Combined Linear Congruential Generators (1)

● Reason: Longer period generator is needed because of the increasing

complexity of stimulated systems.
● Approach: Combine two or more multiplicative congruential generators.
● Let Xi,1, Xi,2, …, Xi,k, be the ith output from k different multiplicative congruential
generators.
– The jth generator:
●Has prime modulus mj and multiplier aj and period is mj-1
●Produces integers Xi,j is approx ~ Uniform on integers in [1,
m-1]
●Wi,j = Xi,j -1 is approx ~ Uniform on integers in [1, m-2]

18
3. Techniques for Generating RNs (cont.)
3.2. Combined Linear Congruential Generators (2)
– Suggested form:

The coefficient:
Performs the
subtraction Xi,1- 1

●The maximum possible period is:

19
 3. Techniques for Generating RNs (cont.)
3.2. Combined Linear Congruential Generators (3)
● Example: For 32-bit computers, L’Ecuyer [1988] suggests combining k = 2
generators with m1 = 2,147,483,563, a1 = 40,014, m2 = 2,147,483,399 and a2 =
20,692. The algorithm becomes:
Step 1: Select seeds
● X1,0 in the range [1, 2,147,483,562] for the 1st generator
● X2,0 in the range [1, 2,147,483,398] for the 2nd generator.
Step 2: For each individual generator,
X1,j+1 = 40,014 X1,j mod 2,147,483,563
X2,j+1 = 40,692 X1,j mod 2,147,483,399.
Step 3: Xj+1 = (X1,j+1 - X2,j+1 ) mod 2,147,483,562.
Step 4: Return

Step 5: Set j = j+1, go back to step 2.

– Combined generator has period: (m1 – 1)(m2 – 1)/2 ~ 2 x 1018
20
3. Techniques for Generating RNs (cont.)
3.3. Random-Number Streams

● The seed for a linear congruential random-number generator:

– Is the integer value X0 that initializes the random-number sequence.
– Any value in the sequence can be used to “seed” the generator.
● A random-number stream:
– Refers to a starting seed taken from the sequence X0, X1, …, XP.
– If the streams are b values apart, then stream i could defined by starting seed:

– Older generators: b = 105; Newer generators: b = 1037.

● A single random-number generator with k streams can act like k distinct virtual
random-number generators
● To compare two or more alternative systems.
– Advantageous to dedicate portions of the pseudo-random number sequence to
the same purpose in each of the simulated systems.
21
 4. Tests for RNs
● Two categories:
– Testing for uniformity:
H0: Ri ~ U[0,1]
H/1: Ri ~ U[0,1]
●Failure to reject the null hypothesis, H0, means that evidence of
non-uniformity has not been detected.
– Testing for independence:
H0: Ri ~ independently
H/1: Ri ~ independently
●Failure to reject the null hypothesis, H0, means that evidence of
dependence has not been detected.

● Level of significance α, the probability of rejecting H0 when it is true:

α = P(reject H0|H0 is true)

22
 4. Tests for RNs (cont.)

● When to use these tests:

– If a well-known simulation languages or random-number generators is
used, it is probably unnecessary to test
– If the generator is not explicitly known or documented, e.g.,
spreadsheet programs, symbolic/numerical calculators, tests should be
applied to many sample numbers.

● Two type of tests:

– Frequency Test: Uses the Kolmogorov-Smirnov or the Chi-square test
to compare the distribution of the set of numbers generated to a uniform
distribution
– Autocorrelation test: Tests the correlation between numbers and
compares the sample correlation to the expected correlation, zero

23
4. Tests for RNs (cont.)
4.1. Frequency Tests
● Test of uniformity
● Two different methods:
– Kolmogorov-Smirnov test
– Chi-square test

24
4. Tests for RNs (cont.)
4.1. Frequency Tests
4.1.1. Kolmogorov-Smirnov Test

● Compares the continuous cdf, F(x), of the uniform distribution with the
empirical cdf, SN(x), of the N sample observations.
– We know:
– If the sample from the RN generator is R1, R2, …, RN, then the empirical
cdf, SN(x) is:

● Based on the statistic: D = max| F(x) - SN(x)|

– Sampling distribution of D is known (a function of N, tabulated in Table A.8)
● A more powerful test, recommended.

25
 4. Tests for RNs (cont.)
4.1. Frequency Tests
4.1.1. Kolmogorov-Smirnov Test (2)
● Example: Suppose 5 generated numbers are 0.44, 0.81, 0.14, 0.05, 0.93.
Arrange R(i) from
smallest to largest
Step 1: R(i) 0.05 0.14 0.44 0.81 0.93

i/N 0.20 0.40 0.60 0.80 1.00

D+ = max {i/N – R(i)}
i/N – R(i) 0.15 0.26 0.16 - 0.07
Step 2:
R(i) – (i-1)/N 0.05 - 0.04 0.21 0.13
D- = max {R(i) - (i-1)/N}
Step 3: D = max(D+, D-) = 0.26
Step 4: For α = 0.05,
Dα = 0.565 > D

Hence, H0 is not rejected.

26
4. Tests for RNs (cont.)
4.1. Frequency Tests
4.1.1. Chi-Square Test
● Chi-square test uses the sample statistic:

n is the # of classes Ei is the expected

# in the ith class

Oi is the observed
# in the ith class

– Approximately the chi-square distribution with n-1 degrees of

freedom (where the critical values are tabulated in Table A.6)
– For the uniform distribution, Ei, the expected number in each equally
spaced class is:

● Valid only for large samples, e.g. N >= 50

n and N be chosen so that each Ei ≥ 5 27

28
29
30
4. Tests for RNs (cont.)
4.2. Tests for Autocorrelation (1)

● Testing the autocorrelation between every l numbers (l is as known as.

the lag), starting with the ith number
– The autocorrelation ρil between numbers: Ri, Ri+l, Ri+2l, Ri+(M+1)l
– M is the largest integer such that

● Hypothesis:

● If the values are uncorrelated:

– For large values of M, the distribution of the estimator of ρil, denoted
is approximately normal.

31
4. Tests for RNs (cont.)
4.2. Tests for Autocorrelation (2)

● Test statistics is:

– Z0 is distributed normally with mean = 0 and variance = 1, and:

● If ρil > 0, the subsequence has positive autocorrelation

– High random numbers tend to be followed by high ones, and vice versa.
● If ρil < 0, the subsequence has negative autocorrelation
– Low random numbers tend to be followed by high ones, and vice versa.
32
Normal Hypothesis Test

33
4. Tests for RNs (cont.)
4.2. Tests for Autocorrelation (4)
Example

34
4. Tests for RNs (cont.)
4.2. Tests for Autocorrelation (4)

35 35
 4. Tests for RNs (cont.)
4.2. Tests for Autocorrelation (5)
Shortcomings
● The test is not very sensitive for small values of M, particularly when the
numbers being tests are on the low side.
● Problem when “fishing” for autocorrelation by performing numerous tests:
– If α = 0.05, there is a probability of 0.05 of rejecting a true hypothesis.
– If 10 independence sequences are examined,
●The probability of finding no significant
autocorrelation, by chance alone, is 0.9510 = 0.60.
●Hence, the probability of detecting significant
autocorrelation when it does not exist = 40%

36
 5. Summary

● In this chapter, we described:

– Generation of random numbers
– Testing for uniformity and independence

● Caution:
– Even with generators that have been used for years, some of which still
in used, are found to be inadequate.
– This chapter provides only the basic
– Also, even if generated numbers pass all the tests, some underlying
pattern might have gone undetected.

37
 What’s next?

How does the computer generate Random Variates

following a Probability Distribution?

… Starting with Random Numbers

How does the computer use Random Numbers to

generate Random Variates that follow a Probability
Distribution ?

38
Homework

(use excel)

39