Chapter 7

Random-Number
Generation

Banks, Carson, Nelson & Nicol

Discrete-Event System Simulation
Contents
• Properties of Random Numbers
• Pseudo-Random Numbers
• Generating Random Numbers
• Linear Congruential Method
• Combined Linear Congruential Method
• Tests for Random Numbers
 Frequency test
 Autocorrelation test.

2
Discuss characteristics and the generation of
random numbers.
• Subsequently, introduce tests for randomness:
• Frequency test
• Autocorrelation test

3
Random Number Generators
Random number generation is a method of
producing a sequence of numbers that lack any
discernible pattern.

Random Number Generators (RNGs) have

applications in a wide range of different fields and
are used heavily in computational simulations.

4
What is Random?
What do we mean by Random?
•Uncorrelated (with what you're interested in)
•Quantum systems are truly random (we
believe)
•Some times we want repeatability from a random
number generator.

5
Properties of Random Numbers
 Two important statistical properties:
 Uniformity
 Independence.
 Random Number, Ri, must be independently drawn from a
uniform distribution with pdf:

1, 0  x  1
f ( x)  
0, otherwise
1
2
1 x 1

E ( R )  xdx 
0 2
0

2
Figure: pdf for
 Calculate variance? random numbers
6
Continue
 Some consequences of the informality and
independence properties are the following:
1. If the interval [0,1] is divided into n classes,or
subinterval of equal length , the expected
number of observations in each interval is N/n,
where N is the total number of observations.
2. The probability of observing a values in a
particular interval is independent of the previous
values drawn

7
Generation of Pseudo-Random Numbers
 “Pseudo”, because generating numbers using a known method
removes the potential for true randomness.
 Goal: To produce a sequence of numbers in [0,1] that simulates, or
imitates, the ideal properties of random numbers (RN).
 Important considerations in RN routines:
 Fast
 Portable to different computers
 Have sufficiently long cycle
 Replicable
• Verification and debugging
• Use identical stream of random numbers for different systems
 Closely approximate the ideal statistical properties of uniformity and
independence.

8
Techniques for Generating Random
Numbers
 Linear Congruential Method (LCM).
 Widely used
 Combined Linear Congruential Generators (CLCG).
 Random-Number Streams.

9
Linear Congruential Method [Techniques]

X i 1  (aX i  c) mod m, i  0,1,2,...

The The The

multiplier increment modulus

Xi
Ri  , i  1,2,...
m
10
Continue

11
Example [LCM]

 Use X0 = 27, a = 17, c = 43, and m = 100.

 The Xi and Ri values are:
X1 = (17*27+43) mod 100 = 502 mod 100 = 2, R1 = 0.02;
X2 = (17*2+32) mod 100 = 77, R2 = 0.77;
X3 = (17*77+32) mod 100 = 52, R3 = 0.52;
…

12
Linear Congruential Generator
 Example Sequences:

m=10, a=2, c=1 m=10, a=1, c=7

1 1
3 (2*1 + 1 % 10) 8 (1*1 + 7 % 10)
7 (2*3 + 1 % 10) 5 (1*8 + 7 % 10)
5 (2*7 + 1 % 10) 2 (1*5 + 7 % 10)
1 (2*5 + 1 % 10) 9 (1*2 + 7 % 10)
3 (2*1 + 1 % 10) 6 (1*9 + 7 % 10)
7 (2*3 + 1 % 10) 3 (1*6 + 7 % 10)
5 (2*7 + 1 % 10) 0 (1*3 + 7 % 10)
… …

13
Characteristics of a Good Generator
[LCM]
 Maximum Density
 Such that the values assumed by Ri, i = 1,2,…, leave no large
gaps on [0,1]
 Problem: Instead of continuous, each Ri is discrete
 Solution: a very large integer for modulus m
 Approximation appears to be of little consequence
 Maximum Period
 To achieve maximum density and avoid cycling.
 Achieve by: proper choice of a, c, m, and X0.
 Most digital computers use a binary representation of
numbers
 Speed and efficiency are aided by a modulus, m, to be (or close
to) a power of 2.

14
15
Example

16
Combined Linear Congruential Generators
[Techniques]
 Reason: Longer period generator is needed because of the
increasing complexity of stimulated systems.
 Approach: Combine two or more multiplicative congruential
generators.
 Let Xi,1, Xi,2, …, Xi,k, be the ith output from k different
multiplicative congruential generators.
 The jth generator:
 Has prime modulus mj and multiplier aj and period is mj-1

 Produces integers Xi,j is approx ~ Uniform on integers in [1,

m-1]
 Wi,j = Xi,j -1 is approx ~ Uniform on integers in [1, m-2]

17
Combined Linear Congruential Generators
[Techniques]
 Suggested form:
 Xi
 k   m , X i 0
X i    (1) X i , j  mod m1  1
 j 1
Hence, Ri   1
 j 1  m 1
 1 , Xi  0
 m1
The coefficient:
Performs the
subtraction Xi,1-1

 The maximum possible period is:

(m1  1)(m2  1)...(mk  1)
P
2 k 1

18
Combined Linear Congruential Generators
[Techniques]
 Example: For 32-bit computers, L’Ecuyer [1988] suggests combining
k = 2 generators with m1 = 2,147,483,563, a1 = 40,014, m2 =
2,147,483,399 and a2 = 20,692. The algorithm becomes:
Step 1: Select seeds
 X1,0 in the range [1, 2,147,483,562] for the 1st generator

 X2,0 in the range [1, 2,147,483,398] for the 2nd generator.

Step 2: For each individual generator,
X1,j+1 = 40,014 X1,j mod 2,147,483,563
X2,j+1 = 40,692 X1,j mod 2,147,483,399.
Step 3: Xj+1 = (X1,j+1 - X2,j+1 ) mod 2,147,483,562.
Step 4: Return  X j 1
 , X j 1  0
R j 1   2,147,483, 563
 2,147,483,562 , X j 1  0
 2,147,483,563
Step 5: Set j = j+1, go back to step 2.
 Combined generator has period: (m1 – 1)(m2 – 1)/2 ~ 2 x 1018
19
Random-Numbers Streams [Techniques]

 The seed for a linear congruential random-number generator:

 Is the integer value X0 that initializes the random-number sequence.
 Any value in the sequence can be used to “seed” the generator.
 A random-number stream:
 Refers to a starting seed taken from the sequence X0, X1, …, XP.
 If the streams are b values apart, then stream i could defined by starting
seed: Si  X b (i 1)

 Older generators: b = 105; Newer generators: b = 1037.

 A single random-number generator with k streams can act like k
distinct virtual random-number generators
 To compare two or more alternative systems.
 Advantageous to dedicate portions of the pseudo-random number
sequence to the same purpose in each of the simulated systems.

20
Tests for Random Numbers
 Two categories:
 Testing for uniformity:
H0: Ri ~ U[0,1]
H1: Ri ~/ U[0,1]
 Failure to reject the null hypothesis, H0, means that evidence of
non-uniformity has not been detected.
 Testing for independence:
H0: Ri ~ independently
H1: Ri ~/ independently
 Failure to reject the null hypothesis, H0, means that evidence of
dependence has not been detected.
 Level of significance the probability of rejecting H0 when it
is true:  = P(reject H0|H0 is true)
21
Tests for Random Numbers
 When to use these tests:
 If a well-known simulation languages or random-number
generators is used, it is probably unnecessary to test
 If the generator is not explicitly known or documented, e.g.,
spreadsheet programs, symbolic/numerical calculators, tests
should be applied to many sample numbers.
 Types of tests:
 Theoretical tests: evaluate the choices of m, a, and c without
actually generating any numbers
 Empirical tests: applied to actual sequences of numbers
produced. Our emphasis.

22
Frequency Tests [Tests for RN]

 Test of uniformity
 Two different methods:
 Kolmogorov-Smirnov test
 Chi-square test

23
Kolmogorov-Smirnov Test [Frequency Test]

 Compares the continuous cdf, F(x), of the uniform

distribution with the empirical cdf, SN(x), of the N sample
observations.
 We know: F ( x )  x, 0  x  1
 If the sample from the RN generator is R1, R2, …, RN, then the
empirical cdf, SN(x) is:
number of R1 , R2 ,..., Rn which are  x
S N ( x) 
N

 Based on the statistic: D = max| F(x) - SN(x)|

 Sampling distribution of D is known (a function of N, tabulated in Table
A.8.)
 A more powerful test, recommended.

24
 Kolmogorov-Smirnov Test [Frequency Test]

 Example: Suppose 5 generated numbers are 0.44, 0.81, 0.14,

0.05, 0.93. Arrange R from (i)

Step 1: R(i) 0.05 0.14 0.44 0.81 0.93 smallest to largest

i/N 0.20 0.40 0.60 0.80 1.00

D+ = max {i/N – R(i)}
i/N – R(i) 0.15 0.26 0.16 - 0.07
Step 2:
R(i) – (i-1)/N 0.05 - 0.04 0.21 0.13
D- = max {R(i) - (i-1)/N}

Step 3: D = max(D+, D-) = 0.26

Step 4: For  = 0.05,
D = 0.565 > D

Hence, H0 is not rejected.

25
Chi-square test [Frequency Test]

 Chi-square test uses the sample statistic:

n is the # of classes Ei is the expected
(Oi  Ei )
n 2 # in the ith class
 
2
0
i 1 Ei Oi is the observed
# in the ith class

 Approximately the chi-square distribution with n-1 degrees of

freedom (where the critical values are tabulated in Table A.6)
 For the uniform distribution, Ei, the expected number in the each
class is: N
Ei  , where N is the total # of observation
n

 Valid only for large samples, e.g. N >= 50

26
Chi-square Test: Example

27
Tests for Autocorrelation [Tests for RN]

 Testing the autocorrelation between every m numbers

(m is a.k.a. the lag), starting with the ith number
 The autocorrelation im between numbers: Ri, Ri+m, Ri+2m, Ri+(M+1)m
 M is the largest integer such that
i  (M  1 )m  N
 Hypothesis:

H 0 :  im  0, if numbers are independent

H1 :  im  0, if numbers are dependent
 If the values are uncorrelated:
 For large values of M, the distribution of the estimator of im,
denoted is approximately normal.
̂ im

28
Tests for Autocorrelation [Tests for RN]

 Test statistics is: ˆ im

Z0 
ˆ ˆim
 Z0 is distributed normally with mean = 0 and variance = 1, and:
1 M 
ˆρim   
M  1  k 0
Ri  km Ri (k 1 )m   0.25

13M  7
σˆ ρim 
12(M  1 )
 If im > 0, the subsequence has positive autocorrelation
 High random numbers tend to be followed by high ones, and vice versa.
 If im < 0, the subsequence has negative autocorrelation
 Low random numbers tend to be followed by high ones, and vice versa.

29
Example [Test for Autocorrelation]

 Test whether the 3rd, 8th, 13th, and so on, for the following
output on P. 265.
 Hence,  = 0.05, i = 3, m = 5, N = 30, and M = 4

1 (0.23)(0.28)  (0.25)(0.33)  (0.33)(0.27)

ρˆ 35    0.25
4  1  (0.28)(0.05)  (0.05)(0.36) 
 0.1945
13(4)  7
σˆ ρ35   0.128
12( 4  1 )
0.1945
Z0    1.516
0.1280
 From Table A.3, z0.025 = 1.96. Hence, the hypothesis is not
rejected.
30
Shortcomings [Test for Autocorrelation]

 The test is not very sensitive for small values of M,

particularly when the numbers being tests are on the low
side.
 Problem when “fishing” for autocorrelation by performing
numerous tests:
 If  = 0.05, there is a probability of 0.05 of rejecting a true
hypothesis.
 If 10 independence sequences are examined,
 The probability of finding no significant autocorrelation, by

chance alone, is 0.9510 = 0.60.

 Hence, the probability of detecting significant autocorrelation

when it does not exist = 40%

31
Summary
 In this chapter, we described:
 Generation of random numbers
 Testing for uniformity and independence

 Caution:
 Even with generators that have been used for years, some of
which still in used, are found to be inadequate.
 This chapter provides only the basic
 Also, even if generated numbers pass all the tests, some
underlying pattern might have gone undetected.

32