Section 7.

2: Generating Continuous Random Variates

Discrete-Event Simulation: A First Course
c 2006 Pearson Ed., Inc. 0-13-142917-5

Discrete-Event Simulation: A First Course

Section 7.2: Generating Continuous Random Variates

1/ 29

Section 7.2: Generating Continuous Random Variates

The inverse distribution function (idf) of X is the function F 1 : (0, 1) X for all u (0, 1) as F 1 (u ) = x where x X is the unique possible value for F (x ) = u
Assumes the cdf is strictly monotone increasing True if f (x ) > 0 for all x X

There is a one-to-one correspondence between possible values x X and cdf values u = F (x ) (0, 1)

Discrete-Event Simulation: A First Course

Section 7.2: Generating Continuous Random Variates

2/ 29

Continuous Random Variable idfs

Unlike the a discrete random variable, the idf for a continuous random variable is a true inverse 1.0
F 1 (u) = x F () u
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

0.0

Can sometimes determine the idf in closed form by solving F (x ) = u for x

Discrete-Event Simulation: A First Course

Section 7.2: Generating Continuous Random Variates

3/ 29

Examples

If X is Uniform(a, b ), F (x ) = (x a)/(b a) for a < x < b x = F 1 (u ) = a + (b a)u 0<u<1

If X is Exponential(), F (x ) = 1 exp(x /) for x > 0 x = F 1 (u ) = ln(1 u ) 0<u<1

If X is a continuous variable with possible value 0 < x < b and pdf f (x ) = 2x /b 2 , the cdf is F (x ) = (x /b )2 0<u<1 x = F 1 (u ) = b u

Discrete-Event Simulation: A First Course

Section 7.2: Generating Continuous Random Variates

4/ 29

Random Variate Generation By Inversion

X is a continuous random variable with idf F 1 () Continuous random variable U is Uniform(0, 1) Z is the continuous random variable dened by Z = F 1 (U ) Theorem (7.2.1) Z and X are identically distributed Algorithm 7.2.1
If X is a continuous random variable with idf F 1 (), a continuous random variate x can be generated as u = Random(); return F 1 (u);

Discrete-Event Simulation: A First Course

Section 7.2: Generating Continuous Random Variates

5/ 29

Inversion Examples
Example 7.2.4: Generating a Uniform(a, b ) Random Variate
u = Random(); return a + (b - a) * u;

Example 7.2.5: Generating an Exponential() Random Variate

u = Random(); return * log(1 - u);

U is Uniform(0, 1) i 1 U is Uniform(0, 1) Can generate Exponential() with

u = Random(); return * log(u);

Because this algorithm has reverse monotonicity, the algorithm in Example 7.2.5 is preferred.
Discrete-Event Simulation: A First Course Section 7.2: Generating Continuous Random Variates 6/ 29

Examples 7.2.4 and 7.2.5

Algorithms in Example 7.2.4 and 7.2.5 are ideal Both are portable, exact, robust, ecient, clear, synchronized and monotone It is not always possible to solve for a continuous random variable idf explicitly by algebraic techniques Two other options may be available
Use a function that accurately approximates F 1 () Determine the idf by solving u = F (x ) numerically

Discrete-Event Simulation: A First Course

Section 7.2: Generating Continuous Random Variates

7/ 29

Approximate Inversion

If Z is a Normal(0, 1), the cdf is the special function () The idf can be approximated as the ratio of two fourth degree polynomials (Odeh and Evans, 1974) The approximation is ecient and essentially has negligible error The idf 1 () cannot be evaluated in closed form

Discrete-Event Simulation: A First Course

Section 7.2: Generating Continuous Random Variates

8/ 29

Approximation of ()
For any u (0, 1), a Normal(0, 1) idf approximation is 1 1 (u ) a (u ) where
1 a (u ) =

t + p (t )/q (t ) 0.0 < u < 0.5 t p (t )/q (t ) 0.5 u < 1.0 2 ln(u ) 0.0 < u < 0.5 2 ln(1 u ) 0.5 u < 1.0

and t= and q (t ) = b0 + b1 t + + b4 t 4 The ten coecients can be chosen to produce an absolute error less than 109 for all 0.0 < u < 1.0
Discrete-Event Simulation: A First Course Section 7.2: Generating Continuous Random Variates 9/ 29

p (t ) = a0 + a1 t + + a4 t 4

Example 7.2.6

Inversion can be used to generate Normal(0, 1) variates: Example: 7.2.6: Generating a Normal(0, 1) Random Variate
u = Random(); 1 return a (u );

This algorithm is portable, essentially exact, robust, reasonably ecient, synchronized and monotone Clarity?

Discrete-Event Simulation: A First Course

Section 7.2: Generating Continuous Random Variates

10/ 29

Alternative Method 1

If U1 , U2 , . . . , U12 is an iid sequence of Uniform(0, 1), Z = U1 + U2 + . . . + U12 6 is approximately Normal(0, 1)

The mean is 0.0 and the standard deviation is 1.0 Possible values are 6.0 < z < 6.0 Justication is provided by the central limit theorem (Section 8.1) This algorithm is: portable, robust, relatively ecient and clear This algorithm is not: exact, synchronized or monotone

Discrete-Event Simulation: A First Course

Section 7.2: Generating Continuous Random Variates

11/ 29

Alternative Method 2

If U1 and U2 are independent Uniform(0, 1) RVs then Z1 = and Z2 = 2 ln(U1 ) sin(2 U2 ) will be independent Normal(0, 1) RVs (Box and Muller, 1958) This algorithm is: portable, exact, robust and relatively ecient; This algorithm is not: clear or monotone The algorithm is synchronized only in pair-wise fashion 2 ln(U1 ) cos(2 U2 )

Discrete-Event Simulation: A First Course

Section 7.2: Generating Continuous Random Variates

12/ 29

Normal and Lognormal Random Variates

Random variates corresponding to Normal(, ) and Lognormal(a, b ) can be generated by using a Normal(0, 1) random variate generator Example 7.2.7: Generating a Normal(, ) Random Variate
z = Normal(0.0, 1.0); return + z ; /* see Definition 7.1.7 */

Example 7.2.8: Generating a Lognormal(a, b ) Random Variate

z = Normal(0.0, 1.0); return exp(a + b * z); /* see Definition 7.1.8 */

Both algorithms are essentially ideal

Discrete-Event Simulation: A First Course Section 7.2: Generating Continuous Random Variates 13/ 29

Numerical Inversion
Numerical inversion provides another way to generate continuous random variates; that is, u = F (x ) can be solved for x iteratively Newtons method provides a good compromise between rate of convergence and robustness Given u (0, 1), let t be close to the value of x for which u = F (x ) If F () is expanded in a Taylors series about the point t F (x ) = F (t ) + F (t )(x t ) + Recall F (t ) = f (t ) For small |x t |, ignore (x t )2 and higher order terms
Discrete-Event Simulation: A First Course Section 7.2: Generating Continuous Random Variates 14/ 29

1 F (t )(x t )2 + 2!

Newtons Method
Set u = F (x ) F (t ) + f (t )(x t ) and solve for x to obtain x t+ u F (t ) f (t )

Use initial guess t0 and iterate to solve for x : ti x as i ti +1 = ti +

1.0
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

u F (ti ) f (ti )

i = 0, 1, 2,
F ()

u F (ti )

0.0 ti
Discrete-Event Simulation: A First Course

x ti+1
15/ 29

Section 7.2: Generating Continuous Random Variates

Two Issues Relative to Newtons Method

The choice of an initial value t0

The best choice for the initial value is the mode For most continuous RVs described in text, t0 = is an essentially equivalent choice

The test for convergence

Given a convergence parameter > 0 Iterate until |ti +1 ti | <

Discrete-Event Simulation: A First Course

Section 7.2: Generating Continuous Random Variates

16/ 29

Algorithm 7.2.2
Algorithm 7.2.2
Given u (0, 1), the pdf f (), the cdf F () and a convergence parameter > 0, this algorithm will solve for x = F 1 (u ) x = ; /* is E[X]*/ do { t = x; x = t + (u - F(t)) / f(t); } while (|x-t| > ); return x; /* x is F 1 (u )*/

If u is small and X is non-negative, a negative value of x may occur early in the iterative process. Negative t will cause F (t ) and f (t ) to be undened for positive RVs
Discrete-Event Simulation: A First Course Section 7.2: Generating Continuous Random Variates 17/ 29

Modied Algorithm 7.2.2

The following modication can be used to avoid the problem Modied Algorithm 7.2.2
x = ; /* is E[X]*/ do { t = x; x = t + (u - F(t)) / f(t); if (x <= 0.0) x = 0.5 * t; } while (|x-t| > ); return x; /* x is F 1 (u )*/

Algorithms 7.2.1 and 7.2.2 together provide a general purpose inversion approach to continuous random variate generation E.g., the Erlang(n, b ) idf function in rvms is based on Alg.7.2.2 and can be used with Algorithm 7.2.1
Discrete-Event Simulation: A First Course Section 7.2: Generating Continuous Random Variates 18/ 29

Alternative Random Variate Generation Algorithms

Erlang Random Variates An Erlang(n, b ) random variate can be generated by summing n Exponential(b ) random variates Generating an Erlang(n, b ) Random Variate
x = 0.0; for (i = 0; i < n; i++) x += Exponential(b); return x;

The algorithm is: portable, exact, robust, and clear The algorithm is not ecient (it is O(n)), synchronized or monotone

Discrete-Event Simulation: A First Course

Section 7.2: Generating Continuous Random Variates

19/ 29

Modied Algorithms for Erlang Random Variates

To increase computational eciency, use Generating an Erlang(n, b ) Random Variate

t = 1.0; for (i = 0; i < n; i++) t *= (1.0 - Random()); return -b * log(t);

This algorithm requires only one log() evaluation, rather than n Can further improve eciency by using t *= Random(); The algorithm remains O(n), so is not ecient if n is large

Discrete-Event Simulation: A First Course

Section 7.2: Generating Continuous Random Variates

20/ 29

Chisquare Random Variates

If n is an even positive integer, an Erlang(n/2, 2) random variate is equivalent to a Chisquare(n) random variable X is a Chisquare (n) random variable i 2 + Z 2 + + Z 2 where Z , Z , . . . , Z are iid X = Z1 1 2 n n 2 Normal(0, 1) random variables Generating a Chisquare(n) Random Variate
x = 0.0; for (i = 0; i < n; i++){ z = Normal(0.0, 1.0); x += (z * z); } return x;

The algorithm is: portable, exact, robust, clear The algorithm is not: ecient(it is O(n)), synchronized or monotone
Discrete-Event Simulation: A First Course Section 7.2: Generating Continuous Random Variates 21/ 29

Student Random Variates

X is Student(n) i X = Z / V /n where
Z is Normal(0, 1) V is Chisquare(n) Z and V are independent

Generating a Student(n) Random Variate

z = Normal(0.0, 1.0); v = Chisquare(n); return z / sqrt(v / n);

The algorithm is: portable, exact, robust, clear The algorithm is not synchronized or monotone Eciency depends on algs. used for Normal and Chisquare

Discrete-Event Simulation: A First Course

Section 7.2: Generating Continuous Random Variates

22/ 29

Testing for Correctness using Histograms

A natural way to do this at the computational level is:

use the algorithm to generate a sample of n random variates and construct a k -bin continuous-data histogram with bin width is the histogram density and f (x ) is the pdf f f (x ) f as n and 0

In practice, using a large but nite value of n and a small but (x ) and f (x ) non-zero value of , perfect agreement between f will not be achieved
In the discrete case, it is due to natural sampling variability In the continuous case, the quantization error associated with binning the sample is an additional factor

Discrete-Event Simulation: A First Course

Section 7.2: Generating Continuous Random Variates

23/ 29

Quantization Error

Let B = [m /2, m + /2] be a small histogram bin Use the Taylor expansion of f (x ) at x = m
f (x ) = f (m)+f (m)(x m)+ 1 1 f (m)(x m)2 + f (m)(x m)3 + 2! 3!

The probability of falling within the bin is Pr(x B ) = f (x )dx = = f (m) + 1 f (m)3 + 24

Discrete-Event Simulation: A First Course

Section 7.2: Generating Continuous Random Variates

24/ 29

Quantization Error (2)

For all x B , the histogram density is (x ) = 1 Pr(X B ) f (m) + 1 f (m)2 f 24 Unless f (m) = 0, there is a positive or negative bias between
(x ), the experimental density of the histogram bin and f f (m), the theoretical pdf evaluated at the bin midpoint

This bias may be signicant if the curvature of the pdf is large at the bin midpoint

Discrete-Event Simulation: A First Course

Section 7.2: Generating Continuous Random Variates

25/ 29

Example 7.2.9
X is a continuous random variable with pdf f (x ) = The cdf X is
x

2 (x + 1)3

x >0

F (x ) =
0

f (t )dt = 1

1 (x + 1)2

x >0

The idf is F 1 (u ) = 1 1 1u 0<u<1

Note the pdf curvature is very large close to x = 0; therefore, the histogram will not match the pdf well for the bins close to x =0
Discrete-Event Simulation: A First Course Section 7.2: Generating Continuous Random Variates 26/ 29

Example 7.2.9 ctd.

Random variates for X can be generated using inversion Correctness of the inversion can be tested by constructing a histogram (x ) and Using histogram bin widths of = 0.5, as n , f f (m) are (with d.dddd precision): m : 0.25 0.75 1.25 1.75 2.25 2.75 (x ) : 1.1111 0.3889 0.1800 0.0978 0.0590 0.0383 f f (m) : 1.0240 0.3732 0.1756 0.0962 0.0583 0.0379 For the rst bin (m = 0.25), the curvature bias is 1 f (m)2 = 0.08192 24

Discrete-Event Simulation: A First Course

Section 7.2: Generating Continuous Random Variates

27/ 29

Testing for Correctness using the Empirical cdf

Compare the empirical cdf (section 4.3) with the population cdf F (x ) Eliminates binning quantization error (x ) F (x ) For large samples (as n ), F

Discrete-Event Simulation: A First Course

Section 7.2: Generating Continuous Random Variates

28/ 29

Library rvgs

Contains 7 continuous random variate generators

double double double double double double double Chisquare(long n) Erlang(long n, double b ) Exponential(double ) Lognormal(double a, double b ) Normal(double , double ) Student(long n) Uniform(double a, double b )

Discrete-Event Simulation: A First Course

Section 7.2: Generating Continuous Random Variates

29/ 29