32.

Monte Carlo techniques 1

32. MONTE CARLO TECHNIQUES

Revised July 1995 by S. Youssef (SCRI, Florida State University).
Updated February 2000 by R. Cousins (UCLA) in consultation with
F. James (CERN).
Monte Carlo techniques are often the only practical way to
evaluate difficult integrals or to sample random variables governed
by complicated probability density functions. Here we describe an
assortment of methods for sampling some commonly occurring
probability density functions.

32.1. Sampling the uniform distribution

Most Monte Carlo sampling or integration techniques assume a random number
generator which generates uniform statistically independent values on the half open
interval [0, 1). There is a long history of problems with various generators on a finite
digital computer, but recently, the RANLUX generator [1] has emerged with a solid
theoretical basis in chaos theory. Based on the method of L
uscher, it allows the user to
select different quality levels, trading off quality with speed.
Other generators are also available which pass extensive batteries of tests for statistical
independence and which have periods which are so long that, for practical purposes, values
from these generators can be considered to be uniform and statistically independent. In
particular, the lagged-Fibonacci based generator introduced by Marsaglia, Zaman, and
Tsang [2] is efficient, has a period of approximately 1043 , produces identical sequences
on a wide variety of computers and, passes the extensive DIEHARD battery of
tests [3]. Many commonly available congruential generators fail these tests and often have
sequences (typically with periods less than 232 ) which can be easily exhausted on modern
computers and should therefore be avoided [4].

32.2. Inverse transform method

If the desired probability density function is f (x) on the range < x < , its
cumulative distribution function (expressing the probability that x a) is given by
Eq. (30.6). If a is chosen with probability density f (a), then the integrated probability
up to point a, F (a), is itself a random variable which will occur with uniform probability
density on [0, 1]. If x can take on any value, and ignoring the endpoints, we can then find
a unique x chosen from the p.d.f. f (s) for a given u if we set
u = F (x) ,

(32.1)

provided we can find an inverse of F , defined by

x = F 1 (u) .

(32.2)

This method is shown in Fig. 32.1a. It is most convenient when one can calculate by
hand the inverse function of the indefinite integral of f . This is the case for some common
functions f (x) such as exp(x), (1 x)n , and 1/(1 + x2 ) (Cauchy or Breit-Wigner),
although it does not necessarily produce the fastest generator. CERNLIB contains
routines to implement this method numerically, working from functions or histograms.

CITATION: K. Hagiwara et al., Physical Review D66, 010001-1 (2002)

available on the PDG WWW pages (URL:

http://pdg.lbl.gov/)

June 18, 2002

13:58

32. Monte Carlo techniques

1

(a)

Continuous
distribution

F(x)
u
0

x = F 1(u)

(b)

F(x)
u

Discrete
distribution

} f(xk)

xk xk+1

Figure 32.1: Use of a random number u chosen from a uniform distribution (0,1)
to find a random number x from a distribution with cumulative distribution function
F (x).
For a discrete distribution, F (x) will have a discontinuous jump of size f (xk ) at each
allowed xk , k = 1, 2, . Choose u from a uniform distribution on (0,1) as before. Find
xk such that
k
X
F (xk1 ) < u F (xk ) Prob (x xk ) =
f (xi ) ;
(32.3)
i=1

then xk is the value we seek (note: F (x0 ) 0). This algorithm is illustrated in Fig. 32.1b.

32.3. Acceptance-rejection method (Von Neumann)

Very commonly an analytic form for F (x) is unknown or too complex to work with, so
that obtaining an inverse as in Eq. (32.2) is impractical. We suppose that for any given
value of x the probability density function f (x) can be computed and further that enough
is known about f (x) that we can enclose it entirely inside a shape which is C times an
easily generated distribution h(x) as illustrated in Fig. 32.2.
Frequently h(x) is uniform or is a normalized sum of uniform distributions.
Note that both f (x) and h(x) must be normalized to unit area and therefore the
proportionality constant C > 1. To generate f (x), first generate a candidate x according
to h(x). Calculate f (x) and the height of the envelope C h(x); generate u and test
if uC h(x) f (x). If so, accept x; if not reject x and try again. If we regard x and
uC h(x) as the abscissa and ordinate of a point in a two-dimensional plot, these points
June 18, 2002

13:58

32. Monte Carlo techniques 3

(a)

C h(x)

f(x)
(b)
C h(x)
f(x)

Figure 32.2: Illustration of the acceptance-rejection method. Random points are

chosen inside the upper bounding figure, and rejected if the ordinate exceeds f (x).
Lower figure illustrates importance sampling.
will populate the entire area C h(x) in a smooth manner; then we accept those which
fall under f (x). The efficiency is the ratio of areas, which must equal 1/C; therefore we
must keep C as close as possible to 1.0. Therefore we try to choose C h(x) to be as close
to f (x) as convenience dictates, as in the lower part of Fig. 32.2. This practice is called
importance sampling, because we generate more trial values of x in the region where f (x)
is most important.

32.4. Algorithms
Algorithms for generating random numbers belonging to many different distributions
are given by Press [5], Ahrens and Dieter [6], Rubinstein [7], Everett and Cashwell [8],
Devroye [9], and Walck [10]. For many distributions alternative algorithms exist, varying
in complexity, speed, and accuracy. For time-critical applications, these algorithms may
be coded in-line to remove the significant overhead often encountered in making function
calls. Variables named u are assumed to be independent and uniform on (0,1). (Hence,
u must be verified to be non-zero where relevant.)
In the examples given below, we use the notation for the variables and parameters
given in Table 30.1.

June 18, 2002

13:58

32. Monte Carlo techniques

32.4.1. Exponential decay:

This is a common application of the inverse transform method, also using the fact that
(1 u) is uniform if u is uniform. To generate decays between times t1 and t2 according
to f (t) = exp(t/ ): let r2 = exp(t2 / ) and r1 = exp(t1 / ); generate u and let
t = ln(r2 + u(r1 r2 )).

(32.4)

For (t1 , t2 ) = (0, ), we have simply t = ln u. (See also Sec. 32.4.6.)

32.4.2. Isotropic direction in 3D:
Isotropy means the density is proportional to solid angle, the differential element of
which is d = d(cos )d. Hence cos is uniform (2u1 1) and is uniform (2u2 ). For
alternative generation of sin and cos , see the next subsection.
32.4.3. Sine and cosine of random angle in 2D:
Generate u1 and u2 . Then v1 = 2u1 1 is uniform on (1,1), and v2 = u2 is uniform
on (0,1). Calculate r2 = v12 + v22 . If r2 > 1, start over. Otherwise, the sine (S) and cosine
(C) of a random angle are given by
S = 2v1 v2 /r2

and

C = (v12 v22 )/r2 .

(32.5)

32.4.4. Gaussian distribution:

If u1 and u2 are uniform on (0,1), then
z1 = sin 2u1

2 ln u2

and z2 = cos 2u1

2 ln u2

(32.6)

are independent and Gaussian distributed with mean 0 and = 1.

There are many faster variants of this basic algorithm. For example, construct
v1 = 2u1 1 and v2 = 2u2 1, which are uniform on (1,1). Calculate r2 = v12 + v22 , and
if r2 > 1 start over. If r2 < 1, it is uniform on (0,1). Then
s
z1 = v1

2 ln r2
r2

s
and z2 = v2

2 ln r2
r2

(32.7)

are independent numbers chosen from a normal distribution with mean 0 and variance 1.
zi0 = + zi distributes with mean and variance 2 .
A recent implementation of the fast algorithm of Leva Ref. 11 is in CERNLIB.
For a multivariate Gaussian, see the algorithm in Ref. 12.

June 18, 2002

13:58

32. Monte Carlo techniques 5

32.4.5. 2 (n) distribution:
For n even, generate n/2 uniform numbers ui ; then

n/2
Y
y = 2 ln
ui is 2 (n) .

(32.8)

i=1

For n odd, generate (n 1)/2 uniform numbers ui and one Gaussian z as in Sec. 32.4.4;
then

(n1)/2
Y
y = 2 ln
ui + z 2 is 2 (n) .
(32.9)
i=1

For n & 30 the much faster Gaussian approximation for the 2 may be preferable:
generate z as in Sec. 32.4.4 and use

2

y = z + 2n 1 /2; if z < 2n 1 reject and start over.

32.4.6. Gamma distribution:
All of the following algorithms are given for = 1. For 6= 1, divide the resulting
random number x by .
If k = 1 (the exponential distribution), accept x = (ln u). (See also Sec. 32.4.1.)
If 0 < k < 1, initialize with v1 = (e + k)/e (with e = 2.71828... being the natural log
base). Generate u1 , u2 . Define v2 = v1 u1 .
Case 1: v2 1. Define x = v2 . If u2 ex , accept x and stop, else restart
by generating new u1 , u2 .
Case 2: v2 > 1. Define x = ln([v1 v2 ]/k). If u2 xk1 , accept x and stop,
else restart by generating new u1 , u2 . Note that, for k < 1, the probability
density has a pole at x = 0, so that return values of zero due to underflow must
be accepted or otherwise dealt with.
1/k

Otherwise, if k > 1, initialize withpc = 3k 0.75. Generate u1 and compute

v1 = u1 (1 u1 ) and v2 = (u1 0.5) c/v1 . If x = k + v2 1 0, go back and
generate new u1 ; otherwise generate u2 and compute v3 = 64v13 u22 . If v3 1 2v22 /x
or if ln v3 2{[k 1] ln[x/(k 1)] v2 }, accept x and stop; otherwise go back and
generate new u1 .
32.4.7. Binomial distribution:
If p 1/2, iterate until a successful choice is made: begin with k = 1; compute Pk = q n
[for k 6= 1 use Pk f (rk ; n, p), and store Pk into B; generate u. If u B accept rk =
k 1 and stop; otherwise increment k by 1 and compute next Pk and add to B; generate
a new u and repeat. If we arrive at k = n + 1, stop and accept rn+1 = n. If p > 1/2 it
will be more efficient to generate r from f (r; n, q), i.e., with p and q interchanged, and
then set rk = n r.
June 18, 2002

13:58

32. Monte Carlo techniques

32.4.8. Poisson distribution:

Iterate until a successful choice is made: Begin with k = 1 and set A = 1 to start.
Generate u. Replace A with uA; if now A < exp(), where is the Poisson parameter,
accept nk = k 1 and stop. Otherwise increment k by 1, generate a new u and repeat,
always starting with the value of A left from the previous try. For large ( & 10) it may
be satisfactory (and much faster) to approximate the Poisson distribution by a Gaussian
distribution (see our Probability chapter, Sec. 30.4.3) and generate z from f (z;0,1);

then accept x = max(0, [ + z + 0.5]) where [ ] signifies the greatest integer the
expression. [13]
32.4.9. Students t distribution:
For n > 0 degrees of freedom (n not necessarily integer), generate x from a
Gaussian with mean 0 and 2 = 1 according to the method of 32.4.4. Next generate
y, an independent gamma random variate with k = n/2 degrees of freedom. Then

z = x 2n/ y is distributed as a t with n degrees of freedom.

For the special case n = 1, the Breit-Wigner distribution, generate u1 and u2 ; set
v1 = 2u1 1 and v2 = 2u2 1. If v12 + v22 1 accept z = v1 /v2 as a Breit-Wigner
distribution with unit area, center at 0.0, and FWHM 2.0. Otherwise start over. For
center M0 and FWHM , use W = z/2 + M0 .
References:
1. F. James, Comp. Phys. Comm. 79 111 (1994), based on M. L
uscher, Comp. Phys.
Comm. 79 100 (1994). This generator is available as the CERNLIB routine V115,
RANLUX.
2. G. Marsaglia, A. Zaman, and W.W. Tsang, Towards a Universal Random
Number Generator, Supercomputer Computations Research Institute, Florida State
University technical report FSU-SCRI-87-50 (1987). This generator is available as
the CERNLIB routine V113, RANMAR, by F. Carminati and F. James.
3. Much of DIEHARD is described in: G. Marsaglia, A Current View of Random
Number Generators, keynote address, Computer Science and Statistics: 16th
Symposium on the Interface, Elsevier (1985).
4. Newer generators with periods even longer than the lagged-Fibonacci based
generator are described in G. Marsaglia and A. Zaman, Some Portable Very-LongPeriod Random Number Generators, Compt. Phys. 8, 117 (1994). The Numerical
Recipes generator ran2 [W.H. Press and S.A. Teukolsky, Portable Random Number
Generators, Compt. Phys. 6, 521 (1992)] is also known to pass the DIEHARD tests.
5. W.H. Press et al., Numerical Recipes (Cambridge University Press, New York, 1986).
6. J.H. Ahrens and U. Dieter, Computing 12, 223 (1974).
7. R.Y. Rubinstein, Simulation and the Monte Carlo Method (John Wiley and Sons,
Inc., New York, 1981).

June 18, 2002

13:58

32. Monte Carlo techniques 7

8. C.J. Everett and E.D. Cashwell, A Third Monte Carlo Sampler, Los Alamos report
LA-9721-MS (1983).
9. L. Devroye, Non-Uniform Random Variate Generation (Springer-Verlag, New York,
1986).
10. Ch. Walck, Random Number Generation, University of Stockholm Physics Department Report 1987-10-20 (Vers. 3.0).
11. J.L. Leva, ACM Trans. Math. Softw. 18 449 (1992). This generator has been
implemented by F. James in the CERNLIB routine V120, RNORML.
12. F. James, Rept. on Prog. in Phys. 43, 1145 (1980).
13. This generator has been implemented by D. Drijard and K. K
olbig in the CERNLIB
routine V136, RNPSSN.

June 18, 2002

13:58