Devroye Random Variate Generation One Line of Code
Devroye Random Variate Generation One Line of Code
Luc Devroye
School of Computer Science
McGill University
Montreal, Canada H3A 2A7
[email protected]
ABSTRACT Alternatively,
265
266
n ~1 C:) xn
k
~n k
= e (n k 3 2 n
- / 4 ) .
arctan( x ) /7r.
Other notable examples include the logistic
(F{x) == 1/(1 + e- X )) which can be obtained as
This limits the numbers of families we may construct ~ 10g(U/(1 - U)). (U-1/a - l)l/c yields a Burr XII
using the basic operations. Still, by making n even random variate (F(x) == -l/(xC + l)a (x > 0)), and
moderately large, the possibilities are virtually un- (U-l/a - 1)-l/c yields a Burr III random variate
limited. We will take a little tour of the popular dis- (F(x) == l/{x- C + l)a (x > 0)). A Frechet or Weibull
tributions and exhibit a number of (mostly known) random variate with F(x) == 1 - e- xG (x > 0) may
one-liners. Notable exceptions are the gamma and be obtained as logl/a(l/U). For the Gumbel distribu-
z
Poisson distributions. tion (F(x) == e- ae - ), we suggest -log((log(l/U)/a)).
A Pareto or Pearson XI (F(x) == a/x a + 1 (x ~ 1)) may
2 NOTATION be obtained by u-l/a. A tail of the Rayleigh distribu-
2 2
tion has distribution function F(x) = 1- e~ (x ~
We give different symbols for different random vari- a > 0), so that random variates may be obtained
ables. For example, U is uniform [0,1], N is stan- J
as a 2 - 2 log U. The hyperbolic secant distribution
dard normal (with density e- /2 / ~), A is arc-
x2 function is F(x) == 1- (2/1r) arctan(e- 1rx / 2 ). Random
sine (with density 1/(7rV1 - x 2 ) on [-1,1]), B a,b is variates may be obtained as (2/1r) log tan ('rrU /2).
beta (a, b) (with density x a - l (l - x)b-l / B(a, b) on
[0,1] where a, b > 0), Ga is gamma (a) (with den-
sity xa-le- x /r(a) on [0,00), where a > 0), E is ex-
ponential (with density e- x , x > 0), L is Laplace
Random Variate Generation in One Line of Code 267
4 COMBINATIONS OF TWO OR MORE which is the Rayleigh density (so that R == J -210g(U)
RANDOM VARIABLES has density f when U is uniform [0,1]), then the den-
sity of RA is
Mixtures of the form X == Y with probability p and
X == Z with probability 1 - p are easily taken care of e-x2 /2 e - y 2 /2 1
1
(X) 2
x == m + (a + U1(b - a) - m) max(U2 , U3 ) .
h(x) ==
l o
2c(1 - X
2
)C-l/2
7f
1 1
dy
Here all the Ui's are i.i.d. uniform [0, 1] random vari- == x (1 - U 2 )C-l du
ables. 1r 0
2c(1- X 2 )C-l/2 r
22C - 1 2 (c)
== x ----
5 THE POLAR METHOD 1r r(2c)
2
r(2c+ 1)(1_x )C-l/2
In the standard polar method, one generates a ran-
22c r 2 (c + 1/2)
dom pair (X, Y) as (Rcos8,Rsin8), where Rand in which we recognize a shifted version of the sym-
8 are (random) polar coordinates. Typically, 8 is metric beta density. In the last step, we used a prop-
uniformly distributed on [0, 21r], and R has a given erty due to Binet (property 23 on page 261 of Whit-
distribution that is easy to sample from. Often, R taker and Watson, 1927). The last density is that of
is independent of 8. In the context of one-liners, we 2Bc + I / 2 ,c+I/2 - 1. Thus, our one-liner for symmetric
thus have betas with parameter a > 1/2 is based on
X == R cos(21rU)
where U is uniform [0, 1]. Another way of writing this Ba,a ==
£. 1+ VI - U~ cOS(21rU2)
2 .
is X == RA, where A is a random variable on [-1,1]
with the arcsine density (note: our arcsine density is This is Ulrich's formula (Ulrich, 1984). For a == 1/2,
in fact a linear transformation of the standard arc- note B I / 2 ,1/2 £: (1 + cOS(21rU2))/2 == cOS2(rrU2). Just
sine density 2/(1rJz(1 - z)), 0 < z < 1). A simple as for the normal distribution, equivalent respresen-
exercise in analysis shows that if R and A are inde- tations are easy to construct. We note here that
pendent, and R has density f on [0,00), then X == RA the density h plays a central role in nonparametric
has density estimation theory. The case c == 3/2 leads to the
Epanechnikov or Bartlett density. The case c == 5/2
is usually referred to as the quartic kernel. Note that
Ulrich's formula is not valid for a < 1/2. A fixed one-
liner for the entire symmetric beta family is given in
EXAMPLE 1: THE NORMAL DENSITY. If the next section.
2
f( r ) == re - r /2 , r > 0 ,
268 Devro.ye
EXAMPLE 3: THE HALF-BETA DISTRIBUTION. The This method for t-variates was pointed out by Bai-
half-beta density is the density of B 1 / 2 ,a. If X has ley (1994). This could also have been obtained by
density h from the previous section, then on [0,1], the method of the previous section, but the method
Y == X 2 has density above requires less integration work. We do not make
claims that this is the fastest method for generating
r(2c + 1)(1 - y)C-l/2 r(c + 1)(1 - y)C-l/2 t variates. There is another interesting observation
22c r 2 (c + 1/2)VY == r(I/2)r(c + 1/2)VY ' here: the t density is a mixture of bimodal densities
which is the density of B I / 2 ,c+l/2. Thus, for a 2: 1/2, wi th infinite peaks (arcsine densities)!
B 1 / 2 ,a I= (1 - 2
U 12 0 - 1
) cos 2 (21l"U2 ) .
ANOTHER ONE-LINER FOR SYMMETRIC BETAS.
(1978) has proved that
Best
6 THE POLAR METHOD AND NORMAL This (new) method is applicable for all values of a-
SCALE MIXTURES Ulrich's formula required a > 1/2.
standard normal, and Y is an arbitrary positive ran- another one-liner for halfbetas:
dom variable, independent of N. If N I , N 2 are two B £ Tia 1
independent normals independent of Y, then 1/2,a - 2a + Tia - 1 + 2a/Tia .
(X I ,X2) ~f (YN1 , YN2) £:. (ZI,Z2) x (YJ2E) , Thus, if S denotes a random sign (3 == IV~1/2 for V
uniform [0,1]), the following distributional identity
where E is exponential, and (Zl, Z2) is uniform on
yields a one-liner for all halfbeta distributions:
the unit circle and independent of E and Y. This
says that (X1 ,X2 ) has a radially symmetric distribu-
tion with random radius distributed as Y V2E. Gen-
erating Xl based upon this formula might be called
a polar method for X. In some cases, the distribu- This (new) method is applicable for all values of a.
tion of Y V2E is very simple. For example, if Y £:.
Ja/2G a / 2 , then simultaneously 7 DISTRIBUTIONS DEFINED AS
ONE-LINERS
yJ2E = VGaE a/2
; YN f=. T a .
the model. There are literally hundreds of such at- with parameter a E [1,2]:
where U is uniform [0,1], b' 2: 0, b+b' 2: 0, and either Not in partcular that if E is exponential, S is a ran-
°
c + c' > or c + b' == c' == O.
Burr (1942) (see Tadikamalla, 1980), Johnson
dom sign, L is Laplace, C is Cauchy, and N 1 , N 2 are
i.i.d. normal random variates, then
(1949) and many others since then have invented their
own families of distributions based on this convenient L 1 £. EN
N
1 .
'
L
1
£. EC ., L 1 ==L C;
E L 2 ==
£. SE C L
==.
2
principle. Tadikamalla (1980) reviews many systems.
For example, in the Tadikamalla-Johnson system
Tadikamalla and Johnson (1990), we begin with a THE STABLE LAWS. Ibragimov and Chernin (1952)
logistic random variate and Zolotarev (1966, 1986) derived various useful rep-
resentations for the stable laws. As an example, de-
U
Z == l o g - - fine K(a) == a - 2Ia >1, and let Ga,b be the distribu-
I- U '
tion function for Sa,b. Set () == bK(a)/a. Note that
and define three random variables: for x > 0, b > 0,
YL == ~ + Ae(Z---y)/fJ
1 - -21 fl-(J e -x ~ U (z (J) dZ
,
Q I (a > 1)
YB = ~ + AI (1 + e-(Z-I')/6) ,
Ga,b == -1-6 + -1 fl e -x~U (z6)d Z Q, (a < 1) .
2 2-6
Yu == ~ + Asinh((Z - ,)/b) . ! JI e-e-:l:/bU 1 (z,b)
2 -1
dz (a == 1)
Just as with the Johnson family, the family covers the The values for x < 0 are obtained by noting that
entire skewness-kurtosis plane. And the one-liners are Ga,b(X) + Ga,-b( -x) == 1. The values for b < 0 are
fixed as well for the family.
obtained by noting that Ba,b £. -Sa,-b. The func-
tions U are defined as follows:
8 REPRESENTATION THEOREMS
U ( b) == 7r(I + bz) e~(z+l/b) tan( T) .
1 Z, 2 cos (1rZ)
2 ,
Somethnes densities, distribution functions or charac-
teristic functions can be written as integrals, which,
. (1ra(Z+6)))
Sin 2
~ cos (7r«a-l)z+a(J))
2
upon closer inspection, reveal some method for gen- Ua ( Z , (J) = ( COS ( "2z ) COS (,,2")
erating random variates. Two examples follow that
lead to useful one-liners.
All Sa,b'S are supported on the real line except Sa,l variables for which one-liners are already known. The
for a < 1 (which is supported on the positive halfline) following are prime candidates: Y == U a , Y is normal,
and Sa,-l for a < 1 (which is supported on the neg- Y is exponential, and Y is Cauchy. At this junction,
ative halfline). Another representation is that the it is impossible to be exhaustive. We will rather limit
distribution of S::l(a-l) for a < 1 is given by ourselves to a few nice examples.
For a < 1, this would suffice for all values of b E Three main examples come to mind:
[-1,1] as A. If X is exponential (thus, L(s) == 1/(1 + s)),
L
Sa,b ==
(1 + b)
-2-
l/a
Sa,l +
(1 - b)
-2-
1/a
Sa,-1
then N JX/2 is Laplace, as it has characteristic
function 1/{1 + t 2 ).
B. Assume that 0 < Q < 1 and set X == So:,l. From
(Zolotarev, 1986, p. 61). Defining () as above and Zo-lo-ta-rev (1986, p. 112), we know that for the
sOl
positive stable distribution, £( s) == e- if s 2:
sin (~) (cos ('If((a-~)z+a/l»)) ~ J
O. Therefore, N 80:,1/ 2 has characteristic func-
Ba(z) == ( 1r«a-l)z+aB) ) cos (1rz) tion e-ltI2a. That is,
cos 2 2
J
N So:, 1 /2 ~ S20:,0. Symmetric stables can be
Chambers, Mallows and Stuck (1976) (see also Zolota- built up from positive stables and normals. For
rev, 1986) suggest the extended one-liner the latter distributions, one-liners were exhib-
ited aerlier in the paper.
Sa,b ~ Ba(U - 1/2)E 1 - 1
/
a
C. If we take in the previous example a == 1, then
valid for all a =1= 1 and b E [-1, 1]. For a == 1, they we note that N JSl,I/2 has characteristic func-
obtain tion
L 2b
8 1 ,b == B l (U -1/2) - -logE,
7r where we used the fact that the two-sided La-
place transform of SI,l is e- s log S for s > 0 (Zo-
where
lo-ta-rev, 1986, p. 112).
2b
B I (z) == -log
7r
(1
cos
+(bz) ) + (1 + bz) tan ( -7r Z )
'1r
2
Z 2
.
CAUCHY SCALE MIXTURES. Let C be standard Cauchy
and let X be a positive random variable with two-
sided La-place transform L(8). Then Y == ex has
9 SCALE MIXTURES characteristic function L(ltl). This is easily see by
noting that
We say that X is a scale mixture if X == Y Z can Ee itY == Ee itXN
be decomposed as the product of two independent = Ee- 1t1X (condition on X)
random variables Y and Z. Such mixtures are con-
== L(ltl) .
venient ways of trying to discover one-liners. Famous
scale mixtures occur when Y is uniform [0,1]. In that Three examples follow:
case, the distribution of X is unimodal with a peak A. Since G a has two-sided La-place transform £(s) =
at the origin, and the mixture is called a Khinchine 1/{1 + s)a, valid for ~(s) 2: 0, eGa has charac-
mixture. One should try replacing Y with all random teristic function 1/(1 + It\)a.
Random Variate Generation in One Line of Code 271
°
B. Assume that < Q < 1. Arguing as in the pre- yields a random variable X with characteristic func-
vious subsection, we note that So., 1 C £. So.,o,
C (~it)a r'
tion
Thus, symmetric stables can also be obtained
from Cauchy variables and positive extreme sta- cp(t) = + a E (0,1], b > 0 .
bles.
C. Finally, S 1,1 C has characteristic function This distribution was studied by Pillai (1990). A re-
e-Itllog Itl. lated distribution was studied by Klebanov, 11aniya
and Melamed (1984). For b =: 1, we obtain the
SYMMETRIC STABLE MIXTURES. Scale mixtures with Mittag-Leffler distribution with parameter a. Note
stable distributions are best studied via characteristic that the stable (a, 1) random variate mentioned here
functions. In particular, if X has density j, then the _ has characteristic function
characteristic function of XC Sa,O is (t > 0)
cp(t) = EeitXCSo,o = Ee-ltXClo = J !(x)e-IWlxlco dx .
(t < 0) .
Using Kanter's one-liner for Sa,l, we see that if E, E*
This is particularly helpful if j(x) has a factor e- lx1b . are i.i.d. exponential random variables and U is uni-
formly distributed on [0,1], then
For example, if f(x) == e- If(l + lib) , x > 0, then,
xb
N. L. Johnson, "Systems of frequency curves gen- P. R. Tadikamalla, "A look at the Burr and re-
erated by methods of translation," Biometrika, lated distributions," International Statistical Re-
vol. 36, pp. 149-176, 1949. vie ,v, vol. 48, pp. 337-344, 1980.
N. L. Johnson, "Systems of frequency curves de- P. R. Tadikamalla , "Random sampling from the ex-
rived from the first law of Laplace," Trabajos de Es- ponential power distribution," Journal of the Amer-
tadistica, vol. 5, pp. 283-291, 1954. ican Statistical Association, vol. 75, pp. 683-686,
N. L. Johnson and S. Kotz, "Encyclopedia of the Sta- 1980.
tistical Sciences," vols. 1-4 (1983), vols. 5-6 (1985), P. R. Tadikamalla, "On simulating non-normal dis-
vol. 7 (1986), vols. 8-9 (1988), supplement (1989), tributions," Psychometrika, vol. 45, pp. 273-279,
John Wiley, New York, 1980.
N. L. Johnson and S. Katz, "Extended and multi- P. R. Tadikamalla and N. L. Johnson, "Tables to fa-
variate Thkey lambda distributions," Biometrika, cilitate fitting Tadikamalla and Johnson's L B dis-
vol. 60, pp. 655-661, 1973. tributions," Communications in Statistics, Series
M. Kanter, "Stable densities under change of scale Simulation, vol. 19, pp. 1201-1229,1990.
and total variation inequalities," Annals of Proba- J. W. Tukey, "The practical relationship between the
bility, vol. 3, pp. 697-707, 1975. common transformations of percentages of counts
T. Kawata, Fourier Analysis in Probability The- and of amounts," Technical Report 36, Statisti-
ory, Acadenlic Press, New York, N.Y., 1972. cal Techniques Research Group, Princeton Univer-
L. B. Klebanov, G. V. Maniya, and I. A. Melamed, sity, 1960.
"A problem of Zolotarev and analogs of infinitely G. Ulrich, "Computer generation of distributions on
divisible and stable distributions in a scheme for the m-sphere," Applied Statistics, vol. 33, pp. 158-
summing a random number of random variables," 163, 1984.
Theory of Probability and its Applications, vol. 29, E. T. Whittaker and G. N. Watson, A Course
pp. 791-794, 1984. of Modern Analysis, Cambridge University Press,
R. G. Laha, "On a class of unimodal distribu- Cambridge, 1927 (1980).
tions," Proceedings of the American Mathemati- V. M. Zolotarev, "On the representation of stable
cal Society, vol. 12, pp. 181-184,1961. laws by integrals," Selected Translations in Math-
R. G. Laha and E. Lukacs, "On a problem connected ematical Statistics and Probability, vol. 6, pp. 84-
with quadratic regression," Biometrika, vol. 47, 88, 1966.
pp. 335-343, 1960. V. M. Zolotarev, One-Dimensional Stable Distri-
Yu. V. Linnik, "Linear forms and statistical cri- butions, American Mathematical Society, Provi-
teria: I, II," Selected Translations in Matbemati- dence, R.I., 1986.
cal Statistics and Probability, vol. 3, pp. 1-40, 41-
90, 1962.
AUTHOR BIOGRAPHY
E. Lukacs, Characteristic Functions, Griffin, Lon-
don, 1970.
R. N. Pillai, "Semi-a Laplace distributions,n Com- LUC DEVROYE is professor in the School of
munications in Statistics - Theory and Meth- Computer Science at McGill University in Mon-
ods, vol. 14, pp. 991-1000, 1985. treal, Canada. His research interests include simu-
I. Popescu, "Algorithms for generating the logistic lation, random number generation, and the proba-
variable," Economic Computation and &onomic bilistic analysis of algorithms. His 1986 book Non-
Cybernetics Studies and Research, vol. 10(1), pp. 55- Uniform Random Variate Generation (now out of
61, 1976. print) may be consulted as background for the present
J. S. Ramberg and B. W. Schmeiser, "An approx- article. Although he has tried hard, he still does
imate method for generating asymmetric random not know how to generate a gamma random vari-
variables," Communications of the ACM, vol. 17, ate in one line of code.
pp. 78-82, 1974.
B. W. Schmeiser and S. J. Deutch, "A versa-
tile four parameter family of probability distri-
butions suitable for simulation," AIEE Transac-
tions, vol. 9, pp. 176-182,1977.