13 Generating Random

Variables for Simulation

At this point, we have discussed many discrete and continuous distributions.

This chapter shows how we can generate instances of these distributions and
others. This is helpful when performing simulations of computer systems, as in
Chapter 14. For example, we might have a computer system where the inter-
arrival times of jobs are well modeled by an Exponential distribution and the
job sizes (service requirements) are well modeled by a Pareto distribution. To
simulate the system, we need to be able to generate instances of Exponential and
Pareto random variables. This chapter presents the two basic methods used for
generating instances of random variables. Both of these methods assume that we
already have a generator of Uniform(0, 1) random variables, 1 as is provided by
most operating systems. 2

13.1 Inverse Transform Method

To generate instances of a random variable (r.v.), 𝑋, this method assumes that:

1. We know the cumulative distribution function (c.d.f.) of 𝑋, that is, we know

𝐹𝑋 (𝑥) = P {𝑋 ≤ 𝑥}.
2. We can easily invert 𝐹𝑋 (𝑥), that is, we can get 𝑥 from 𝐹𝑋 (𝑥).

The high-level plan is that we will generate a random instance of Uniform(0, 1)

(this is already available from our operating system), and then find a way to
translate that to an instance of 𝑋.

1 Actually, most operating systems provide a random integer between 1 and 𝑁 = 232 − 1. This is
easy to convert into a Uniform(0, 1) by just dividing by 𝑁 .
2 One cannot always trust the random number generator provided by one’s operating system. It is
worth reading the literature on what guarantees different random number generators provide and
on how to “seed” the random number generator [10].

Mor Harchol-Balter. Introduction to Probability for Computing,

Cambridge University Press, 2024. Not for distribution.
230 13 Generating Random Variables for Simulation

13.1.1 The Continuous Case

We assume without loss of generality that 𝑋 ranges from 0 to ∞. The method

works just as well when 𝑋 has some finite upper bound.

Idea: Let 𝑢 be our random instance from 𝑈 (0, 1). We want to map 𝑢 to 𝑥, where
𝑥 is an instance of the r.v. 𝑋. The key point is that the 𝑥 that we output needs to
be consistent with the distribution of 𝑋.

Let’s suppose there is some mapping which takes each 𝑢 and assigns it a unique
𝑥. Such a mapping is illustrated by 𝑔 −1 (·) in Figure 13.1. Here, the y-axis shows
𝑢, between 0 and 1, being mapped to an 𝑥 on the x-axis between 0 and ∞.

U(0,1)

g
1

X
0 x

Figure 13.1 Illustration of mapping 𝑔(·).

Question: Can you figure out what the mapping, 𝑔 −1 (·), should be?

Hint: Think about what property we want for our output. What should be the
probability of outputting a value between 0 and 𝑥?

Answer: A value in (0, 𝑥) should be output with probability 𝐹𝑋 (𝑥).

Question: What is the actual probability that 𝑔 −1 (·) outputs a value in (0, 𝑥)?

Answer: Because 𝑔 −1 (·) only maps values in (0, 𝑢) to values in (0, 𝑥), the
probability of outputting a value in (0, 𝑥) is the probability that the uniform
instance is in (0, 𝑢).

Question: And what is the probability that the uniform instance is in (0, 𝑢)?

Answer: 𝑢.

So we want that
𝑢 = P {0 < 𝑈 < 𝑢} = P {0 < 𝑋 < 𝑥} = 𝐹𝑋 (𝑥).

Mor Harchol-Balter. Introduction to Probability for Computing,

Cambridge University Press, 2024. Not for distribution.
 13.1 Inverse Transform Method 231

That is, we want

𝑢 = 𝐹𝑋 (𝑥) or equivalently 𝑥 = 𝐹𝑋−1 (𝑢). (13.1)

Question: So what was the 𝑔(·) function in Figure 13.1?

Answer: 𝑔(·) = 𝐹𝑋 (·), the c.d.f. of 𝑋.

Algorithm 13.1 (Inverse Transform method to generate continuous r.v. X)

1. Generate 𝑢 ∈ 𝑈 (0, 1).

2. Return 𝑥 = 𝐹𝑋−1 (𝑢). That is, return 𝑥 such that 𝐹𝑋 (𝑥) = 𝑢.

Example 13.2 Generate 𝑋 ∼ Exp(𝜆):

For the Exp(𝜆) distribution,

𝐹 (𝑥) = 1 − 𝑒 −𝜆𝑥 .

So, by (13.1) we want,

𝑥 = 𝐹 −1 (𝑢)
𝐹 (𝑥) =𝑢
1 − 𝑒 −𝜆𝑥 =𝑢
−𝜆𝑥 = ln(1 − 𝑢)
1
𝑥 = − ln(1 − 𝑢). (13.2)
𝜆

Given 𝑢 ∈ 𝑈 (0, 1), setting 𝑥 = − 𝜆1 ln(1−𝑢) produces an instance of 𝑋 ∼ Exp(𝜆).

13.1.2 The Discrete Case

The discrete case follows the same basic idea as the continuous case (see Fig-
ure 13.2). This time, we want to generate a discrete r.v., 𝑋, with the following
probability mass function (p.m.f.):


 𝑥0 w/prob 𝑝 0
w/prob 𝑝 1

 𝑥1

𝑋= .

 ...
w/prob 𝑝 𝑘

 𝑥𝑘


Mor Harchol-Balter. Introduction to Probability for Computing,

Cambridge University Press, 2024. Not for distribution.
232 13 Generating Random Variables for Simulation

U(0,1)

1
p3
p2
p1
Êp0
x0 x1 x2 x3 X
0

Figure 13.2 Generating a discrete r.v. with four values.

Algorithm 13.3 (Inverse Transform method to generate discrete r.v. X)

1. Arrange 𝑥0 , . . . , 𝑥 𝑘 , the possible values of 𝑋, s.t. 𝑥0 < 𝑥1 < . . . < 𝑥 𝑘 .

2. Generate 𝑢 ∈ 𝑈 (0, 1).
3. If 0 < 𝑢 ≤ 𝑝 0 , then output 𝑥0 .
If 𝑝 0 < 𝑢 ≤ 𝑝 0 + 𝑝 1 , then output 𝑥 1 .
If 𝑝 0 + 𝑝 1 < 𝑢 ≤ 𝑝 0 + 𝑝 1 + 𝑝 2 , then output 𝑥 2 .
Í −1 Í
If ℓ𝑖=0 𝑝 𝑖 < 𝑢 ≤ ℓ𝑖=0 𝑝 𝑖 , then output 𝑥ℓ , where 0 ≤ ℓ ≤ 𝑘.

Notice that again our 𝑔(·) function, shown in blue in Figure 13.2, is 𝐹𝑋 (·), the
c.d.f.

This sounds easy enough, but it is not always practical. If 𝑋 can take on many
Í
values, then we have to compute many partial sums: ℓ𝑖=0 𝑝 𝑖 for all 0 ≤ ℓ ≤ 𝑘.
For this method to be practical, we therefore need closed-form expressions for
Íℓ
𝑖=0 𝑝 𝑖 for all ℓ. Equivalently, we need a closed form for 𝐹𝑋 (𝑥) = P {𝑋 ≤ 𝑥} for
any 𝑥. Then we could do the same thing as in the continuous case, as in (13.1):
generate 𝑢 ∈ 𝑈 (0, 1), and set 𝑥 = 𝐹𝑋−1 (𝑢), where a ceiling may be necessary
since 𝑥 is discrete. Thus, as in the continuous case, we need to both have a
closed-form expression for the c.d.f. and also know how to invert this function.

13.2 Accept–Reject Method

The Inverse Transform method required both knowing the c.d.f., 𝐹𝑋 (𝑥), of the
r.v. 𝑋 that we’re trying to generate, and also being able to invert 𝐹𝑋 (𝑥). However,
there are many cases where we aren’t able to satisfy both of these requirements.

The Accept–Reject method has easier requirements:

Mor Harchol-Balter. Introduction to Probability for Computing,

Cambridge University Press, 2024. Not for distribution.
 13.2 Accept–Reject Method 233

1. We need the p.d.f., 𝑓 𝑋 (𝑡) (or p.m.f.) of the r.v. 𝑋 that we’re trying to generate.
2. We need to know how to generate some other r.v. 𝑌 , where 𝑌 and 𝑋 take on
the same set of values, that is,
𝑓 𝑋 (𝑡) > 0 ⇐⇒ 𝑓𝑌 (𝑡) > 0.

The Accept–Reject method is very simple. We generate an instance of 𝑌 . Then

with some probability we return that value as our instance of 𝑋, and otherwise
we reject that value and try again.

13.2.1 Discrete Case

Here’s the algorithm for a discrete r.v. 𝑋, with p.m.f. 𝑝 𝑋 (𝑖) = P {𝑋 = 𝑖}.

Algorithm 13.4 (Accept-Reject algorithm to generate discrete r.v. X)

1. Find a discrete r.v. 𝑌 , which we already know how to generate, where
𝑝𝑌 (𝑖) > 0 ⇐⇒ 𝑝 𝑋 (𝑖) > 0.
2. Let 𝑐 > 1 be the smallest constant such that
𝑝 𝑋 (𝑖)
≤ 𝑐, ∀𝑖 s.t. 𝑝 𝑋 (𝑖) > 0.
𝑝𝑌 (𝑖)
3. Generate an instance of 𝑌 . Call this instance 𝑖.
4. With probability Accept-Ratio(𝑖) = 𝑐𝑝𝑝𝑋𝑌(𝑖)
(𝑖) , accept 𝑖 and return 𝑋 = 𝑖.
Else, reject 𝑖 and return to step 3.

Question: In Step 4 of the Accept–Reject algorithm, how do we implement

accepting 𝑖 with probability Accept-Ratio(𝑖)?

Answer: We generate a Uniform(0, 1) r.v. and accept if the generated Uniform

is smaller than Accept-Ratio(𝑖).

Question: What’s the intuition behind Accept-Ratio(𝑖) in Step 4?

Answer: We can think of

𝑝 𝑋 (𝑖)
Accept-Ratio(𝑖) =
𝑐 𝑝𝑌 (𝑖)
as representing the relative likelihood of 𝑖 being an instance of 𝑋 versus 𝑌 . If
this likelihood is high, then we are more likely to trust 𝑖 as a reasonable instance
of 𝑋. If the likelihood is low, then even if 𝑖 is a common instance for 𝑌 , it is not
a common instance for 𝑋 and hence we are more likely to reject 𝑖 as an instance
of 𝑋.

Mor Harchol-Balter. Introduction to Probability for Computing,

Cambridge University Press, 2024. Not for distribution.
234 13 Generating Random Variables for Simulation

Question: What role does 𝑐 play in the accept ratio?

Answer: 𝑐 is just a normalizing constant which is needed to ensure that the

accept ratio is a probability (< 1).

Formally, we have the following argument:

Fraction of time 𝑖 is generated and accepted
= P {𝑖 is generated} · P {𝑖 is accepted given 𝑖 is generated}
𝑝 𝑋 (𝑖)
= 𝑝𝑌 (𝑖) ·
𝑐 𝑝𝑌 (𝑖)
𝑝 𝑋 (𝑖)
= . (13.3)
𝑐

Fraction of time any value is accepted

∑︁
= Fraction of time 𝑖 is generated and is accepted
𝑖
∑︁ 𝑝 𝑋 (𝑖)
=
𝑖
𝑐
1
= . (13.4)
𝑐

Combining (13.3) and (13.4), we have:

𝑝𝑋 (𝑖)
Frac. time 𝑖 is generated and accepted 𝑐
P {𝑋 is set to 𝑖} = = 1
= 𝑝 𝑋 (𝑖),
Frac. time any value is accepted 𝑐
as desired.

Question: On average, how many values of 𝑌 are generated before one is ac-
cepted?

Answer: 𝑐. Because the fraction of time any value is accepted is 𝑐1 .

13.2.2 Continuous Case

The Accept–Reject method works the same way for continuous random variables,
except that we now use the p.d.f. instead of the p.m.f.

Mor Harchol-Balter. Introduction to Probability for Computing,

Cambridge University Press, 2024. Not for distribution.
 13.2 Accept–Reject Method 235

Algorithm 13.5 (Accept-Reject algorithm to generate continuous r.v. X)

1. Find a continuous r.v. 𝑌 , which we already know how to generate, where

𝑓𝑌 (𝑡) > 0 ⇐⇒ 𝑓 𝑋 (𝑡) > 0.
2. Let 𝑐 > 1 be the smallest constant such that
𝑓 𝑋 (𝑡)
≤ 𝑐, ∀𝑡 s.t. 𝑓 𝑋 (𝑡) > 0.
𝑓𝑌 (𝑡)
3. Generate an instance of 𝑌 . Call this instance 𝑡.
4. With probability Accept-Ratio(𝑡) = 𝑐·𝑓 𝑋𝑓𝑌(𝑡(𝑡) ) , accept 𝑡 and return 𝑋 = 𝑡.
Else, reject 𝑡 and return to step 3.

Similarly to the Accept–Reject algorithm for the discrete case, we can show that:
Density of returning 𝑡 on an iteration = Density of generating 𝑡 · P {accept 𝑡}
𝑓 𝑋 (𝑡)
= 𝑓𝑌 (𝑡) ·
𝑐 · 𝑓𝑌 (𝑡)
1
= 𝑓 𝑋 (𝑡) · .
𝑐
Hence,
∫
1 1
P {Return some value on a given iteration} = 𝑓 𝑋 (𝑡) · 𝑑𝑡 = ,
𝑡 𝑐 𝑐
so the expected number of iterations needed to get an instance of 𝑋 is 𝑐.

Example 13.6 Generate r.v. 𝑋 with p.d.f. 𝑓 𝑋 (𝑡) = 20𝑡 (1 − 𝑡) 3 , 0 < 𝑡 < 1.

If you plot 𝑓 𝑋 (𝑡), it looks like Figure 13.3. Observe that 𝑋 has positive p.d.f.
only in the interval (0, 1). Thus we want to choose a 𝑌 that is easy to generate
and also has positive p.d.f. only in (0, 1).

Question: Any ideas for what 𝑓𝑌 (𝑡) should be?

Answer: Consider simply 𝑓𝑌 (𝑡) = 1, where 0 < 𝑡 < 1.

Question: Suppose we now apply the Accept–Reject method. What will 𝑐 be?

Answer: Based on the plot, 𝑐 should not be too bad – just over 2. To determine
𝑐 precisely, we want to determine
 
𝑓 𝑋 (𝑡)
= max 20𝑡 (1 − 𝑡) 3 .

max
𝑡 𝑓𝑌 (𝑡) 𝑡

Mor Harchol-Balter. Introduction to Probability for Computing,

Cambridge University Press, 2024. Not for distribution.
236 13 Generating Random Variables for Simulation

f X (t)

2.5
2.0
1.5
1.0
0.5
t
0 0.2 0.4 0.6 0.8 1

Figure 13.3 Plot of 𝑓 𝑋 (𝑡).

Taking the derivative with respect to 𝑡, and setting it equal to zero, we have
𝑑 1
(20𝑡 (1 − 𝑡) 3 ) = 0 ⇐⇒ 𝑡 = .
𝑑𝑡 4

So the maximum value is obtained when 𝑡 = 14 :

 
𝑓 𝑋 14    3
1 3 135
𝑐=   = 20 = . (13.5)
𝑓𝑌 14 4 4 64

Observe how easy it was to make a good guess for 𝑓𝑌 (𝑡) just by looking at the
plot of 𝑓 𝑋 (𝑡).

Question: Could we have used the Inverse Transform method to generate 𝑋?

Answer: No. While it is easy to get 𝐹𝑋 (𝑥), unfortunately 𝐹𝑋 (𝑥) is not easy to
invert. Thus we won’t be able to solve 𝑢 = 𝐹𝑋 (𝑥) for 𝑥.

Example 13.7 (Generating a Normal r.v.)

We now turn to generating the Normal distribution. By the Linear Transformation

Property (Theorem 9.5), it suffices to generate a standard Normal. Here it’s clearly
impossible to use the Inverse Transform method since we don’t know the c.d.f.

Goal: Generate 𝑁 ∼ Normal(0, 1).

Idea: It suffices to generate 𝑋 = |𝑁 | and then multiply 𝑋 by −1 with probability

0.5.

Mor Harchol-Balter. Introduction to Probability for Computing,

Cambridge University Press, 2024. Not for distribution.
 13.2 Accept–Reject Method 237

So how do we generate such an 𝑋? A plot of 𝑋 is shown in Figure 13.4.

2 𝑡2
𝑓 𝑋 (𝑡) = √ 𝑒 − 2 , 0 < 𝑡 < ∞.
2𝜋

Question: What is a good choice for a r.v. 𝑌 that we know how to generate, such
that 𝑓𝑌 (𝑡) fits 𝑓 𝑋 (𝑡) reasonably well?

Answer: Let 𝑌 ∼ Exp(1).

𝑓𝑌 (𝑡) = 𝑒 −𝑡 , 0 < 𝑡 < ∞.
Observe that 𝑓 𝑋 (𝑡) is not too much higher than 𝑓𝑌 (𝑡), according to Figure 13.4.

0.8
f X (t)
0.6
f Y (t)
0.4

0.2
t
0 2 4 6 8 10

Figure 13.4 Solid line shows 𝑓 𝑋 (𝑡). Dashed line shows proposed 𝑓𝑌 (𝑡).

Question: How many iterations are needed on average?

Answer: We need to determine 𝑐.

√︂
𝑓 𝑋 (𝑡) 2 𝑡2 2 𝑡 − 𝑡2
= √ 𝑒 − 2 +𝑡 = 𝑒 2.
𝑓𝑌 (𝑡) 2𝜋 𝜋

2
So, the maximum value occurs when 𝑡 − 𝑡2 is maximized.
𝑡2
 
𝑑
0= 𝑡− = 1 − 𝑡 =⇒ 𝑡 = 1.
𝑑𝑡 2

So,
√︂
𝑓 𝑋 (1) 2𝑒
𝑐= = ≈ 1.3.
𝑓𝑌 (1) 𝜋

Thus we only need 1.3 iterations on average!

Mor Harchol-Balter. Introduction to Probability for Computing,

Cambridge University Press, 2024. Not for distribution.
238 13 Generating Random Variables for Simulation

13.2.3 A Harder Problem

𝑒 −𝜆 𝜆𝑖
Consider 𝑋 ∼ Poisson(𝜆), with 𝑝 𝑋 (𝑖) = 𝑖! .

Question: Can we use the Inverse Transform method to generate an instance of

a Poisson r.v.?

Answer: There is no closed form for 𝐹 (𝑥) = P {𝑋 ≤ 𝑥} so the Inverse Transform

method will not work.

Question: Can we use the Accept–Reject method?

Answer: It looks like we should be able to apply the Accept–Reject method, but
it is hard to find the right 𝑌 distribution to match up to (see [48, p. 503]).

We will show a different way to generate 𝑋 ∼ Poisson(𝜆) in Exercise 13.5 by

relating 𝑋 to a Poisson process with rate 𝜆.

13.3 Readings

A lot more is known about generating random variables than we have described
in this chapter. Some particularly well-written texts are [63] and [48].

13.4 Exercises

13.1 Generating random variables for simulation – (from [63])

Give an algorithm for generating 𝑋 with p.d.f. 𝑓 𝑋 (𝑥) = 30(𝑥 2 − 2𝑥 3 + 𝑥 4 )
where 0 < 𝑥 < 1.

13.2 Inverse Transform method

Provide a simple algorithm for generating values from a continuous distri-
bution with p.d.f. 𝑓 (𝑡) = 54 𝑡 −2 , where 1 < 𝑡 < 5.

13.3 Generating a Geometric distribution

Give a simple and efficient algorithm for generating values from a
Geometric( 𝑝) distribution. Now use your algorithm to generate 50 in-
stances of Geometric(0.2). Determine the sample mean (the average of
the 50 generated instances). Compare the sample mean with the desired
answer.

Mor Harchol-Balter. Introduction to Probability for Computing,

Cambridge University Press, 2024. Not for distribution.
 13.4 Exercises 239

13.4 Simulation of heavy-tailed distributions

Write a short program to generate 100 instances of
𝑋 ∼ BoundedPareto(𝑘 = 332.067, 𝑝 = 1010 , 𝛼 = 1.1),
as defined in Definition 10.5. Take their average. Record your answer. Now
generate 1000 instances of this distribution, and again take their average
and record your answer. Keep going. This time, generate 10,000 instances
of this distribution and take their average and record it. Next, generate
100,000 instances. Keep going until you run out of patience. You should
find that your sample averages are well below the true average (compute
this!). Explain why this is. What trend do you see in your sample averages?

13.5 Generating a Poisson r.v.

Describe an efficient algorithm for generating instances of a Poisson r.v.
with mean 1. It will be helpful to start by recalling what you learned in
Chapter 12 about the Poisson process and where the Poisson distribution
comes up in the context of a Poisson process.

13.6 Simulating jointly distributed random variables

Let 𝑋 and 𝑌 be non-negative, continuous random variables whose joint
density is given by:
𝑓 𝑋,𝑌 (𝑥, 𝑦) = 𝜆𝑒 −𝜆𝑥 𝑥𝑒 −𝑥 𝑦 , 𝑥 ≥ 0, 𝑦 ≥ 0.
Provide a simple algorithm, using the Inverse Transform method, that
generates a point (𝑥, 𝑦) drawn from the above joint p.d.f. Explain your
reasoning.

Mor Harchol-Balter. Introduction to Probability for Computing,

Cambridge University Press, 2024. Not for distribution.