STAT 32562

Statistical Simulations
 Random Variates

 A random variate is a variable generated from pseudorandom

numbers.

 Random variates are frequently used as the input to simulation

models.

 In modern applications, such simulations would derive random

variates corresponding to any given probability distribution from
computer procedures.
 Generation of Random Variates

 Procedures to generate random variates corresponding to a

given distribution are known as procedures for random variate
generation or pseudo-random number sampling.

 Different techniques are available for generating random

variates.
Techniques in Generating Random Variates

 Inverse transform technique

 Acceptance/rejection method

 Composition method

 Convolution method
Inverse Transformation Technique

 The inverse-transform technique can be used in principle for

any distribution.

 Most useful when the CDF F(x) has an inverse F -1(x) which is
easy to compute.
Inverse Transformation Technique cont…

 For a given probability density function, cumulative density

function is calculated as
𝑓(𝑥) → 𝐹(𝑥) = 𝑃(𝑋 ≤ 𝑥)

 Set 𝐹 𝑥 = 𝑈 where 𝑈 ~ 𝑢𝑛𝑖 0,1 and solve for 𝑥.

−1
𝑥= 𝐹 (𝑈)
Inverse Transformation Technique cont…
Inverse Transformation Technique contd...

Required steps

1. Compute the CDF of the desired random variable X

2. Set 𝐹(𝑋) = 𝑢 on the range of X

3. Solve the equation 𝐹(𝑋) = 𝑢 for X in terms of u. 𝑋 = 𝐹 −1 (𝑢)

4. Generate uniform random numbers u1 , u2 , u3 , ... and compute the

desired random variate by 𝑋𝑖 = 𝐹 −1 (𝑢𝑖)
Inverse Transformation Technique contd...

Example:
Inverse Transformation Technique contd...

Example cont…
Consider the case a = 1, The cdf for x > 0 is
F (x) = 1 - exp(-x).
Two random variates has been generated using Equation 2.

The first r.n. generated is u1 = 0.7505 and the corresponding x is

x1 = - ln(1 - 0.7505) = 1. 3883

Similarly, the random number u2 = 0.1449 generates the

exponential variate x2 = - ln(1 - 0.1449) = 0.156 54
Inverse Transformation Technique contd...

Example:

1. Determine an algorithm for the generation of a random

variable with probability density function.

𝑥 Τ2 , 0≤𝑥<1
1Τ2 , 1≤𝑥<2
𝑓 𝑥 =
(3 − 𝑥)Τ2 , 2≤𝑥<3
0 , 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
Inverse Transformation Technique contd...

Answer:

• 𝑓(𝑥) has 3 pieces so we need 3 pieces for 𝐹(𝑥).

For 0 ≤ 𝑥 < 1,
𝑥
𝑡 𝑥2
𝐹 𝑥 = 𝐹1 𝑥 = න 𝑑𝑡 =
0 2 4
Inverse Transformation Technique contd...
For 1 ≤ 𝑥 < 2,
𝑥
1
𝐹 𝑥 = 𝐹2 𝑥 = 𝐹 1 + න 𝑑𝑡
1 2
1 (𝑥 − 1)
= +
4 2

For 2 ≤ 𝑥 < 3
𝑥
3−𝑡
𝐹 𝑥 = 𝐹3 𝑥 = 𝐹 2 + න 𝑑𝑡
2 2
3−𝑥 2
=1−
4
Inverse Transformation Technique contd...

Algorithm: First generate 𝑈 ~ 𝑈𝑛𝑖(0,1)

• If 𝑈 < 1/4,

Invert 𝐹1(𝑥) = 𝑈, and set RV 𝑋 = 4𝑈 = 2 𝑈

3
• elseif 𝑈 < ,
4
1 1
Invert 𝐹2 𝑥 = 𝑈, and set RV 𝑋 = 1 + 2 𝑈 − = 2𝑈 +
4 2

• else,

Invert 𝐹3 𝑥 = 𝑈, and set RV 𝑋 = 3 − 4 1 − 𝑈 = 3 − 2 1 − 𝑈

Inverse Transformation Technique contd...

Example:

2. Determine an algorithm for the generation of a random variable

with probability density function.

0.1 , 𝑥=1
𝑓 𝑥 = ቐ0.3 , 𝑥=2
0.6 , 𝑥=3
Inverse Transformation Technique contd...

Answer:
0, 𝑥<0
0.1, 1≤𝑥<2
𝐹 𝑥 =
0.4, 2≤𝑥<3
1.0, 3≤𝑥

If 0 ≤ 𝑈𝑖 ≤ 0.1, then 𝑥𝑖 = 1
elseIf 0.1 ≤ 𝑈𝑖 ≤ 0.4, then 𝑥𝑖 = 2
Else 0.4 ≤ 𝑈𝑖 ≤ 1.0, then 𝑥𝑖 = 3
Inverse Transformation Technique contd...

Random Variates for Discrete Distribution.

(01). Discrete Uniform Distribution

• Suppose that 𝑆 is some finite set of real numbers. Then a

random variable 𝑥 has a discrete uniform distribution over 𝑆.
• The probability 𝑃(𝑥) are all equal for each 𝑥𝜖𝑆 . i.e. if
𝑆 contains n number of different possible values for 𝑥, then
𝑃(𝑥) = 1/𝑛 for each value.
Inverse Transformation Technique contd...

Algorithm:

• Generate a random number 𝑈

X 1 2 3 ……….. j -1 j …..…… N

𝑃(𝑋 = 𝑥) 1/n 1/n 1/n ……….. 1/n 1/n ……….. 1/n

𝐹𝑥 (𝑋) 1/n 2/n 3/n ……….. (j -1)/n j/n ……….. 1

Inverse Transformation Technique contd...

If 𝐹 𝑥 − 1 < 𝑢 < 𝐹(𝑥) 𝑥= 𝑗

𝑗−1 𝑗
<𝑢<
𝑛 𝑛
𝑗 − 1 < 𝑛𝑢 < 𝑗
𝑥= 𝑗
𝑥 = ⌈𝑛𝑢⌉ or 𝑥 = 𝑛𝑢 + 1

𝑥 = smallest integer value greater than 𝑥

𝑥 =largest integer value smaller than 𝑥
Example: 3.6 = 4

3.6 = 3
Inverse Transformation Technique contd...

(02). Bernoulli Distribution

Suppose we want to generate a Bernoulli (𝑝) random variate X, in which

case 𝑃(𝑋 = 0) = 1 − 𝑝 and 𝑃(𝑋 = 1) = 𝑝 for some 𝑝 ∈ (0, 1).
Then the discrete inverse-transform method yields:

Algorithm for generating a Bernoulli (𝑝) random variate 𝑋:

i. Generate 𝑈 ∼ 𝑢𝑛𝑖(0, 1).
ii. Set 𝑋 = 0 if 𝑈 ≤ 1 − 𝑝;
𝑋 = 1 if 𝑈 > 1 − 𝑝.
Inverse Transformation Technique contd...

(03). Binomial Distribution

Suppose 𝑥 follows a binomial distribution with parameters,

𝑥 ~ 𝐵𝑖𝑛 𝑛, 𝑝

𝑃 𝑋=𝑥 = 𝑛𝐶 𝑃 𝑥 1−𝑃 𝑛 _𝑥
𝑥

= 𝑛!
𝑥 𝑛_𝑥
𝑥! 𝑛−𝑥 ! 𝑃 1−𝑃
Inverse Transformation Technique contd...

We can note that 𝑋 can be represented (in distribution) as the sum of

𝑛 𝑖𝑖𝑑 Bernoulli (𝑝) Random variates, 𝑌1, . . . , 𝑌𝑛;

𝑋 = ෍ 𝑌𝑖 ,
𝑖=1

the number of successes out of 𝑛 𝑖𝑖𝑑 Bernoulli (𝑝) trials.

Inverse Transformation Technique contd...

Alternative algorithm for generating a binomial (𝑛, 𝑝) rv 𝑋:

I. Generate 𝑛 𝑖𝑖𝑑 random variates 𝑈1, 𝑈2, . . . , 𝑈𝑛 ∼ 𝑢𝑛𝑖𝑓(0, 1).

II. For each 1 ≤ 𝑖 ≤ 𝑛, set 𝑌𝑖 = 0 if 𝑈𝑖 ≤ 1 − 𝑝;
𝑌𝑖 = 1 if 𝑈𝑖 > 1 − 𝑝.
(This yields 𝑛 𝑖𝑖𝑑 Bernoulli (𝑝) random variates .)
III. Set 𝑋 = σ𝑛𝑖=1 𝑌𝑖 .
Inverse Transformation Technique contd...

Example:
X ∼ Bin(4, 0.25). The possible values of X are xi = i, i = 0, ..., 4, and
the values of F are given in the Table .
Suppose U = 0.6122 is a given random number.
Looking along the rows of F (xi) values, we see that
F (x0) = 0.3164 < U = 0.6122 < F (x1) = 0.7383.
Thus x1 is the first xi such that U ≤ F (xi); therefore X = 1
Inverse Transformation Technique contd...

Random Variates for Empirical Continuous Distribution

Inverse Transformation Technique contd...

Random Variates for Empirical Continuous Distribution Example

Inverse Transformation Technique contd...

Random Variates for Empirical Continuous Distribution

Inverse Transformation Technique contd...
Random Variates for Empirical Continuous Distribution Example:
Suppose the data collected for 100 broken-widget repair times are as follows
Inverse Transformation Technique contd...

Problems with empirical distributions

• The data in the previous example is restricted in the range 0.25 ≤ X ≤ 2.0
• The underlying distribution might have a wider range .
• Thus, try to find a theoretical distribution.
Inverse Transformation Technique contd...

Random Variates for Continuous Distribution.

(01). Uniform Distribution

1
, 𝑎≤𝑥<𝑏
𝑓 𝑥 =൝ 𝑏−𝑎
0 , 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
Inverse Transformation Technique contd...
CDF;
𝑥
1
𝐹 𝑥 =න 𝑑𝑥
𝑏−𝑎
0
𝑥
1
= 𝑥ቤ
𝑏−𝑎 𝑎
𝑥−𝑎
𝐹 𝑥 =
𝑏−𝑎

Thus, the Random Variate Generator is,

𝒙𝒊 = 𝒂 + 𝒃 − 𝒂 𝑹𝒊
Inverse Transformation Technique contd...

(02). Exponential Distribution

Suppose we need to generate variates from the exponential

distribution with mean 𝛽. Built random variate generator for
exponential distribution.

𝑥
1 −𝛽
𝑒 ;𝑥 >0
𝛽
𝑓(𝑥) =
0 ; o/w
Inverse Transformation Technique contd...
𝑥
1 −𝛽𝑥
𝐹 𝑥 = න 𝑒 𝑑𝑥
𝛽
0

𝑥 𝑥
−
1𝑒 𝛽
= อ
𝛽 −1
𝛽
0

𝑥 𝑥
−𝛽 −𝛽 −𝛽
𝐹 𝑥 = × 𝑒 − 𝑒 −0 = 1 − 𝑒
𝛽

𝑥
−𝛽
𝐹 𝑥 = ൝1 − 𝑒 ;𝑥 > 0
0 ; 𝑜/𝑤
Inverse Transformation Technique contd...
𝑥
−
𝐹 𝑥 =1 − 𝑒 𝛽 =𝑢
𝑥
−
𝑒 𝛽 =1−𝑢
𝑥
− = 𝑙𝑛 1 − 𝑢
𝛽

𝑥 = −𝛽𝑙𝑛(1 − 𝑢)

If u is U(0,1), 1 − 𝑢 is also U(0,1)

Thus, exponential variates can be generated by:

𝒙 = −𝜷 𝐥𝐧 𝒖
Inverse Transformation Technique contd...

(03). Normal Distribution – Direct Transformation

• Let Z1, Z2 be two standard normal variates.

• Plot the two as a point in the plane and represent them in a polar
coordinate system as,

𝑍1 = 𝐵 cos𝜃

𝑍2 = 𝐵 sin𝜃
Inverse Transformation Technique contd...

• It is known that 𝐵2 = 𝑍12 + 𝑍22 has the chi-square distribution

with 2 degrees of freedom, which is equalent to an exponential
distribution with mean 2

𝑌 = 𝜆𝑒 −𝜆𝑡 , 𝑡 ≥ 0

𝐸[𝑌] = 2 = 𝜆

thus the raidus 𝐵 can be generated,

𝐵 = −2 𝑙𝑛 𝑅
Inverse Transformation Technique contd...
• So a normal distribution can be generated by any one of the
following.

𝑍1 = −2 𝑙𝑛 𝑅1cos(2𝜋𝑅2)

𝑍2 = −2 𝑙𝑛 𝑅1sin(2𝜋𝑅2)

where 𝑅1 and 𝑅2 are uniformly distributed over (0,1).

• To obtain normal variates 𝑋𝑖 with mean 𝜇 and variance 𝜎2,

transform
𝑋𝑖 = 𝜇 + 𝜎𝑍𝑖