Bangabandhu Sheikh Mujibur Rahman

Science & Technology University, Gopalganj-8100

Course Code: STA401

Course Title: Simulation and Modeling
Topic: Generating Continuous Random Variables

Submitted by: (Student ID)

19STA037 , 19STA039 , 19STA041 , 19STA042 ,
19STA043 , 19STA044 , 19STA045 , 19STA046

Submitted to :
Teacher name: Dr. Md. Murad Hossain
Assistant Professor,

Department of : Statistics

BSMRSTU, GOPALGANJ

Date of submission: 05- November - 2024

Index:
1. Rayhan Ivan Rimon (ID:19STA044):
(Part-1) The Inverse Transform Algorithm (Page 1-6)
o Discusses the theory about the inverse transform algorithm, method, some
distribution use with example and mathematical examples.
2. Mohammad Abdul Halim (ID: 19STA043):
(Part-2) Theory of the Rejection Method (page 7 – 11)
o Discusses the theoretical framework of the rejection method, covering its
principles, mathematical basis, applications, and limitations.
3. Mostofa Shahariar Shuvo (ID: 19STA037):
(Part-2) Mathematical Problem Solving (page 11 – 17)
o Focuses on various mathematical problem-solving techniques, including types
of problems, solution strategies, and practical examples.
4. Iffat Ahmed Promi (ID: 19STA041):
(Part-3) The Polar Method (pages 18-20)
o Discuss the polar method for generating normal random variables
5. Sujoy Ghosh (ID: 19STA042):
(Part-3) The Polar Method (pages 18-20)
o Discuss the polar method for generating independent standard normal
random variables
6. Kawser Uddin (ID: 19STA039):
(Part-4) Poisson Process (page 21-24)
o Discusses the theory of Generating Poisson Process & Simulating a Two-
Dimensional Poisson Process.
7. Ariful Islam (ID: 19STA046):
(Part-5) Nonhomogeneous Poisson Process (page 25-28)
o Discusses the nonhomogeneous poisson process , definition , Steps, Properties
and mathematical examples.
8. Md Golam Rabby (ID:19STA045):
(Part-6) Summary (page 29-30)
o Summary of Random number Generation of Continuous Variable
 Part-1
The Inverse Transform Algorithm:
The Invers Transform Algorithm is a method for generating random variables from any
continuous probability distribution given its cumulative distribution function (CDF). The basic
idea is to use a uniform random variable and the inverse of the CDF to generate samples that
follow the desired distribution .
This method works because if U is uniformly distributed on [0,1] , then 𝑋 = 𝐹 −1 (𝑈) will have
the distribution described by the CDF F(x) .

The Inverse Transform Algorithm:

1. Generate a uniform random variable U in [0,1].
2. Find the CDF 𝐹𝑥 (𝑥) of the target distribution.
3. Invert the CDF to find 𝐹𝑥−1 (𝑢).
4. Calculate 𝑋 = 𝐹𝑥−1 (𝑈).
5. Output X as a sample from the target distribution.

Proposition: Let U be a uniform(0,1) random variable. For any continuous distribution

function F the random variable X define by
𝑋 = 𝐹 −1 (𝑈)
has distribution F . [ 𝐹 −1 (𝑢) is defined to be that value of x such that F(x)=u]

Proof: Let 𝐹𝑥 denote the distribution function 𝑋 = 𝐹 −1 (𝑈) . Then

𝐹𝑥 (𝑥) = 𝑃{𝑋 ≤ 𝑥}
= 𝑃{𝐹 −1 (𝑈) ≤ 𝑥}
Now since F is a distribution function it follows that F(x) is a monotone increasing function of
x and so the inequality “a ≤ b” is equivalent to the inequality “𝐹(𝑎) ≤ 𝐹(𝑏)”. Hence,form
Equation , we see that

𝐹𝑥 (𝑥) = 𝑃{𝐹(𝐹 −1 (𝑈))} Since

= 𝑃{𝑈 ≤ 𝐹(𝑥)} Since U is uniform(0,1)

= 𝐹(𝑥)
The above proposition thus shows that we can generate a random variable X from the
continuous distribution function F by generating a random number U and then setting 𝑋 =
𝐹 −1 (𝑈).

Page 1
Example : If X is an exponential random variable with rate 1, then its distribution function is
given by
𝐹(𝑥) = 1 − 𝑒 −𝑥
Let F(x) = u , then
𝑢 = 𝐹(𝑥) = 1 − 𝑒 −𝑥
1 − 𝑢 = 𝑒 −𝑥
Taking logarithms,
𝑥 = − 𝑙𝑜𝑔 (1 − 𝑢)
Hence we can generate an exponential with parameter 1 by generating a random number U and
then setting
𝑋 = 𝐹 −1 (𝑈) = −𝑙𝑜𝑔 (1 − 𝑈)
Generate random samples
● Generate a uniform random variable U between 0 and 1.
● Set 𝑋 = − 𝑙𝑜𝑔 (1 − 𝑈) .

Example : If X is an exponential random variable with rate 𝜆, then its distribution function is
given by

𝐹(𝑥) = 1 − 𝑒 −𝜆𝑥
1
Hence, an exponential random variable X with rate 𝜆 (𝑚𝑒𝑎𝑛 𝜆).

Let U be a uniform(0,1) random variable and F(x) = u , then

𝑢 = 𝐹(𝑥) = 1 − 𝑒 −𝜆𝑥

1 − 𝑢 = 𝑒 −𝜆𝑥
Taking logarithms,
𝜆𝑥 = − 𝑙𝑜𝑔 (1 − 𝑢)
1
𝑥 = − 𝑙𝑜𝑔 (1 − 𝑢)
𝜆

1
Hence we can generate an exponential with parameter 𝜆 (𝑚𝑒𝑎𝑛 𝜆) by generating a random
number U and then setting

Page 2
 1
𝑋 = 𝐹 −1 (𝑈) = − 𝑙𝑜𝑔 (1 − 𝑈)
𝜆
Since U ~ Uniform[0,1], we can also use 1 – U ~ Uniform[0,1], so
1
𝑋 = − 𝑙𝑜𝑔 (1 − 𝑈)
𝜆
Generate random samples
● Generate a uniform random variable U between 0 and 1.
1
● Set 𝑋 = − 𝜆 𝑙𝑜𝑔 (1 − 𝑈) .

Example : let U1 =0.147, U2 =0.282, U3= 0.436 with parameter 0.34

Now
1
𝑋1 = − 𝑙𝑜𝑔 (1 − 𝑈1 )
𝜆
1
= − 𝑙𝑜𝑔 (1 − 0.147) = 0.203
0.34
1
𝑋2 = − 𝑙𝑜𝑔 (1 − 0.282) = 0.423
0.34
1
𝑋1 = − 𝑙𝑜𝑔 (1 − 0.436) = 0.732
0.34

Exercises 1: Give a method for generating a random variable having density function
𝑒𝑥
𝑓(𝑥) = , 0≤𝑥≤1
𝑒−1

Solution: To generate a random variable with this density function, we can use the inverse
transform method.
The given PDF is,
𝑒𝑥
𝑓(𝑥) = , 0≤𝑥≤1
𝑒−1
There the CDF is F(x),
𝑥 𝑥
𝑒𝑡 𝑒𝑥 − 1
𝐹(𝑥) = ∫ 𝑓(𝑡) 𝑑𝑡 = ∫ 𝑑𝑡 =
0 0 𝑒−1 𝑒−1
𝑒𝑥 − 1
𝐹(𝑥) = , 0≤𝑥≤1
𝑒−1

Page 3
Let U be a uniform(0,1) random variable and F(x) = u , then
𝑒𝑥 − 1
𝑈=
𝑒−1
𝑈(𝑒 − 1) = 𝑒 𝑥 − 1
𝑒 𝑥 = 𝑈(𝑒 − 1) + 1
𝑥 =𝑙𝑛 (𝑈(𝑒 − 1) + 1)
Generate Random Samples
To generate a random variable X with the given f(x):
1. Generate a uniform random variable U between 0 and 1.
2. Set 𝑥 =𝑙𝑛 (𝑈(𝑒 − 1) + 1) .

Example: Uniform Distribution

For a uniform distribution on an interval [a,b] :
The given PDF is,
1
𝑓(𝑥) = ,𝑎 ≤ 𝑥 ≤ 𝑏
𝑏−𝑎

There the CDF is F(x),

𝑥
1 𝑥−𝑎
𝐹(𝑥) = ∫ 𝑑𝑡 =
𝑎 𝑏−𝑎 𝑏−𝑎
Let U be a uniform(0,1) random variable and F(x) = u , then
𝑥−𝑎
𝑈=
𝑏−𝑎
𝑈(𝑏 − 𝑎) = (𝑥 − 𝑎)
𝑥 = 𝑈(𝑏 − 𝑎) + 𝑎
Generate Random Samples
To generate a random variable X with the given f(x):
1. Generate a uniform random variable U between 0 and 1.
2. Set 𝑥 = 𝑈(𝑏 − 𝑎) + 𝑎

Page 4
Example :- Give a method to generate a random variable having density function
𝑥−2
𝑖𝑓 2 ≤ 𝑥 ≤ 3
2
𝑓(𝑥) = 𝑥
2−3
𝑖𝑓 3 ≤ 𝑥 ≤ 6
{ 2

Solution: To generate a random variable with this density function, we can use the inverse
transform method
The PDF is,
𝑥−2
𝑓1 (𝑥) = 𝑓𝑜𝑟 2 ≤ 𝑥 ≤ 3
2
The CDF is,
𝑥 (𝑥 − 2)2
𝑡−2
𝐹1 (𝑥) = ∫ 𝑑𝑡 =
2 2 4
Let U be a uniform(0,1) random variable and F(x) = u , then
(𝑥 − 2)2
𝑈 =
4
2√𝑈 = (𝑥 − 2)

𝑥 = 2√𝑈 + 2
Generate Random Samples
To generate a random variable X with the given f(x):
1. Generate a uniform random variable U between 0 and 1.
2. Set 𝑥 = 2√𝑈 + 2

Again the PDF is,

2 − 𝑥/3
𝑓1 (𝑥) = 𝑖𝑓3 ≤ 𝑥 ≤ 6
2
The CDF is,
𝑥
2 − 𝑡/3 1 5𝑥 − 𝑥 2 /6 − 15
𝐹1 (𝑥) = ∫ 𝑑𝑡 = +
3 2 4 6

Let U be a uniform(0,1) random variable and F(x) = u , then

Page 5
 1 5𝑥 − 𝑥 2 /6 − 15
𝑈 = +
4 6
30 ∓ √900 − 4(36𝑈 − 81)
𝑥 =
60

Generate Random Samples

To generate a random variable X with the given f(x):
1. Generate a uniform random variable U between 0 and 1.
30∓√900−4(36𝑈−81)
2. Set 𝑥 = 60

Example:- Give a method to generate a random variable having density function

1
𝑓(𝑥) = 𝑥 + ,0 ≤ 𝑥 ≤ 1
2

Solution: To generate a random variable with this density function, we can use the inverse
transform method
The PDF is,
1
𝑓(𝑥) = 𝑥 + ,0 ≤ 𝑥 ≤ 1
2
Then the CDF is ,
𝑥
1 𝑥2 + 𝑥
𝐹(𝑥) = ∫ (𝑡 + ) 𝑑𝑡 =
0 2 2
Let U be a uniform(0,1) random variable and F(x) = u , then
𝑥2 + 𝑥
𝑈 =
2
2𝑈 = 𝑥 2 + 𝑥
𝑥 2 + 𝑥 − 2𝑈 = 0

−1 + √1 + 8𝑈
𝑥 =
2

Generate Random Samples

To generate a random variable X with the given f(x):
1. Generate a uniform random variable U between 0 and 1.
−1 + √1+8𝑈
2. Set 𝑥 = .
2

Page 6
 Part-2
Rejection Method:

The Rejection Method (or Acceptance-Rejection Sampling) is a technique in probability

and statistics for generating random samples from a target probability distribution, f(x), that
may be complex or difficult to sample from directly. The method uses a simpler, easier-to-
sample distribution, called the proposal distribution g(x), to help generate samples that
approximate f(x).

How the Rejection Method Works:

The basic idea is to generate candidate samples from the proposal distribution g(x), then
accept or reject each candidate based on how closely it aligns with the target distribution f(x).
Samples that better match f(x) are more likely to be accepted, while those that do not align
are rejected. By doing this, the method ensures that the accepted samples follow the target
distribution f(x).

Proposal distribution:
A proposal distribution g(x) in rejection sampling is a probability distribution used to
generate candidate values for a target distribution f(x). The proposal distribution should be
simple to sample from and should be scaled by a constant c so that it envelopes f(x), allowing
us to accept or reject samples based on how well they match the target distribution.

Page 7
Assumptions:
1. Proposal Distribution: There exists a probability density g(x) from which it is easy
to sample.
𝑓(𝑥)
2. Constant c: There is a constant c ≥ 1 such that 𝑔(𝑥) ≤ 𝑐 for all x. This ensures that
f(x) is always bounded above by c ⋅ g(x).

Steps of the Rejection Method:

Figure 5.1. The rejection method for simulating a random variable X having density function
f.

1. Generate Y from g(x):

o Sample a random variable Y from the proposal distribution g(x).
2. Generate a Uniform Random Variable U:
o Generate a random number U from the uniform distribution on the interval
(0,1).
3. Acceptance Check:
𝑓(𝑥)
o Check if 𝑈 ≤ 𝑐 . 𝑔(𝑥)

▪ If yes, accept Y and set X = Y as a sample from the target distribution

f(x).
▪ If no, reject Y and repeat from Step 1 with a new sample.

Explanation:
𝑓(𝑥)
The ratio 𝑈 ≤ gives the probability of accepting the candidate sample Y. If Y is in a
𝑐 . 𝑔(𝑥)
region where f(x) is high relative to g(x), it is more likely to be accepted, thereby
approximating f(x) as the distribution of accepted samples.

Page 8
The Rejection Method has the following properties:
1. Density Matching: The distribution of accepted samples X has the target density f(x).
2. Geometric Iteration: The number of iterations required follows a geometric
1
distribution with success probability 𝑐 .

Generating Samples from Common Proposal Distributions:

1. Uniform Distribution:
For the uniform distribution defined as:
1
g(x) = 𝑏−𝑎 ; for x ∈ [a , b]

• Generate a random number U uniformly between 0 and 1.

• Transform U to obtain Y:
Y = a + U × (b−a)

2. Exponential Distribution:
For the exponential distribution defined as:

g(x) = λ𝑒 −λx ; x≥ 0
• Generate a uniform random number U between 0 and 1.
1
• Transform U to obtain Y: Y = − λ ln (𝑈)

3. Beta Distribution:
For the Beta distribution defined as:
𝑥 α−1 ( 1−𝑥 )β−1
g(x) = ; for x ∈ [0,1]
𝐵(α ,β)

where B(α, β) is the beta function.

• Generate U1 and U2 uniformly between 0 and 1

Page 9
 1 1
• Transform to obtain Y: Y = 𝑈1α (1 − 𝑈1)β

3.Gamma Distribution:

For the Gamma distribution defined as:

λ𝑘 𝑥 𝑘−1 𝑒 −λx
g(x) = ; for x ≥ 0
𝛤 (𝑘)

where λ is the rate parameter and k is the shape parameter.

1
• Generate k independent exponential random variables Xi with mean λ for

Xi ~ 𝐸𝑥𝑝𝑜𝑛𝑒𝑛𝑡𝑖𝑎𝑙 ( λ ) ; i = 1,2,……..…,k

• Sum the generated values to obtain Y: Y = X1 + X2 +…+ Xk

4. Normal Distribution:
For the Normal distribution defined as:
( 𝑥−μ) 2
1 −
g(x) = √(2𝜋𝜎2 ) 𝑒 2𝛿2

• Use a random number generator μ that provides samples from the

normal distribution with 𝛿 2 .
• Directly generate Y:
Y ∼ N (μ, 𝛿 2 )

Applications of the Rejection Method:

1. Monte Carlo Simulations: Used in simulations to generate samples from complex
distributions without easy-to-invert CDFs.
2. Sampling Non-Standard Distributions: Allows sampling from distributions with
unusual shapes (e.g., heavy-tailed or multimodal).
3. Bayesian Inference & MCMC: Useful in Bayesian methods for sampling posterior
distributions in parameter estimation.

Page 10
 4. Financial Modeling: Assists in simulating risk variables with empirical or complex
distributions.
5. Engineering & Reliability: Generates variables for reliability testing and failure
analysis with non-standard probabilities.

Limitations of the Rejection Method:

1. High Rejection Rates: Computationally inefficient if the proposal distribution poorly
fits the target, causing many rejections.
2. Proposal Distribution Selection: Requires careful choice of a proposal distribution,
especially for complex distributions.
3. High Dimensionality: Inefficient in high-dimensional problems, leading to higher
rejection rates.
4. Density Ratio Bound: Needs an upper bound for the density ratio, which can be hard
to set accurately.
5. Variable Execution Time: Efficiency depends on the acceptance rate, making timing
unpredictable.

Example 1: Let us use the rejection method to generate a random variable having density
function,
f (x) = 20x (1 − x) 3 ; 0<x<1

Since this random variable (which is beta with parameters 2, 4) is concentrated in the interval
(0, 1), let us consider the rejection method with

g(x) = 1 ; 0<x<1

Solution:
To determine the smallest constant c such that f (x)/g(x) _ c, we use calculus to
determine the maximum value of,

𝑓(𝑥)
= 20x (1 − x)3
𝑔(𝑥)

Differentiation of this quantity yields,

𝑑 𝑓(𝑥)
( ) = 20 [(1 − x)3 − 3x (1 − x)2]
𝑑𝑥 𝑔(𝑥)

Page 11
 1
Setting this equal to 0 shows that the maximal value is attained when x = 4 and Thus,

𝑓(𝑥) 1 3 135
≤ 20 (4) (4)4 = =c
𝑔(𝑥) 64

Hence,

𝑓(𝑥) 256
= x(1 − 𝑥)3
𝑐𝑔(𝑥) 27

and thus, the rejection procedure is as follows:

step 1: Generate random numbers 𝑈1 and 𝑈2 .
256
step 2: If 𝑈2 ≤ 27 𝑈1 (1 − 𝑈1 )3, stop and set X = 𝑈1 . Otherwise, return to
Step 1.
135
The average number of times that Step 1 will be performed is c = ≈ 2.11
64

3
Example 2: Suppose we wanted to generate a random variable having the gamma (2,1)
density
1
F(x) = K 𝑥 2 𝑒 −𝑥 ; x>0
3
Where K=1/Г(2)= 2/Гπ Because such a random variable is concentrated on the positive axis
3
and has mean 2, it is natural to try the rejection technique with an exponential random
variable with the same mean. Hence, let

2𝑥
2
g(x)= 3 𝑒 3 ; x>0

Solution:
1 −𝑥
𝑓(𝑥) 3𝐾
= 2 𝑥2 𝑒 3
𝑔(𝑥)

3
that is, when x=2 Hence,

𝑓(𝑥) 3𝐾 3 1/2
c = Max = ( ) 𝑒 −1/2
𝑔(𝑥) 2 2

33/2
= (2𝜋𝑒)1/2 , Since K = 2/√π

Page 12
Since,
1
𝑓(𝑥) 2𝑒
= ( 3 )1/2 𝑥 2 𝑒 −𝑥/3
𝑐𝑔(𝑥)

3
we see that a gamma (2,1) random variable can be generated as follows:
step 1: Generate a random number U1 and set Y=-3/2 log U1.
step 2: Generate a random number U2.
step 3: If U2 < (2𝑒𝑦/3)1/2 𝑒 −𝑦/3 , set X = Y. Otherwise, return to Step 1.

The average number of iterations that will be needed is:

3
C = 3(2𝜋𝑒)1/2 ≈ 1.257

In the previous example, we generated a gamma random variable using the

rejection approach with an exponential distribution having the same mean as the
gamma. It turns out that this is always the most efficient exponential to use when
generating a gamma random variable. To verify this, suppose we want to generate a random
variable having density function:

𝐾𝑒 −𝜆𝑥 𝑥 𝛼−1 𝐾
f(x) = = 𝜇 𝑥 (𝛼−1) 𝑒 (𝜇−𝜆)𝑥
𝜇𝑒 𝜇𝑥

We see that when 0 < 𝛼 < 1,

𝑓(𝑥)
lim 𝑔(𝑥) =∞
𝑥→0

Thus, showing that the rejection technique with an exponential can not be used inthis case. As
the gamma density reduces to the exponential when α = 1, let us
suppose that α > 1. Now, when μ ≥ λ,

𝑓(𝑥)
lim =∞
𝑥→0 𝑔(𝑥)

and so we can restrict attention to values of μ that are strictly less than λ. With
such a value of μ, the mean number of iterations of the algorithm that will be
required is,

𝑓(𝑥) 𝐾
c(𝜇) = Max 𝑔(𝑥) = Max 𝜇 𝑥 (𝛼−1) 𝑒 (𝜇−𝜆)𝑥
𝑥 𝑥

To obtain the value of x at which the preceding maximum occurs, we differentiate and set equal
to 0 to obtain,

Page 13
 0 = 𝑥 (𝛼−1) 𝑒 (𝜇−𝜆)𝑥 − (𝜆 − 𝜇)𝑥 (𝛼−1) 𝑒 (𝜇−𝜆)𝑥

yielding that the maximum occurs at,

𝛼−1
x = 𝜆−𝜇

Substituting back yields that,

𝛼−1
𝐾 𝛼−1 (𝜇−𝜆)( )
c (𝜇) = 𝜇 (𝜆−𝜇)𝛼−1 𝑒 𝜇−𝜆

𝐾 𝛼−1
= 𝜇 (𝜆−𝜇)𝛼−1 𝑒 1−𝛼
Hence, the value of μ that minimizes c(μ) is that value that maximizes 𝜇 (𝜆 − 𝜇)𝛼−1
Differentiation gives
𝑑
{𝜇(𝜆 − 𝜇)𝛼−1 } =(𝜆 − 𝜇)𝛼−1 − (𝛼 − 1) {𝜇(𝜆 − 𝜇)𝛼−2 }
𝑑𝑥

Setting the preceding equal to 0 yields that the best value of μ satisfies,

λ − μ = (α − 1) μ
or, μ = λ/α

That is, the exponential that minimizes the mean number of iterations needed by
the rejection method to generate a gamma random variable with parameters α and
λ has the same mean as the gamma; namely, α/λ.

Example 3: Generating a Normal Random Variable

To generate a standard normal random variable Z (i.e., one with mean 0 and variance 1, note
first that the absolute value of Z has probability density function
2 2 /2
f(x) = 2𝜋 𝑒 −𝑥 ; 0<x<∞

We start by generating from the preceding density function by using the rejection
method with g being the exponential density function with mean 1—that is,

g(x) = 𝑒 −𝑥 ; 0<x<∞

Solution:

𝑓(𝑥) 2 2 /2
= √𝜋 𝑒 −𝑥
𝑔(𝑥)

Page 14
and so the maximum value of f (x)/g(x) occurs at the value of x that maximizes
x − x2/2. Calculus shows that this occurs when x = 1, and so we can take

𝑓(𝑥) 𝑓(1) 2𝑒
c = Max 𝑔(𝑥) = =√( 𝜋 )
𝑔(1)

Because,

𝑓(𝑥) 𝑥2 1
= exp {𝑥 − − 2}
𝑐𝑔(𝑥) 2
(𝑥−1)2
= exp {-
2

it follows that we can generate the absolute value of a standard normal random
variable as follows:

step 1: Generate Y, an exponential random variable with rate 1.

step 2: Generate a random number U.
step 3: If U _ exp {−(Y − 1)2/2}, set X = Y . Otherwise, return to Step 1.

Once we have simulated a random variable X having density function as in

Equation (5.1), —and such a random variable is thus distributed as the absolute
value of a standard normal—we can then obtain a standard normal Z by letting Z
be equally likely to be either X or −X.

In Step 3, the value Y is accepted if U ≤ exp{−(Y −1)2/2}, which is equivalent

to −log(U) ≥ (Y − 1)2/2. However, in Example 5b it was shown that −log(U) is
exponential with rate 1, and so the above is equivalent to the following:

step 1: Generate independent exponentials with rate 1, Y1 and Y2.

step 2: If Y2 ≥ (Y1 − 1)2/2, set X = Y1. Otherwise, return to Step 1.

Suppose now that the foregoing results in Y1 being accepted—and so we know

that Y2 is larger than (Y1 −1)2/2. By how much does the one exceed the other? To answer this,
recall that Y2 is exponential with rate 1, and so, given that it exceeds some value, the amount
by which Y2 exceeds (Y1 −1)2/2 [i.e., its “additional life” beyond the time (Y1 − 1)2/2] is (by
the memoryless property) also exponentially distributed with rate 1. That is, when we accept
in Step 2 not only do we obtain X (the absolute value of a standard normal) but by computing
Y2 − (Y1 − 1)2/2 we can also generate an exponential random variable (independent of X)
having rate 1.

Hence, summing up, we have the following algorithm that generates an

exponential with rate 1 and an independent standard normal random variable.

step 1: Generate Y1, an exponential random variable with rate 1.

step 2: Generate Y2, an exponential random variable with rate 1.
step 3: If Y2 − (Y1 − 1)2/2 > 0, set Y = Y2 − (Y1 − 1)2/2 and go to Step 4.

Page 15
Otherwise, go to Step 1.
step 4: Generate a random number U and set

1
𝑌1 𝑖𝑓 𝑈 ≤ 2
Z={ 1
−𝑌1 𝑖𝑓 𝑈 > 2

The random variables Z and Y generated by the foregoing are independent with
Z being normal with mean 0 and variance 1 and Y being exponential with rate 1.
(If you want the normal random variable to have mean μ and variable σ2, just take μ + σ Z).

Example 4: Suppose we want to generate a gamma (2, 1) random variable

conditional on its value exceeding 5. That is, we want to generate a random variable having
density function

𝑥𝑒 −𝑥 𝑥𝑒 −𝑥
f(x) = ∞ =6𝑒 −5 ; x≥5
∫5 𝑥𝑒 𝑥 𝑑𝑥

where the preceding integral was evaluated by using integration by parts. Because a gamma (2,
1) random variable has expected value 2, we will use the rejection method based on an
exponential with mean 2 that is conditioned to be at least 5. That is, we will use
1 −𝑥/2
𝑒
2
g(x) = ; 𝑥≥5
𝑒 −5/2
Solution:

𝑓(𝑥) 𝑒 5/2
= 𝑥𝑒 −𝑥/2 ; x≥ 5
𝑔(𝑥) 3

Because x𝑒 −𝑥/2 is a decreasing function of x when x ≥ 5, it follows that the number of

iterations needed in the algorithm will be geometric with mean

𝑓(𝑥) 𝑓(5)
C = max {𝑔(𝑥)} = 𝑔(5) = 5/3
𝑥≥5

To generate an exponential with rate 1/2 that is conditioned to exceed 5, we use

the fact that the amount by which it exceeds 5 is (by the lack of memory property
of exponential random variables) also exponential with rate 1/2. Therefore, if X is exponential
with rate 1/2, it follows that 5+ X has the same distribution as does X conditioned to exceed 5.
Therefore, we have the following algorithm to simulate a random variable X having density
function f .

Page 16
step 1: Generate a random number U.
step 2: Set Y = 5 − 2 log(U).
step 3: Generate a random number U.
5
𝑒2
step 4: if U ≤ 𝑌 𝑒 −𝑌/2 , set X = Y and stop; otherwise return to step 1.
5

Page 17
 Part-3

Polar Method : The polar method is a specific technique for generating normally
distributed random variables , also known as the Marsaglia polar method.

The Polar Method for Generating Normal Random Variables :

Figure : Polar Co-ordinates

Let X and Y be independent standard normal random variables and let R and
denote the polar coordinates of the vector (X, Y).
R2=X2+Y2
𝑌
tanΘ= 𝑋

Since X and Y are independent, their joint density is the product of their individual
densities and is thus given by
𝑥2 𝑦2
1 − 1 −
F(x,y) = 𝑒 2 𝑒 2
√2𝜋 √2𝜋

(𝑥2 +𝑦2 )
1 −
=2𝜋 𝑒 2

Now we can elaborately discuss the matter step by step

STEP 1: Generate random numbers U1, and U3.
STEP 2: R2 = —2 log U1, (and thus R2 is exponential with mean 2). Θ= 22π U2,
(and thus Θ is uniform between 0 and 2π).

STEP 3: Now let

X = R cos Θ = √−2𝑙𝑜𝑔𝑈1 cos(2πU2 )

Page 18
 Y = R sin Θ=√−2𝑙𝑜𝑔𝑈1 sin(2πU2 )

The transformations given by are known as the Box—Muller transformations.

When the polar method used for generating independent standard normals:

Figure : (V1,V2) Uniformly Distributed in the Square

(V1,V2) in the circle and then setting

𝑉2 𝑉2
sinΘ = = 1
𝑅
(𝑉12 +𝑉22 )2

𝑉1 𝑉1
cosΘ = = 1
𝑅
(𝑉12 +𝑉22 )2

It now follows from the Box-Muller transformation that we can generate independent
standard normals by generating a random number U and setting
1
𝑉1
X = (−2𝑙𝑜𝑔 𝑈)2 1
(𝑉12 +𝑉22 )2

1
𝑉2
Y = (−2𝑙𝑜𝑔 𝑈)2 1
(𝑉12 +𝑉22 )2

Now we can elaborately discuss the matter step by step

STEP 1: Generate random numbers, 𝑈1 and 𝑈2 .

STEP 2: Set V1 = 2U1-1, V2 = 2U2-1, S = 𝑉12 +𝑉22 .
STEP 3: If S > 1 return to Step 1.
STEP 4: Return the independent standard normals.

Page 19
 −2𝑙𝑜𝑔𝑆 −2𝑙𝑜𝑔𝑆
X=√ V1 , Y = √ V2
𝑆 𝑆

Page 20
 Part-4

Generating a Poisson Process

Generating a Poisson process refers to simulating or creating a sequence of random events

that occur over time or space, according to the principles of the Poisson process. In a Poisson
process, events happen independently and randomly, but at a constant average rate, often
denoted by λ, which is the expected number of events per unit time or space.

Purpose of Generating a Poisson Process

Generating a Poisson process helps to model and analyze systems where random events occur
at a steady average rate, such as:

• Customer arrivals at a service center

• Calls arriving at a call center
• Requests to a web server
• Particle emissions in a radioactive sample

Steps in Generating a Poisson Process

1. Define the Rate Parameter (λ): This is the average rate at which events are expected
to occur.
2. Generate Event Times:
o Interarrival Times: In a Poisson process, the times between consecutive events
(interarrival times) are exponentially distributed with mean 1/λ.
o You can sample these interarrival times from an exponential distribution and
add them successively to get the actual timestamps of each event.
3. Repeat Until Desired: Continue generating events until you reach a desired end time
or a specified number of events.

Example

Suppose we want to simulate a Poisson process for customer arrivals at a store, where
customers arrive at an average rate of 5 per hour:

1. Set λ=5.
2. Draw the time between each customer arrival from an exponential distribution with
mean 1/λ=1/5.
3. Sum the interarrival times to get cumulative arrival times of each customer.

This approach is commonly used in simulation modeling to represent real-world systems,

allowing for performance analysis, optimization, and planning.

Generating a Poisson process involves simulating the first n event that occur randomly over
time, with a constant average rate (λ).To do so we make use of the result that the times between

Page 21
successive events for such a process are independent exponential random variables each with
rate (λ). Thus, one way to generate the process is to generate these interarrival times. So if we
generate n random numbers U1, U2,……..Un and set Xi = -1/λ log Ui, then Xi can be regarded
as the time between the (I – 1)st and the ith event of the Poisson process. Since the actual time
of the jth event will equal the sum of the first j interarrival times, it thus follows that the
genetrated values of the first n events times are ∑Xi, j = 1,2,…….n.
If we wanted to generate the first T time units of the Poisson process, we can follow the
preceding procedure of successive generating the interarrivals times, stopping when their sum
exceeds T. That is, the following algorithm can be used to generate all the event times occurring
in (0, T) of a poisson process having rate (λ). In the algorithm t refers to time, I is the number
of events that have occurred by time t and S(I) is the most recent event time.

Generating the First T Time Units of a Poisson Process with Rate λ

STEP 1: T = 0, I = 0.
STEP 2: Generate a random number U
STEP 3: t = t - = -1/λ log U. If t > T, stop.
STEP 4: I = I + 1, S(I) = t
STEP 5: Go to step 2.

The final value of I in the preceding algorithm will represent the number of events that occur
by time T, and the values S(1),…………S(I) will be the I event times in increasing order.
Another way to simulate the first T time units of a poisson process with rate λ starts by
simulating N(T), the total number of events that occur by the time T. Because N(T) is a poisson
process with mean λT, this is easily accomplished by one of the approaches. If the simulated
value of N(T) is n, then n random numbers U1,…………Un are generated and
{TU1,…………TUn} are taken as the set of event times by time T of the poisson process. N(T)
= n, the unordered set of event times are distributed as the set of n independent uniform (0, T)
random variables.
To verify that the preceding method works, let N(t) equal the number of values in the set
{TU1,…………….,TUn(t)} that are less than t. we must now argue that N(t), 0 ≤ t ≤ T, is a
poisson process. To show that it has independent and stationary increments, let I1,………..Ir
be r disjoint time intervals, I =1 ,……r and say it is type r+1 if it is does not lie in any of the r
intervals. Because the Ui, I ≥1 are independent , it follows that each of the poisson number of
events N(T) is independently classified as being of one of the types 1,………….r+1, with
respective probabilities P1,………Pr+1, where Pi is the length of the interval Ii divided by T
when I ≤ r, and Pr+1 = 1 - ∑Pi. N1,……….Nr, the numbers of events of the disjoint intervals,
are are independent Poisson random variables, with E[Ni] equal to λ multiplied by the length
of the interval Ii; which established that N(t), 0 ≤ t ≤ T, has stationary as well as independent

Page 22
increments. Because the number of events in any interval of length h is poisson distributed with
mean λh,
We have,

𝑃{𝑁(ℎ)=1} 𝛾ℎ𝑒 −𝛾ℎ

lim = lim =𝛾
ℎ→0 ℎ ℎ→0 ℎ

And
𝑃{𝑁(ℎ)≥2} 1− 𝑒 −𝛾ℎ −𝛾ℎ𝑒 −𝛾ℎ
lim = lim =0
ℎ→0 ℎ ℎ→0 ℎ

Which complete the verification.

Simulating a Two – Dimentional Poisson Process

A two-dimensional Poisson process extends the concept of a one-dimensional Poisson

process to model random events occurring over an area (e.g., points scattered across a
geographical map, defects on a surface, or star locations in an astronomical image). The events
occur independently and uniformly across the area, at an average rate per unit area.

Key Characteristics of a 2D Poisson Process

1. Intensity (λ) is defined as the average number of events per unit area (e.g., events per
square meter).
2. Random Locations: The location of each event within the area is uniformly distributed.

Steps to Simulate a Two-Dimensional Poisson Process

Suppose we want to simulate events over a rectangular area of width W and height H, with an
event rate of λ events per unit area.

Step-by-Step Simulation:

1. Determine the Expected Number of Events:

o The expected number of events, N, in the area
o A=W×H is N=λ×A
2. Draw Actual Event Count:
o Draw a random sample for the actual number of events, N, from a Poisson
distribution with mean λ×A.
3. Generate Random Event Locations:
o For each event, generate its (x,y) coordinates independently:
▪ x-coordinates are sampled uniformly from [0,W].
▪ y-coordinates are sampled uniformly from [0,H].

Explanation

1. Intensity Setting: The parameter λ represents the density of events per unit area.

Page 23
 2. Uniform Distribution: Each point’s (x,y) coordinates are chosen uniformly within the
area, reflecting the Poisson process's assumption that events are uniformly and
independently distributed.

Applications of a 2D Poisson Process

• Geographical Modeling: Locations of trees in a forest or stars in the sky.

• Defect Testing: Positions of defects on a manufactured surface.
• Event Distribution: Modeling events scattered over any two-dimensional space, like
urban planning or resource distribution.

Page 24
 Part-5

Non-Homogeneous Poisson Process (NHPP)

Definition: A non-homogeneous Poisson process (NHPP) is a generalization of the
Poisson process where the rate at which events occur is not constant over time. Instead, the
rate (or intensity function) λ (t) varies with time.
This is such a process that generates random points in time, but the average rate of arrivals
varies with time.

Properties of Non-homogeneous Poisson process:

• Time-varying rate: The rate of events in a NHPP is a function of time, represented by
the rate function λ (t).
• Independent increments: The increments of a NHPP are independent.
• Poisson distribution: For any measurable interval, the number of events has a Poisson
distribution.
• Not stationary: NHPPs do not have the stationary increment property. (The
distribution of the number of events between two particular points on the timeline is a
function of both starting-point and endpoint of the time interval.)

Example

Consider the number of incoming phone calls to a customer support center, where the rate of
calls changes based on the time of day. Suppose the rate function λ (t) for the number of calls
per minute is defined as follows:

• From 9:00 am to 12:00 pm: λ(t)= 5 (5 calls per minute)

• From 12:00 pm to 3:00 pm: λ(t)= 10 (10 calls per minute)
• From 3:00 pm to 6:00 pm: λ(t)=7 (7 calls per minute)

This varying rate over time indicates a non-homogeneous Poisson process. The expected
number of calls in each period can be calculated by integrating λ (t) over the respective
intervals.

Steps by Step Calculation

Let’s discuss about Thinning algorithm to generate NHPP:

Page 25
Let's walk through a mathematical example for simulating a non-homogeneous Poisson
process (NHPP). In statistical simulations, we often use a method called thinning to simulate
events from a non-homogeneous Poisson process.

Page 26
Example Setup

Suppose we want to simulate a NHPP with a time-varying intensity function given by:

where:

• λ(t) gives the rate of events per unit time.

• t is the time in the interval [0,10]

To simulate a NHPP, we follow these steps:

Page 27
Example Calculation

Let’s go through a few steps with hypothetical numbers to illustrate this:

Page 28
 Part-6
Summary of Random number Generation of Continuous Variable
Inverse transform Algorithm:
Let U be a The Inverse Transform Sampling Algorithm is a method used in simulations to
generate random samples from a specific probability distribution. Here’s a summary of the
process:
1. Generate Uniform Random Variable: Start by generating a uniform random number
from the interval [0, 1].
2. Cumulative Distribution Function (CDF): Determine the cumulative distribution
function of the target distribution. This function represents the probability that a random
variable is less than or equal to .
3. Inverse CDF: Compute the inverse of the CDF, . This gives you the corresponding value
in the target distribution.
4. Sample Generation: The result is a sample from the desired distribution.

This method is particularly effective when the inverse CDF can be calculated easily. It allows
for efficient random sampling from various distributions, including exponential, uniform, and
others, facilitating simulations in fields such as finance, engineering, and statistics.

Rejection method:
The reverse method of simulation, often referred to as the Reverse Acceptance-Rejection
Sampling or simply the Reverse Method, is a technique used to generate samples from a target
distribution by utilizing a known distribution. Here’s a concise summary:
1. Target Distribution: Identify the probability distribution from which you want to
sample.
2. Proposal Distribution: Select a simpler proposal distribution that is easy to sample from
and covers the support of the target distribution.
3. Sample Generation: Generate samples from the proposal distribution .
4. Acceptance Criterion: For each sample , compute the acceptance probability using the
ratio , where is a constant that bounds the ratio for all .
5. Accept or Reject: Generate a uniform random number from [0, 1]. Accept the sample
if is less than the acceptance probability; otherwise, reject it and repeat the sampling
process.
6. Output Samples: Accepted samples will approximate the target distribution.
This method is useful for complex distributions where direct sampling is difficult, and it
leverages the flexibility of simpler distributions to facilitate effective sampling.

Polar Method:

Page 29
The Polar Method, also known as the Polar Coordinate Method, is a technique for generating
pairs of independent standard normal random variables. Here’s a summary of the process:
1. Uniform Sampling: Generate two independent uniform random variables and from the
interval .
2. Check Radius: Compute . If is greater than or equal to 1, repeat the sampling for and
(this ensures that the points lie within the unit circle).
3. Transform to Normal: Once , compute the polar coordinates

Here, and are independent standard normal random variables.

4. Output: The generated and can be used in simulations requiring normally distributed
samples.

This method is efficient for generating normal variates and is particularly advantageous
because it avoids the use of trigonometric functions, making it computationally efficient.

Poisson process :
In simulation modeling, the Poisson process is utilized to represent and analyze events that
occur randomly and independently over time or space. Here’s a concise summary:
Key Characteristics:
1. Event Rate (λ): Defines the average number of events occurring in a fixed interval (time
or space).
2. Independence: The occurrence of one event does not influence the timing of another.
3. Memory lessness: The time until the next event follows an exponential distribution,
meaning that the past does not affect future intervals.

Non homogeneous poisson process:

A Non-Homogeneous Poisson Process (NHPP) is a generalization of the standard Poisson
process, allowing for the event rate to vary over time or space. Here’s a summary of its key
features and how it is used in simulation:

Key Characteristics:
1. Variable Rate (λ(t)): The rate of events, denoted λ(t) , is a function of time, meaning it
can increase or decrease throughout the observation period.
2. Independence of Events: Like the homogeneous Poisson process, the events in an NHPP
are independent of each other.
3. Non-Constant Intensity: The probability of event occurrence is proportional to the
instantaneous rate, which can change based on the time or other influencing factors.

Page 30
References:
1. Simulation (Fifth Edition): Sheldon M. Ross Epstein (Department of Industrial and
Systems Engineering University of Southern California)
2. Chatgpt & Google

Page 31