Simulation of Random variables

Lecture 2: Introduction to Numerical Simulation

Ahmed Kebaier
[email protected]

HEC, Paris
Simulation of Random variables

Outline of The Talk

1 Simulation of Random variables

Simulation of Random variables

Outline

1 Simulation of Random variables

Simulation of Random variables

Random Number Generators I

Computer algorithms for generating random numbers are

deterministic algorithms. Although the sequence of numbers
produced by a random number generator appears random, the
sequence of numbers is completely predictable and for this reason
they are often called pseudo-random.
Simulation of Random variables

Random Number Generators I

Computer algorithms for generating random numbers are

deterministic algorithms. Although the sequence of numbers
produced by a random number generator appears random, the
sequence of numbers is completely predictable and for this reason
they are often called pseudo-random.

Since computers have only a finite number of different states,

the sequence of pseudo-random numbers produced by any
deterministic algorithm must necessarily repeat itself. The number
of random numbers generated before the sequence repeats is called
the period of the generator.
Simulation of Random variables

Random Number Generators II

A good random number generator should have the following

characteristics:
1 Randomness. It should pass statistical tests of randomness.
2 Long Period. For obvious reasons.
3 Efficiency. This is important since simulations often require
millions of random variables
4 Repeatability. It should produce the same sequence of
numbers if started in the same state. This allows the
repetition and checking of simulations.
Simulation of Random variables

Random Number Generators II

A good random number generator should have the following

characteristics:
1 Randomness. It should pass statistical tests of randomness.
2 Long Period. For obvious reasons.
3 Efficiency. This is important since simulations often require
millions of random variables
4 Repeatability. It should produce the same sequence of
numbers if started in the same state. This allows the
repetition and checking of simulations.
Almost all random number generators used in practice produce
uniform [0, 1] distributed random numbers and from these random
numbers with other distributions can be produced if required.
Simulation of Random variables

Algorithms
Linear Congruential Generators The simplest are the linear
congruential generators. Starting with the seed x0 , these generate
a sequence of integers by

xk = (axk−1 + c) mod M.

where a, c and M are given integers. All the xk are integers

between 0 and M − 1. In order to produce floating point numbers
these are divided by M to give a floating point number in the
interval [0, 1).
Simulation of Random variables

Algorithms
Linear Congruential Generators The simplest are the linear
congruential generators. Starting with the seed x0 , these generate
a sequence of integers by

xk = (axk−1 + c) mod M.

where a, c and M are given integers. All the xk are integers

between 0 and M − 1. In order to produce floating point numbers
these are divided by M to give a floating point number in the
interval [0, 1).
function [x] = lcg(n, a, c, m, x0)
x = zeros(1,n+1)
x(1) = x0
for i = 2:n+1
x(i) = pmodulo(a*x(i-1)+c, m)
end
x = x/m
endfunction
Simulation of Random variables

The following example illustrates a common problem with some

random number generators. We will take a = 1203, c = 0,
m = 2048. With initial value x0 = 1, this generator has period
512. We will run through a full period of the generator:
-->xx = lcg(511, 1203, 0, 2048, 1);
-->xx(1:10)
Simulation of Random variables

The following example illustrates a common problem with some

random number generators. We will take a = 1203, c = 0,
m = 2048. With initial value x0 = 1, this generator has period
512. We will run through a full period of the generator:
-->xx = lcg(511, 1203, 0, 2048, 1);
-->xx(1:10)

Now take successive pairs of values as the x and y coordinates of

a point in the plane and plot the results.
-->x = xx(1:2:511);
-->y = xx(2:2:512);
-->plot2d(x,y,style = -1)
Simulation of Random variables

The following example illustrates a common problem with some

random number generators. We will take a = 1203, c = 0,
m = 2048. With initial value x0 = 1, this generator has period
512. We will run through a full period of the generator:
-->xx = lcg(511, 1203, 0, 2048, 1);
-->xx(1:10)

Now take successive pairs of values as the x and y coordinates of

a point in the plane and plot the results.
-->x = xx(1:2:511);
-->y = xx(2:2:512);
-->plot2d(x,y,style = -1)

This “latticing” is a common problem with random number

generators. the points may tend to lie on a lattice of lower
dimensional subspaces.
Simulation of Random variables

Other Random Number Generators

random number generators are based on number theory and
have quite sophisticated implementations. One in common use is
the Mersenne twister which takes a vector of 625 as its seed.

Scilab has two random number generators rand and grand.

Uniform random numbers are the default. We have already seen
how rand works:
rand(m, n) - gives a m × n matrix of random numbers.
rand(m, n, ’normal’) - gives a m × n matrix of normally
distributed random numbers.
rand(a) or rand(a, ’normal’) - for matrix a gives a matrix
of ran- dom numbers the same size as a.
rand(’seed’, 0) - resets the random number generator to its
original state. This is handy if you want to repeat an
experiment using the same random numbers.
Simulation of Random variables

Some Common Distributions

Uniform distribution
If x is a uniform [0, 1] distribution, then the change of variable

y = (b − a)x + a

Here is an example generating a uniform [-5, 5] distribution:

-->x =rand(1,10000);
-->y = 10*x - 5;
-->histplot(100,y)
Simulation of Random variables

Some Common Distributions

Normal distribution
If x has a normal µ = 0, σ = 1 distribution, then the change of
variable
y = µ + σx
Here is an example generating a normal µ = 100, σ = 10
distribution:
-->x = rand(1,10000,’normal’);
-->y = 10*x + 100;
-->histplot(100,y)
Simulation of Random variables

Non-Uniform Random Numbers

Theorem 1
Let Y be a random variable with distribution function Fy . We
define for all x ∈ [0, 1]

FY−1 (y ) := inf{x ∈ R : FY (x) ≥ y }.

If U ∼ U[0,1] then FY−1 (U) has the same distribution as Y .

Remark:
1 Note that by construction we always have

∀y ∈ R y ≤ FY FY−1 (y ) .

2 If in addition F is continuous then F −1 coincides with the

classical inverse function and we get
∀y ∈ R y = FY FY−1 (y ) .


Simulation of Random variables

Exercise

Simulation of the Exponential Distribution

Let Y be random variable with Exponential distribution E(λ),
λ > 0. That is Y have a density f

0 x <0
f (x) =
λe −λx x ≥0

1 Compute the associated distribution function F and its inverse

G.
2 For λ = 2, simulate a sample of size N = 10000 of random
variables with distribution E(2)
Simulation of Random variables

Solution

-->y = rand(1,10000);
-->x = - log(1-y)/2;
-->histplot(100,x)

We can compare the histogram to exact density

-->xx = 0:0.01:5;
-->plot2d(xx, 2*exp(-2*xx))
Simulation of Random variables

Solution

-->y = rand(1,10000);
-->x = - log(1-y)/2;
-->histplot(100,x)

We can compare the histogram to exact density

-->xx = 0:0.01:5;
-->plot2d(xx, 2*exp(-2*xx))

Remark: One drawback of the inverse transform method is that

we need to know the inverse of the distribution function. For most
distributions there is no explicit formula for this inverse and we
must resort to approximations or seek another method.
Simulation of Random variables

Simulation of a discrete distributions

Let Y be a random varibale taking values in {yk , k ∈ N} and
such that
P(Y = yk ) = pk .
If U is a uniform random variable on [0, 1], then the random
variable X given by
X
X := y0 1{U≤p0 } + yk 1{Pk−1 pi <U≤Pk pi }
i=0 i=0
k=1

has the same distribution as Y .

Simulation of Random variables

Simulation of a discrete distributions

Let Y be a random varibale taking values in {yk , k ∈ N} and
such that
P(Y = yk ) = pk .
If U is a uniform random variable on [0, 1], then the random
variable X given by
X
X := y0 1{U≤p0 } + yk 1{Pk−1 pi <U≤Pk pi }
i=0 i=0
k=1

has the same distribution as Y .

Exercise
Generate uniformly distributed random integers in the range 1 to
10 (inclusive).
Simulation of Random variables

Simulation of a discrete distributions

Let Y be a random varibale taking values in {yk , k ∈ N} and
such that
P(Y = yk ) = pk .
If U is a uniform random variable on [0, 1], then the random
variable X given by
X
X := y0 1{U≤p0 } + yk 1{Pk−1 pi <U≤Pk pi }
i=0 i=0
k=1

has the same distribution as Y .

Exercise
Generate uniformly distributed random integers in the range 1 to
10 (inclusive).
Solution
-->x = rand(1,30);
-->y = floor(10*x+1)
Simulation of Random variables

Exercise
Simulate a Binomial distribution with parameter N and P. (N = 5
and p = 0.25).
Simulation of Random variables

Exercise
Simulate a Binomial distribution with parameter N and P. (N = 5
and p = 0.25). Solution
function [x]=s binomial(s,N,p)
// simulation binomial
// s = Sample size
// N,p = parameters
//-------------------------------
x=0*ones(1:s);
y=rand(s,N);
for i=1:s do ;
for j=1:N do ;
if y(i,j)< p then x(i)=x(i)+1;
elseif y(i,j) >= p then x(i)=x(i);
end;
end;
end;
enfunction -->exec(’s binomial.sci’,-1)
-->x=s binomial(1000,5,0.25)
Simulation of Random variables

Acceptance/Rejection Method
Suppose that ρ(x) is zero outside of the interval [a, b] and
furthermore that ρ(x) is bounded above by c.

Now generate points (xi , yi ) with xi uniformly distributed in

[a, b] and yi uniformly distributed in [0, c].
If yi ≤ ρ(xi ) then we accept the value xi , if yi ≥ ρ(xi ) then we
reject the value xi . The values xi which are accepted will have the
probability density ρ(x).
Simulation of Random variables

Exercise
Consider the hat-shaped probability density defined on [−1, 1] by

x +1 −1 ≤ x ≤ 0
ρ(x) =
1−x 0≤x ≤1

Write a Scilab function to generate n random numbers with this

density using the acceptance/rejection method.
Simulation of Random variables

Exercise
Consider the hat-shaped probability density defined on [−1, 1] by

x +1 −1 ≤ x ≤ 0
ρ(x) =
1−x 0≤x ≤1

Write a Scilab function to generate n random numbers with this

density using the acceptance/rejection method.
Remark. One thing to keep in mind when using the
acceptance/rejection is that we do not know before hand how
many random numbers we need to generate.
Simulation of Random variables

Solution
function [y] = rhohat(x)// First we need the density
function
if (x < 0) then
y = x + 1
else
y = 1 - x end
endfunction
Simulation of Random variables

Solution
function [y] = rhohat(x)// First we need the density
function
if (x < 0) then
y = x + 1
else
y = 1 - x end
endfunction
function [x] = randhat(n)// Now the random number
generator
x = zeros(1,n)
k = 0 // keep count of numbers generated
while (k < n)
xx = -1 + 2*rand(1,1) // uniform on [-1,1]
yy = rand(1,1)
Simulation of Random variables

Solution
function [y] = rhohat(x)// First we need the density
function
if (x < 0) then
y = x + 1
else
y = 1 - x end
endfunction
function [x] = randhat(n)// Now the random number
generator
x = zeros(1,n)
k = 0 // keep count of numbers generated
while (k < n)
xx = -1 + 2*rand(1,1) // uniform on [-1,1]
yy = rand(1,1)
if (yy <= rhohat(xx))// accept xx
k = k + 1
x(k) = xx
end
end
endfunction