MCE321:COMPUTER & COMPUTING II

MONTE CARLO METHODS

Engr. Prof. Okeniyi, Joshua O.

Engr Dr. Udoye N. Ekene
Mechanical Engineering Dept
1
Covenant University, Ota
Therefore Saul said unto the LORD…
Give a perfect lot.
And Saul and Jonathan were taken:
But the people escaped.
And Saul said,
Cast lots between me and Jonathan
my son. And Jonathan was taken.
(1Sam 14:41-42)
2
Learning Objectives:
After completing this lecture the students should be
able to:
❑ Describe Monte Carlo method
❑ Trace the origin of Monte Carlo method
❑ Give examples of the application of Monte Carlo
method

3
 INTRODUCTION
Monte Carlo methods (or Monte Carlo experiments) are a
class of computational algorithms that rely on repeated
random sampling to compute their results.
Monte Carlo methods are often used in simulating physical
and mathematical systems.
Useful in studying systems with a large number of coupled
(interacting) degrees of freedom, such as fluids, disordered
materials, strongly coupled solids, and cellular structures.
Monte Carlo methods are useful for modelling phenomena
with significant uncertainty in inputs, such as the
calculation of risk in business.
Widely successful method in risk analysis when compared
with alternative methods or human intuition. 4
 Practical Applications
• These methods are also widely used in mathematics:
a classic use is for the evaluation of definite
integrals, particularly multidimensional integrals
with complicated boundary conditions.
• Monte Carlo simulations have been applied in
– space exploration
– oil exploration
– actual observations of failures
– cost overruns
– schedule overruns
5
 Overview of Monte Carlo Method
A Monte Carlo algorithm is an algorithm for
computers.
It is used to simulate the behaviour of other
systems.
It is not an exact method, but a heuristic one.
Usually it uses randomness and statistics to
get a result.

6
 Monte Carlo algorithm
These methods tend to follow the algorithm below:
1. Define a domain of possible inputs.
2. Generate inputs randomly from the domain using
a certain specified probability distribution.
3. Perform a deterministic computation using the
inputs.
4. Aggregate the results of the individual
computations into the final result.

7
 Monte Carlo Example
To approximate the value of π using a Monte Carlo method:
1. Draw a square on the ground, then inscribe a circle within
it. From plane geometry, the ratio of the area of an
inscribed circle to that of the surrounding square is π / 4.
2. Uniformly scatter some objects of uniform size throughout
the square. For example, grains of rice or sand.
3. Since the two areas are in the ratio π / 4, the objects should
fall in the areas in approximately the same ratio. Thus,
counting the number of objects in the circle and dividing
by the total number of objects in the square will yield an
approximation for π / 4.
4. Multiplying the result by 4 will then yield an
approximation for π itself.
8
 Monte Carlo Example
Notice how the π approximation follows the general
pattern of Monte Carlo algorithms.
First, we define an input domain: in this case, it's the
square which circumscribes our circle. Next, we
generate inputs randomly (scatter individual grains
within the square),
Then perform a computation on each input (test
whether it falls within the circle). At the end, we
aggregate the results into our final result, the
approximation of π.

9
 Monte Carlo Example
Note, also, two other common properties of Monte Carlo
methods: the computation's reliance on good random
numbers, and its slow convergence to a better
approximation as more data points are sampled. If
grains are purposefully dropped into only, for example,
the center of the circle, they will not be uniformly
distributed, and so our approximation will be poor.
An approximation will also be poor if only a few grains
are randomly dropped into the whole square. Thus, the
approximation of will become more accurate both as
the grains are dropped more uniformly and as more are
dropped.
10
 Understanding The Monte Carlo Method
To understand the Monte Carlo method
theoretically, it is useful to think of it as a general
technique of numerical integration.
It can be shown, at least in a trivial sense, that
every application of the Monte Carlo method can
be represented as a definite integral.
Suppose we need to evaluate a multi-dimensional
definite integral of the form:
11 1
 =   f ( u1, u2 , , un ) du1du2 dun =  n f ( u )du (1)
00 0 ( 0,1)

11
 Understanding The Monte Carlo Method
Most integrals can be converted to this form with a
suitable change of variables, so we can consider this to
be a general application suitable for the Monte Carlo
method.
The integral represents a non-random problem, but the
Monte Carlo method approximates a solution by
introducing a random vector U that is uniformly
distributed on the region of integration. Applying the
function f to U, we obtain a random variable f(U). This
E  f (U ) =  f ( u )  ( u )du
has expectation:
(2)
( 0,1)n
Where ϕ is the probability density function of U. Because
ϕ equals 1 on the region of integration, this (2) becomes:
12
 Understanding The Monte Carlo Method
E  f (U )  =  n f ( u )du (3)
( 0,1)
Comparing (2) and (3), we obtain a probabilistic
expression for the integral Ψ:
 = E  f (U ) 
  (5)
so random variable f(U) has mean and some standard
deviation σ. We define
1
H = f (U ) (6)
as an unbiased estimator for Ψ with standard error σ. This is a
little unconventional, since (6) is an estimator that depends
upon a sample {U[1]} of size one, but it is a valid estimator
nonetheless. 13
 Understanding The Monte Carlo Method
To estimate Ψ with a standard error lower than σ, let’s
generalize our estimator to accommodate a larger
sample {U[1], U[2], …, U[m]}.
Applying the function f to each of these yields m
independent and identically distributed (IID) random
variables f(U[1]), f(U[2]), … , f(U[m]), each with
expectation and standard deviation σ. The
generalization of (6) :
1 m k 
H =  f (U ) (7)
m k =1

is an unbiased estimator for with standard error: m
14
 Understanding The Monte Carlo Method

If we have a realization {u[1], u[2], … , u[m]} for our

sample, we may estimate as:

1 m k 
h =  f ( u) (8)
m k =1

15
 
We call (7) the crude Monte Carlo estimator. m i.e. the
standard error, is important for two reasons. First, it tells us
that the standard error of a Monte Carlo analysis decreases
with the square root of the sample size. If we quadruple the
number of realizations used, we will half the standard error.
Second, standard error does not depend upon the
dimensionality of the integral (1).
Most techniques of numerical integration such as the
trapezoidal rule or Simpson's method suffer from the curse of
dimensionality. When generalized to multiple dimensions, the
number of computations required to apply them, increases
exponentially with the dimensionality of the integral. For this
reason, such methods cannot be applied to integrals of more
than a few dimensions.
16
The Monte Carlo method does not suffer from the curse of
dimensionality. It is as applicable to a 1000- dimensional
integral as it is to a one-dimensional integral. While
increasing the sample size is one technique for reducing the
standard error of a Monte Carlo analysis, doing so can be
computationally expensive. A better solution is to employ
some technique of variance reduction.
These techniques incorporate additional information about the
analysis directly into the estimator. This allows them to make
the Monte Carlo estimator more deterministic, and hence
have a lower standard error. Due to high mathematics
required and burden of understanding at this level, we have to
stop this discussion here.

17
3.1 History of Monte Carlo Method
Physicists at Los Alamos Scientific Laboratory were
investigating radiation shielding and the distance that neutrons
would likely travel through various materials.
Despite having most of the necessary data, such as the average
distance a neutron would travel in a substance before it
collided with an atomic nucleus or how much energy the
neutron was likely to give off following a collision, the
problem could not be solved with analytical calculations.
John von Neumann and Stanislaw Ulam suggested that the
problem be solved by modelling the experiment on a computer
using chance. Being secret, their work required a code name.
Von Neumann chose the name "Monte Carlo".

18
The name is a reference to the Monte Carlo Casino in Monaco
where Ulam's uncle would borrow money to gamble. Random
methods of computation and experimentation (generally
considered forms of stochastic simulation) can be arguably
traced back to the earliest pioneers of probability theory but
are more specifically traced to the pre-electronic computing
era.
The general difference usually described about a Monte Carlo
form of simulation is that it systematically "inverts" the typical
mode of simulation, treating deterministic problems by first
finding a probabilistic analogy. Previous methods of
simulation and statistical sampling generally did the opposite:
using simulation to test a previously understood deterministic
problem.
19
Though examples of an "inverted" approach do exist
historically, they were not considered a general
method until the popularity of the Monte Carlo
method spread.
It was only after electronic computers were first built
(from 1945 on) that Monte Carlo methods began to be
studied in depth.
In the 1950s they were used at Los Alamos for early
work relating to the development of the hydrogen
bomb, and became popularized in the fields of
physics, physical chemistry, and operations research.
The Rand Corporation and the U.S.
20
Air Force were two of the major organizations
responsible for funding and disseminating
information on Monte Carlo methods during this
time, and they began to find a wide application in
many different fields.

Uses of Monte Carlo methods require large amounts

of random numbers, and it was their use that
spurred the development of pseudorandom number
generators, which were far quicker to use than the
tables of random numbers which had been
previously used for statistical sampling.
21
 Applications of Monte Carlo Methods
a. Physical sciences
Monte Carlo methods are very important in computational
physics, physical chemistry, and related applied fields, and
have diverse applications from complicated quantum
calculations to designing heat shields and aerodynamic forms.
In experimental particle physics, these methods are used for
designing detectors, understanding their behaviour and
comparing experimental data to theory, or on vastly large scale
of the galaxy modelling.
Monte Carlo methods are also used in the models that form the
basis of modern weather forecasting operations.
22
b. Engineering
Monte Carlo methods are widely used in engineering
for sensitivity analysis and quantitative probabilistic
analysis in process design. The need arises from the
interactive, co-linear and non-linear behaviour of
typical process simulations. For example,
• In microelectronics engineering, Monte Carlo
methods are applied to analyze correlated and
uncorrelated variations in analog and digital
integrated circuits. This enables designers to estimate
realistic 3 sigma corners and effectively optimise
circuit yields.
23
• In geostatistics and geometallurgy, Monte Carlo
methods strengthen the design of mineral
processing flow sheets and contribute to
quantitative risk analysis.
c. Visual Designs
Monte Carlo methods have also proven efficient in
solving coupled integral differential equations of
radiation fields and energy transport, and thus these
methods have been used in global illumination
computations which produce photorealistic images
of virtual 3D models, with applications in video
games, architecture, design and computer generated
films. 24
d. Finance and business
Monte Carlo methods in finance are often used to
calculate the value of companies, to evaluate
investments in projects at a business unit or corporate
level, or to evaluate financial derivatives.
Monte Carlo methods used in these cases allow the
construction of stochastic or probabilistic financial
models as opposed to the traditional static and
deterministic models, thereby enhancing the treatment
of uncertainty in the calculation.

25
e. Telecommunications
When planning a wireless network, design must be
proved to work for a wide variety of scenarios that
depend mainly on the number of users, their locations
and the services they want to use.
Monte Carlo methods are typically used to generate
these users and their states.
The network performance is then evaluated and, if
results are not satisfactory, the network design goes
through an optimization process.

26
f. Games
Monte Carlo methods have recently been applied in
game playing related artificial intelligence theory.
Most notably the game of Battleship have seen
remarkably successful Monte Carlo algorithm based
computer players.
One of the main problems that this approach has in
game playing is that it sometimes misses an isolated,
very good move. These approaches are often strong
strategically but weak tactically, as tactical decisions
tend to rely on a small number of crucial moves
which are easily missed by the randomly searching
Monte Carlo algorithm.
27
Monte Carlo simulation versus “what if” scenarios
The opposite of Monte Carlo simulation might be
considered deterministic modelling using single-point
estimates. Each uncertain variable within a model is
assigned a “best guess” estimate.
Various combinations of each input variable are
manually chosen (such as best case, worst case, and
most likely case), and the results recorded for each so-
called “what if” scenario. By contrast,
Monte Carlo simulation considers random sampling of
probability distribution functions as model inputs to
produce hundreds or thousands of possible outcomes
instead of a few discrete scenarios.
28
For example, a comparison of a spreadsheet cost
construction model run using traditional “what if”
scenarios, and then run again with Monte Carlo
simulation and Triangular probability distributions
shows that the Monte Carlo analysis has a
narrower range than the “what if” analysis. This is
because the “what if” analysis gives equal weight
to all scenarios.

29
A randomized algorithm or probabilistic algorithm is an
algorithm which employs a degree of randomness as part of
its logic. The algorithm typically uses uniformly distributed
random bits as an auxiliary input to guide its behaviour, in
the hope of achieving good performance in the "average
case" over all possible choices of random bits.
Formally, the algorithm's performance will be a random
variable determined by the random bits; thus either the
running time, or the output (or both) are random variables
In common practice, randomized algorithms are
approximated using a pseudorandom number generator in
place of a true source of random bits; such an
implementation may deviate from the expected theoretical
behaviour.
30
g. Uses in mathematics
In general, Monte Carlo methods are used in
mathematics to solve various problems by generating
suitable random numbers and observing that fraction
of the numbers which obeys some property or
properties. The method is useful for obtaining
numerical solutions to problems which are too
complicated to solve analytically.
The most common application of the Monte Carlo
method in mathematics are:
i. Integration
ii. Optimization
iii. Inverse problems 31
Integration
Deterministic methods of numerical integration
usually operate by taking a number of evenly spaced
samples from a function. In general, this works very
well for functions of one variable. However, for
functions of vectors, deterministic quadrature
methods can be very inefficient.
To numerically integrate a function of a two-
dimensional vector, equally spaced grid points over a
two-dimensional surface are required. For instance a
10x10 grid requires 100 points. If the vector has 100
dimensions, the same spacing on the grid would
require 10100 points which is far too many to be
32
computed.
But 100 dimensions is by no means unusual, since in
many physical problems, a "dimension" is
equivalent to a degree of freedom.
Monte Carlo methods provide a way out of this
exponential time-increase. As long as the function in
question is reasonably well-behaved, it can be
estimated by randomly selecting points in 100-
dimensional space, and taking some kind of average
of the function values at these points.
By the law of large numbers, this method will
display convergence (i.e. quadrupling the number of
sampled points will halve the error, regardless of the
number of dimensions).
33
Optimization
Most Monte Carlo optimization methods are based on
random walks. The program will move around a
marker in multi-dimensional space, tending to move
in directions which lead to a lower function, but
sometimes moving against the gradient.
Another popular application for random numbers in
numerical simulation is in numerical optimization
(choosing the best element from some set of available
alternatives).
These problems use functions of some often large-
dimensional vector that are to be minimized (or
maximized). 34
Many problems can be phrased in this way: for
example a computer chess program could be seen as
trying to find the optimal set of, say, 10 moves which
produces the best evaluation function at the end.
The travelling salesman problem is another
optimization problem.
There are also applications to engineering design, such
as design optimization.

35
Inverse problems
Probabilistic formulation of inverse problems leads to
the definition of a probability distribution in the space
models.
This probability distribution combines a priori (prior
knowledge about a population, rather than that
estimated by recent observation) information with
new information obtained by measuring some
observable parameters (data).
As, in the general case, the theory linking data with
model parameters is nonlinear, the aposteriori
probability in the model space may not be easy to
describe (it may be multimodal, some moments may
36
not be defined, etc.).
When analyzing an inverse problem, obtaining a maximum
likelihood model is usually not sufficient, as we normally
also wish to have information on the resolution power of
the data. In the general case we may have a large number of
model parameters, and an inspection of the marginal
probability densities of interest may be impractical, or even
useless.
But it is possible to pseudorandomly generate a large
collection of models according to the posterior probability
distribution and to analyze and display the models in such a
way that information on the relative likelihoods of model
properties is conveyed to the spectator. This can be
accomplished by means of an efficient Monte Carlo
method, even in cases where no explicit formula for the a
priori distribution is available. 37
Computational mathematics
Monte Carlo methods are useful in many areas of
computational mathematics, where a lucky choice
can find the correct result.
A classic example is Rabin's algorithm for primality
testing (algorithm which determines whether a given
number is prime).
It states that for any n which is not prime, a random x
has at least a 75% chance of proving that n is not
prime.
Hence, if n is not prime, but x says that it might be,
we have observed at most a 1-in-4 event.
38
If 10 different random x say that "n is probably prime"
when it is not, we have observed a one-in-a-million
event.
In general a Monte Carlo algorithm of this kind
produces one correct answer with a guarantee that n is
composite, and x proves it so, but another one without,
but with a guarantee of not getting this answer when it
is wrong too often; in this case at most 25% of the
time.

39
Remark:
In physics, two systems are coupled if they are
interacting with each other.
Of special interest is the coupling of two (or more)
vibratory systems (e.g. pendula or resonant circuits)
by means of springs or magnetic fields, etc.
Characteristic for a coupled oscillation is the effect of
beat.

40
Conclusion:
Monte Carlo methods, relies on repeated computation
of random or pseudo-random numbers.
These methods are most suited to computations by a
computer and tend to be used when it is unfeasible or
impossible to compute an exact result with a
deterministic algorithm (i.e. an algorithm whose
behaviour can be completely predicted from the
input)

41
Tutor Marked Assignment
1. How is Monte Carlo method different in approach
from the typical mode of simulation, in deterministic
problems?
2. How is Monte Carlo method used in Engineering
and Mathematics?

42
The End

43