SIMULATION AND MODELLING

Sujan Karki
Email: [email protected]
Contact No.: 9819387234
Master’s in Information System Engineering (MscIne) *Ongoing
Bachelor in Computer Engineering – Purwanchal Campus,IOE
 UNIT 8

Analysis of Simulation Output

Analysis of simulation output

1. Estimation methods
2. Simulation run statistics
3. Replication of runs
4. Elimination of Initial bias

2
 Introduction
o Output analysis is the examination of data generated by a
simulation. As in stochastic simulations, multiple runs are always
necessary.
o Its purpose is either to predict the performance of a system or to
compare the performance of two or more alternative system designs.
Why analysis of simulation output ?
• Many simulation includes some sort of randomness, which can arise in
a variety of ways e.g. in a simulation of manufacturing system, the
processing times required at a station may have random variations or
the arrival times of new jobs may not be known in advance.
• In a bank, customers arrive at random times & amount of time spent at
a teller is not known beforehand. Because of the randomness of the
components driving a simulation, the o/p from simulation is also
random. So, statistical techniques must be used to analyze the results.
• To test different ideas, to learn about the system behavior in new
situation, to learn about simulation model and the corresponding
3
simulation system.
 Nature of Problem
1. Once a stochastic variable is introduced into a simulation model,
almost all the system variables describing the system behavior also
become stochastic. The values of most of the system variables will
fluctuate as the simulation proceeds so that no one measurement can
be taken to represent the values of a variable.
2. Instead many observations of the variable values must be made in
order to make a statistical estimate of its true values. Some statement
must also be made about the probability of the true value falling
within the given interval about the estimated value. Such a statement
defines a confidence interval, without it simulation result are of little
value to the system analyst.
3. A large body statistical method has been developing over the years to
analyze results in science, engineering and other fields where
experimental observation is made. So, because of the experimental
measurements of the system of simulation for these statistical
methods can be adapted to simulation results to analyze.
4
 Types of Simulation on the basis of
Output
1. Termination simulation or finite simulation:
The termination of a finite simulation takes place at a specified
time or is caused by some specific events.
For example: Banks opens at 8:30am with no customers
present and 8 of 11 teller working and closed at 4:30 pm. The
terminating simulation runs for some specified duration of time
TE where E is a specified event that stops the simulation. It
starts at time 0 under well specified initial condition and ends
at the stooping time TE . For the banking system, the
simulation analyst chooses to consider a terminating system
because object of interest is one day's operations on the bank.

5
 2. Non terminating Simulation (Steady state simulation):
The main propose of steady study simulation is the study of
long run behavior of system. Performance measure is called a
steady state parameter if it is a characteristic of the
equilibrium distribution of an output stochastic process.
For example: Continuously operating communication system
where the objective of computation of mean delay of packet in
the long run.
Hospital emergency rooms, Network of routers, Internet,
Assembly lines that shut down infrequently etc.
Here the non-terminating simulation has:
1. Runs continuously or at least over a very long period of
time. 2. Initial conditions defined by analyst.
3. Study the steady state (long run) properties of the system.

Note: whether a simulation is considered to be terminating or non-

6
terminating depends on both the objective of study and nature of the
system.
 Central limit theorem
Given X1, X2, X3, …..Xn are independent and identically
distributed(IID) with mean μ and standard deviation σ .

S= X1+X2+X3+…+Xn
x̄ = (X1+X2+X3+…+Xn) / n
S and x̄ are normal for large n.
If X is general normal with mean μ and standard deviation σ then
Z is the standard normal then ,

7
 o For SUM:
o In terms of S:

8
 Central limit theorem
For MEAN:
The central limit theorem states that if you have a population with
mean μ and standard deviation σ and take sufficiently large random
samples from the population with replacement, then the distribution
of the sample means will be approximately normally distributed.
This will hold true regardless of whether the source population is
normal or skewed, provided the sample size is sufficiently large
(usually n > 30).

9
1
0
 Estimation Method
o It is used for point estimation for discrete data and continuous data, and
also to estimate the range of random variable so that desired output can
be achieved.
o Interval Estimation/Confidence Interval Estimation:
Given X1, X2, X3, …..Xn are independent and identically distributed(IID) with
mean μ and standard deviation σ.

So from central limit theorem:

Where, s is the sum of all random variables.

Also, in term of sample mean.

1
1
 The probability that Z lies between - μ α/2 and μ α/2 is 1- α.
Prob {- μ α/2 ≤ Z ≤ μ α/2 } = 1- α
Prob {- μ α/2 ≤ ≤ μ α/2 } = 1- α
On simplification,
Prob { x̄ + ) μ α/2 ≥ x̄ - ) μ α/2 } = 1- α

Here, the 1- α is the confidence interval and the confidence interval is

x̄ + ) μ α/2
1
2
 In practice, the population variance is usually not known, in this case
variance is replaced by the estimated calculated by the formula.

Also In t-distribution, with n – 1 degree of freedom. In terms of estimated

variance σ2 , the confidence interval for 𝑥 is defined by

Here, the quantity t(n-1, α/2) is found on the student t- distribution

table.

Hence, the estimation method gives the desired range of the sample
variables taken from the infinite population.
1
3
1
4
 Simulation Run statistics
In the estimation method, it is assumed that the observations are
mutually independent and the distribution from which they are
draws is stationary.
Unfortunately many statistics of interest in simulation do not
meet these conditions.
•For example: consider a single server system in which the
arrival occurs with poison distribution and service time has an
exponential and queue discipline is FIFO.
Where, Poisson distribution means discrete frequency
distribution which gives the probability of a number of
independent events occurring in a fixed time.
First problem: Suppose the study objective is to measure the
mean waiting time.
15
 o In simulation run, the simplest approach to estimate the mean waiting
time by accumulating the waiting time of n successive entities and
dividing by n. this is the sample mean, denoted by Waiting time
measured in this way is not independent

o Whenever a waiting line forms, the waiting time of each entity on the
line clearly depends upon the waiting time of its predecessors. Such
data are called auto-correlated. Any series of data that has the property
of having one value affect other values is said to be auto correlated.
The usual formula for estimating the mean value of distribution
remains same estimate for the mean of auto correlated data. However
the variance of autocorrelated data is not related to population
variance by simple expression 2) as occurs for independent data. A term
is added for auto correlation. The term is +ve in the case of single
server system , so that if it is ignored, the variance is underestimated.
16
But in other system, it can be negative resulting in overestimated.
 o The sample mean of auto correlated data can be shown to
approximate a normal distribution as the sample size increases.

o Another problem that must be faced is the distribution may not be

stationary. It is because a simulation run is started with the system
with some initial state, frequently idle state, in which no service is
being given and no entities are waiting initially thus early arrivals
have a more probability of obtaining service quickly.
o So , sample mean that include the early arrivals will be biased. As
the length of simulation run extended and the sample size increases,
the effect bias will be minimum. This is shown as in the figure
below:

17
 Replication of Runs
One way of obtaining independent result is to repeat simulation.
Repeating the experiment with different random numbers for the sample
size n gives a set of independent determination of sample mean 𝑥 .
Suppose the experiment is repeated p times with independent random
values of n sample sizes.

18
1
9
2
0
 o The length of run of replication is so selected that all compiled
combined comes to sample size 𝑁 i.e. 𝑝. 𝑛 = 𝑁, where 𝑝 = No. of
sample 𝑛 = No. of observation per sample
o By measuring the number of replications and shortening their length
of run, confidence interval can be narrower but due to shortening of
length of replication the effect of starting condition will increase.
o The result obtained won’t be accurate especially when the
initialization of the runs is not proper. There is not established
procedure of dividing the sample size 𝑁 into replications.
o However it is suggested that the number of replications should not
be very large and that the sample means should approximate in
normal distribution.

21
 Elimination of Initial Bias
Two general approaches can be taken into remove the bias.
1. The system can be started in a more representative then an empty state.
2. The first part of the simulation can be removed.
The ideal situation is to know the steady state distribution for the system and
select the initial condition for the distribution.
1. The more common approach to know the initial bias is to eliminate on initial
selection of runs.
2. The run is started from an idle state and stop after the certain period of time.
3. The entities in the system at that time are left as they are.
4. The run is then restarted with statistics being gathered form the point of
restart.
These approaches have the following difficulties:
5. No simple rules can be given to deciding how long an interval should be
eliminated. For this It is advisable to use pilot runs starting form idle state to
judge how long the initial bias remains. This can be done by plotting the
measured statistic against run length.
6. The disadvantage of eliminating the first part of the simulation run is that the
2
estimate of the variance needed to estimate a confidence limit, mostly based
2 on less information
 Assignments
1. Explain the replication of runs for analysis of simulation outputs. [5]
2. Define initial Bias. Explain elimination of initial bias with example. [6]
3. What are the various methods used to analyze simulation output? Explain
any one of them. [5]
4. How is estimation method used in simulation output analysis? Where we
carl apply this method? [5+2]
5. How you can analyze simulation output using simulation runs statistics?
Explain. [6]
6. ln the case of infinite population which output analysis method is
applicable. Why? Explain. [6]
7. Why an analysis of simulation output is important? [2]

2
3
 . . . to be continued !!!

24