IT2032

Output Analysis for Simulation Model o Bias – This pertains to the difference between the expected value of
Output Analysis the estimator and the value of the parameter being estimated. If the
• Output analysis is the analysis of data generated by a simulation run to estimated value and the value of the parameter being estimated are
predict or compare the performance of two (2) or more system designs. In equal, then the estimator is considered as unbiased.
stochastic simulations – a simulation of a system that has variables that o Consistency – This states how close is the point estimator to the
can change randomly with individual probability – multiple runs are always value of the parameter as it increases in size. The point estimator
necessary. requires a large sample size for it to be more consistent and accurate.
• Output analysis is performed since the output data from a simulation o Efficiency – The efficiency of the estimator depends on the
exhibits random variability when random or pseudorandom generators are distribution of the population. The most efficient point estimator is the
in use. one with the smallest variance of all the unbiased and consistent
• Output analysis can determine the estimate of the mean and variance of estimators.
random variables or the number of observations required to achieve a • Point estimation for discrete-time data: [𝑌𝑌1 , 𝑌𝑌2 , … , 𝑌𝑌𝑛𝑛 ] with ordinary mean 𝜃𝜃
desired estimate precision. (theta)
• The output data of simulation models are used to help evaluate
performance measures – reflect the characteristics of the stochastic
process being modeled.
• The basic output data from a simulation run is called the trace.
o It is biased if:
• The most common measures used to represent the results of simulation
runs are the following (Garrido, 2009):
o The mean, also known as the average, gives the fundamental o It is unbiased if the expected value of 𝜃𝜃 is: (this condition is desired)
characteristics of the output data
o The standard deviation, to summarize the variability of the output data
o The frequency plot, also known as the histogram, represents the Where:
distribution of the output data Y – is a specific value relative to a simulation run
Note: The mean and the standard deviation can be computed by n – is the number of simulation run(s)
accumulating values of the random variable while the simulation run is E – is a specific event that stops the simulation
carried out. • Point estimation for continuous-time data: {𝑌𝑌(𝑡𝑡), 0 ≤ 𝑡𝑡 ≤ 𝑇𝑇𝐸𝐸 } with time
weighted mean 𝜙𝜙 (phi)
In output simulation for simulation models, there are two (2) methods to
statistically analyze the collected simulation output, which are point estimation
and interval estimation. In most cases, both methods are needed for a
meaningful data interpretation.
o The point estimator is generally biased.
Point Estimation o An unbiased or low-bias estimator is still desired.
Where:
• Point estimators are functions that are used to find an approximate value
TE – is the period of time or duration of the simulation run
of a population parameter from random samples in the population. It
Y(t) – is a function representing replication(s), relative to time (t)
produces a single value.
• The following are the main characteristics of point estimators (Corporate
Finance Institute®, n.d.):

06 Handout 1 *Property of STI

 [email protected] Page 1 of 3
 IT2032
Interval Estimation • The mathematical representation of prediction interval is (Banks, Carson,
• In interval estimation, the desired parameter to be estimated lies between Nelson & Nicol, n.d.):
the two (2) values, l and u, that define the interval with a given probability.
The interval [l, u] is called the confidence interval.
• Confidence interval is the statistical tool mainly used in output analysis.
It displays the probability that a parameter will fall between a pair of values
around the mean (Hayes, 2021). Where:
• Confidence interval is an interval estimate for a parameter that specifies 𝑌𝑌 : sample estimate or predicted value
the level of confidence that provides a way of quantifying imprecision. Its ± 𝑡𝑡𝛼𝛼⁄2,𝑅𝑅−1 : the t-multiplier
goal is to form an interval with endpoints determined by the samples, that
will contain or “cover” the target parameter with pre-specified (high) 1
𝑆𝑆�1 + : standard error of prediction
probability called the confidence level. 𝑅𝑅
• The Central Limit Theorem (CLT) states that as the number of samples
(n ≥ 30) increases, the distribution of the mean will be approximately
normal. The following must be observed in applying CLT (Garrido, 2009): Types of Simulation with Respect to Output Analysis
o Each single run of a stochastic simulation model can be considered The type of simulation, either terminating or non-terminating, greatly depends
as a single sample. on the following:
o Each independent model replication, where replications are performed o The objective(s) of the modeling and simulation study.
using random numbers, produces another sample point. o The nature of the system.
• The mathematical representation of confidence interval is (Banks, Carson,
Nelson & Nicol, n.d.): Terminating Simulation
• A terminating simulation is carried out to study the behavior of a system
over a particular time interval.
• The simulation starts and ends at a defined state or time.
• Majority of service systems are modeled as terminating systems.
Where: • The analysis of terminating simulations involves multiple runs using
𝑌𝑌 : sample mean different seeds for the random or pseudorandom number generations.
± 𝑡𝑡𝛼𝛼⁄2,𝑅𝑅−1 : the t-multiplier • The data is gathered for successive time intervals during the simulation
𝑆𝑆 period (Garrido, 2009).
: standard error Examples:
√𝑅𝑅
1. A car manufacturer receives a contract to produce 120 cars, which
• Prediction interval is a measure of risk or uncertainty. It is a range of
must be delivered within 18 months. The company would like to
values that is likely to contain the value of a single new observation given
simulate various manufacturing configurations to see which can meet
a specific setting or initial condition.
the delivery request at the least cost.
• Prediction intervals predict the spread for individual observations rather
2. A shop that sells a single product would like to decide how many items
than the mean. The interval in this estimation method is wider than
they should have in the inventory for the next 120 months. Given some
confidence interval due to the uncertainty involved in predicting a single
initial inventory data, the objective is to determine the monthly order
response.
to minimize the expected averaging cost per month in the inventory
system.

06 Handout 1 *Property of STI

 [email protected] Page 2 of 3
 IT2032
Corporate Finance Institute® (n.d.). Point estimators. Retrieved on April 13, 2021 from
• The following are some possible statistical analysis formula, applicable for https://corporatefinanceinstitute.com/resources/knowledge/other/point-estimators/
terminating simulations (Noche, 2014): Banks, Carson, Nelson & Nicol. (n.d.). Chapter 11: Output analysis for a single model. Retrieved April 10, 2021 from
https://cs.wmich.edu/~alfuqaha/Spring09/cs6910/lectures/Chapter11.pdf
o Unbiased estimator for the population mean (µ) Noche, B. (2014, November 26). Output analysis for simulation model. Retrieved on April 10, 2021 from https://www.uni-
due.de/imperia/md/content/tul/download/ews2014_2015_sl02_im_output_analysis.pdf
Garrido, J. (2009). Object oriented simulation a modeling and programming perspective. Springer Science+Business Media, LLC
Hayes, A. (2021, March 31). Confidence interval. Retrieved on April 11, 2021 from
https://www.investopedia.com/terms/c/confidenceinterval.asp

o Sample variance

o Confidence interval for the population mean (µ)

Non-Terminating Simulation
• A non-terminating simulation is carried out to study the steady-state and/or
long-term average behavior of a system.
• The long-term average behavior of the system is analyzed by calculating
the adequate length of the simulation period.
• The simulation runs are performed to gather data for the statistical analysis
of the steady-state behavior of the system.
• The simulation runs begin with a warm-up state, also known as transient
state – the output process for the initial condition at a discrete time, and
gradually moves to a steady state – shows approximately the same
distribution of random variables from a specific point (Garrido, 2009).
Examples:
1. Telecommunications systems
2. Assembly lines that often halt operations
3. Emergency rooms in hospitals

• The following are some possible statistical analysis method, applicable for
non-terminating simulations:
o Welch Method
o Replication-Deletion Approach
o Batch Mean Method

References:
Chaturvedi, D. (2010). Modeling and simulation of systems using MATLAB and Simulink. CRC Press – Taylor & Francis Group, LLC

06 Handout 1 *Property of STI

 [email protected] Page 3 of 3