SimRunner User Guide
SimRunner User Guide
9/22/10
SimRunner iii
User Guide
Table of Contents
Introduction ....................................................................................1
Chapter 1:
SimRunner Concepts ...........................................................................................5
Chapter 2:
Getting Started ...................................................................................................13
Chapter 3:
Building Projects ................................................................................................31
Appendix ......................................................................................41
Glossary ........................................................................................43
Bibliography .................................................................................47
Index .............................................................................................49
SimRunner 1
User Guide
Introduction
Chapter 1 Bibliography
SimRunner Concepts
Referenced materials.
As an introduction to simulation optimization,
this chapter begins by reviewing some of the
theory behind optimization. The remainder of the
discussion revolves around the basic approach to
building, running, and examining results from
models.
Chapter 2
Getting Started
Introduces the product interface and allows you
to become more familiar with the SimRunner
environment. Contains an in-depth discussion
and explanation of each step you must follow to
build, run, and examine an optimization project.
Chapter 3
Building Projects
Steps you through the process of creating a
project.
2
About the User Guide
Chapter 1:
SimRunner Concepts
Typically, most people use simulation tools to Unlike manual simulation experimentation which
predict and improve system performance or to answers only specific what-if questions,
establish the relationships between various SimRunner automatically seeks a course of
system elements. By modeling the actual facility action that will help optimize your entire
or process, you can conduct what-if analyses to system’s performance—essentially answering
determine the best way to improve system how to meet your objectives.
performance—this is optimization. Although To help you present your findings, SimRunner
SimRunner cannot guarantee it will identify the allows you to output various types of reports:
optimal solution for all processes every time, it
will find better solutions than you would likely • Data reports (for spreadsheets)
get on your own. • Analysis reports (for text and word pro-
cessing reports)
With each optimization project, SimRunner runs
• Graphical reports (for charts and graphs)
sophisticated optimization algorithms on your
model to help you optimize multiple factors You can print each of these reports or export them
simultaneously. Each project requires a validated to other applications through common clipboard
model, a standard by which to measure system functions.
performance (an objective function), and a group
of factors that SimRunner may change to
improve system performance.
6 Chapter 1: SimRunner Concepts
Where Do I Begin?
Example
To help you better understand how SimRunner’s
optimization process works, consider the
following analogy. If you and a group of
explorers found yourselves on the slopes of a
mountain, in the dark, with nothing but radios
and altimeters, how would you find the summit?
þ À Œ œ
à Õ
Profile view
8 Chapter 1: SimRunner Concepts
Where Do I Begin?
The first step would be to establish the current Again, after everyone moves various distances,
altitude of everyone in your group by recording you stop to take another altimeter reading.
each person’s altimeter reading.
œ
œ Œ
À À
þ Ã
Œ Õ
þ
à Topographical view: Reading 3
Õ
Topographical view: Reading 1
By taking the average of the group’s new
Next, you would direct your group members to altimeter readings, you can see that the overall
wander out in any direction for various distances, reading increased. This confirms that you are
then stop and review their new altimeter moving in the right direction. Just to be sure, you
readings. At the second reading, you find that repeat the process and take another reading.
some of your group has moved higher and some
have moved lower.
œ
œ þ Œ
ÃÀ Õ
À Œ
Õ
þ Ã Topographical view: Reading 4
Topographical view: Reading 2 As you will notice from the example, your group
is beginning to converge upon a single point. The
By comparing the results of each altimeter more you repeat the data collection process, the
reading, you can determine the general direction closer the group will get to one another, and the
in which to proceed. Since you want to climb to more certain you will be that you are at or near
the summit, you proceed in the general direction the summit. Just to be on the safe side, however,
that had the highest reading (in this case, 2). you even send a few members of your group out
to remote parts of the terrain to be sure that there
SimRunner 9
User Guide
is nothing higher than the point you already value would take a lifetime. More than likely,
identified. you will never get the answer.
ÃŒœ
þÀÕ
1.2.3 Pitfalls
If you follow the general procedure stated
previously, your chances of success are very
good. Typically, projects fail because the:
• Model is not valid
• Analysis considers insignificant factors
• Analysis ignores significant factors
• Objective Function is inappropriately for-
mulated
• Test results are not scrutinized
12 Chapter 1: SimRunner Concepts
Where Do I Begin?
SimRunner 13
User Guide
Chapter 2:
Getting Started
Phase 1, 2, 3
Steps for
current phase
14 Chapter 2: Getting Started
Set Up a Project
Response category
The response category is the type of statistic you
wish to use to evaluate your model. Response
Create new project / Select model Enter the name
categories include model elements such as
of the model you wish to optimize. You will enter
locations, entities, resources, and variables.
a name for the project when you save it.
Open existing project / Select project Select an
existing project. If you open an existing project, it
is not necessary to select the model you will
use—the model is stored as part of the
optimization project.
SimRunner 15
User Guide
Response statistics
Simply put, response statistics are those values
you wish to improve. Once you define your
targeted improvements, you are ready to define Please note
how you want them to perform—the objective for
the response statistic.
Maximized objective functions return positive val-
ues. Minimized objective functions return negative
values.
WIP to make them of equal weight in the combination and displays the updated objective
objective function. function for the project.
Maximize[(W1)*(Throughput)] = 4.56
Minimize[(W2)*(WIP)] = 4.56
4.56
Objective function
.72
The objective function is an expression used to
quantitatively evaluate a simulation model’s
performance. By measuring various performance
characteristics and taking into consideration how
In situations where it is necessary to favor one
you weigh them, SimRunner can measure how
statistic over another, balancing the statistics first
well your system operates. However, SimRunner
will make it easier to control the amount of bias
knows only what you tell it via the objective
you apply. For example, if you apply a weight of
function. For instance, if your objective function
12.67 (W1=12.67) to throughput and 1.0
measures only one variable, Total_Throughput,
(W2=1.0) to WIP, the objective function will
SimRunner will attempt to optimize that variable.
consider throughput to be twice as important as If you do not include an objective function term
WIP (adapted from Harrell, Ghosh, and Bowden to tell it that you also want to minimize the total
2000). number of operators used, SimRunner will
assume that you don't care how many operators
you use. Since the objective function can include
multiple terms, be sure to include all of the
response statistics about which you are
concerned.
Please note
SimRunner’s capacity to include many different
Typically, you will need to experiment with your response statistics in an objective function gives
model to identify the weight ratio necessary to it tremendous capability. For example, the
balance statistics. objective function below signifies that you wish
to maximize the total ovens and cooktops
processed while minimizing the total resource
cost. The numeric weighting factors indicate that
maximizing Total Ovens Processed is the most
Response statistics selected for
important, followed by maximizing Total
objective function Cooktops Processed, then minimizing Total
After you define the objective for the response Resource Cost.
statistic, you may click the Add button to include Z =Max:10 * (Total Ovens Processed) +
the statistic as part of the objective function. Max:5 * (Total Cooktops Processed) +
SimRunner combines the statistics into a linear Min:2 * (Total Resource Cost)
SimRunner 17
User Guide
Perhaps the best way to define your objective 2.2.3 Define Inputs
function is in terms of cost or profit. When
possible, this allows you to use a simple, single In every system, there are controllable and
response statistic like maximize profit without uncontrollable factors that determine the
concern over assigning meaningful weights to outcome of the process. Controllable factors
multiple response statistics. include staffing, equipment, schedules, and
facilities. Uncontrollable factors refer to such
things as arrival rates.
Often, relationships exist between various
controllable factors. How do you identify these
relationships and use them to seek the best value
Example: Maximize up to a target for your objective function?—SimRunner.
SimRunner allows you to target controllable
The following example shows how to achieve a model factors and determine which combination
throughput target level. Suppose you want to target a of values for those factors will elicit the behavior
production rate of 300 to 325 units per day. In the tar- you desire.
get range fields, assign a range of 300 to 325 and enter
a weight of 1. With each test it performs, SimRunner examines
the results to see if the results will produce the
effect you require from the objective function.
For information about how SimRunner performs
Log time these tests and evaluates data, see Concepts and
Because the LOG statement does not have a Theories on page 7.
separate data structure (it occurs during the
processing logic), SimRunner does not have
direct access to log data. However, if you create a
variable to represent the logged time, you can use
the variable as part of the objective function.
value of your objective function. Typically, these few inputs as possible—only those you anticipate
are the controllable factors of your model. will affect the value of the objective function.
Variables
Data type Refers to the numeric type of the data If you want to use a variable from your model as
(integer or real) SimRunner will use. Typically, an input factor, you must define a corresponding
you will use integers to represent the number of macro and set the variable equal to the macro at
resources (e.g., people) and real numbers to some point in your model. To do this, you can:
represent time values or percentages (e.g.,
• Enter the name of the macro in the vari-
machine processing times or product mix).
able’s “initial value” field.
Default value Refers to SimRunner’s initial • Set the variable equal to the macro in the
setting for the macro—the value SimRunner will model’s processing logic.
use to analyze the model before conducting the • Set the variable equal to the macro in the
optimization. See Analyze Model on page 22. model’s initialization logic.
Lower and upper bound The limits (constraints)
within which this value must fall during Resource & location capacities
subsequent tests. If you defined these bounds in In order to use a resource or location capacity as
an RTI for the macro, you may override them an input factor, you will need to define a macro
here if you wish. that represents the capacity or number of
resources and place this macro in the appropriate
Macros selected as input factors field for the resource or location. For instance, if
you have a resource named Operator_1, you can
Displays a list of all macros you selected for use create a macro named Number_Of_Operators to
as input factors. Typically, you will want to use as represent the “number of units” for the operator.
When you run SimRunner, the macro
Number_Of_Operators will appear on the list of
input factors. Testing this factor will change the
number of units of Operator_1 in the model.
SimRunner 19
User Guide
Please note
Distribution parameters
Since macros must be completely numeric in the
"text" field for SimRunner’s use, macros that
represent distributions will not appear in the list
of available input factors. A macro can be used to
represent a distribution parameter. For example, Optimization options
suppose you have a normal distribution, N(10,2),
with an average of 10 and a standard deviation of
2. You can create a macro, Dist_1_Average, with
a value of 10 and use this macro as the text value
of the distribution: N(Dist_1_Average, 2). This
will allow you to manipulate the distribution
from SimRunner. Optimization profile SimRunner provides three
optimization profiles: Aggressive, Moderate, and
Cautious. The optimization profile is a reflection
of the number of possible solutions SimRunner
will examine. For example, the cautious profile
tells SimRunner to consider the highest number
Please note of possible solutions—to cautiously and more
thoroughly conduct its search for the optimum.
Using ProModel, MedModel, or ServiceModel, As you move from aggressive to cautious, you
you can set up macros in your simulation model will most often get better results because
as run-time interfaces (RTI’s). SimRunner reads SimRunner examines more results—but not
the RTI’s as the default values for the input fac- always. Depending on the difficulty of the
tor’s lower and upper bounds. If you do not problem you are trying to solve, you may get
define RTI’s, SimRunner will use the factor’s cur- solutions that are equally good. If you are pressed
rent value as the lower and upper bounds. for time or use relatively few input factors, a
more aggressive approach—the aggressive
profile—may be appropriate.
20 Chapter 2: Getting Started
Set Up a Project
Please note
2.3 Analyze Model does not reflect reality. When you run a non-
terminating simulation, you must run it long
The first step in conducting any analysis of your enough for the operation to stabilize—to reach
simulation model’s output is to make sure that steady state. The warm-up time is how long it
your model produces output with the degree of takes the simulation to reach steady state. The
accuracy you desire. If all of the aspects of your end of the warm-up time is the point at which you
model are deterministic (with no random want to begin sampling statistical data—data
variance in any part of the model), you will get a representative of how the system normally
precise output statistic with one replication. In operates.
most models, however, random variance is
present in several places. The result of using
random inputs in the model is that the output is
also random. Because of this, you should think of
a simulation experiment as a sampling technique.
Since you don’t know the exact, true value or Please note
population statistic, the best you can do is collect
a sample large enough to make a good estimate— When you define a warm-up period, the model
you must run multiple replications. runs for a specific period of time at the beginning
In general, with each additional replication you of each replication before results are recorded.
run, you generate a more accurate estimate of the
population statistic your are trying to measure in
the objective function. For example, suppose you
wish to maximize the utilization of a given How do you know how long the warm-up period
resource by changing several factors in the should be? SimRunner uses an approach based
model. For each experiment, you may need to run on Welch’s graphical method. SimRunner
ten replications to get an accurate estimate of the displays a time series graph of the objective
utilization for a specific combination of settings. function’s current value and moving average
If your estimate isn’t accurate enough, you will based on several replications of the simulation.
not be able to tell if improvements to a system are As the objective function begins to stabilize, the
the result of changes or simply an effect of moving average graph will appear to "flatten out"
random variance. SimRunner helps you select the around a specific value or range. This behavior
appropriate number of replications for both signals the end of the warm-up period.
terminating and non-terminating systems. It is quite possible that you may be unable to
For non-terminating systems, SimRunner’s determine the warm-up period from just one look
Objective Function Time Series graph shows the at the graph. If you are unable to make this
value of the objective function as it changed determination, run the model again with a longer
during the simulation based on several run length.
replications. At the very beginning of a
simulation run, the simulation model is “empty.”
There are no entities in the system and often all
the statistics are set to zero. This is an artificial
condition and, for non-terminating simulations,
SimRunner 23
User Guide
Start analysis
2.4.1 Introduction
Example
Often, the reason for building a simulation model
of a system is to answer questions such as: “What The following diagram shows the relationship between
are the optimal settings for to minimize an optimization algorithm and a simulation model
(or maximize) ?” You can think of the (adapted from Harrell, Ghosh, and Bowden 2000).
simulation model as a black box that imitates the
actual system. When you present inputs to the
black box, the box produces outputs that estimate Input: (x1, x2, x3, ...)
Optimization Simulation
how the actual system will respond. In the Algorithm Model
question above, the first blank represents the
input(s) to the simulation model that you control.
(These inputs are often called decision variables
or factors.) The second blank represents the Output Response: (performance measure =y)
performance measure(s) of interest, computed
from the output response of the simulation model
when you set the factors to specific values (see If you were to evaluate all combinations of the differ-
ent values for the input factors (X1, X2, X3, …) and
example below).
plot the output response generated by the simulation
In the question, “What is the optimal number of model, you would create what is called a response sur-
material handling devices needed to minimize the face of the output for that model. Think of the response
time that workstations wait for material?”, the surface as a mountainous region with many peaks and
input factor is the number of material handling valleys. In the case of a maximization problem, you
devices and the performance measure is the time wish to climb to the highest peak. In the case of a min-
imization problem, you wish to descend to the lowest
that workstations wait (computed from the output
valley.
response of the simulation model). The goal is to
locate the optimal value for each factor that
minimizes or maximizes the performance
measure of interest.
26 Chapter 2: Getting Started
Optimize Model
SimRunner intelligently and reliably seeks the Performance plot Plots the best objective
optimal solution to your problem based on the function value found throughout the optimization
feedback from your simulation model by process.
applying some of the most advanced search Final report The final report contains several of
techniques available today. The SimRunner the best solutions found. SimRunner sorts these
optimization method is based upon Evolutionary solutions to allow you to evaluate them further
Algorithms (Goldberg 1989, Fogel 1992, and before you make your final decision. These
Schwefel 1981). results can be saved to a file using the Export
Evolutionary Algorithms are a class of direct Optimization Data option found in the File menu.
search techniques based on concepts from the
theory of evolution. The algorithms mimic the Convergence status
underlying evolutionary process in that entities
adapt to their environment in order to survive.
Evolutionary Algorithms manipulate a
population of solutions to a problem in such a
way that poor solutions fade away and good
solutions continually evolve in their search for Phase 1 and Phase 2 SimRunner sometimes uses a
the optimum. Search techniques based on this genetic algorithm before using the evolution
concept have proven to be very robust and have strategies algorithm based on the characteristics
solved a wide variety of difficult problems. They of the problem being solved. Phase 1 displays the
are extremely useful because they provide you convergence status of the genetic algorithm.
with not only a single, optimized solution, but Phase 2 displays the convergence status of the
with many good alternatives. evolution strategies algorithm.
Generation The current generation. Each
Seek optimum generation represents a collection of experiments
performed on the model.
Experiment The current simulation experiment.
28 Chapter 2: Getting Started
Optimize Model
Response plot Edit chart Accesses the controls you will need to
change the appearance of the Surface Response
Plot.
Chapter 3:
Building Projects
4. Enter the warm-up time and run time for the model.
Appendix
Glossary
Fitness [F(x)]
simulations and appropriate number of
Population
replications. converged
Best – Average
CvgPercentage ≥ ------------------------------------------
Best
Confidence Level Population average
Independent variable
See Input factors.
44
Moving Average
Output Recording Time Interval
When the model’s output response is erratic, it is
The length of the individual time periods for
useful to "smooth" it with a moving average. A
which SimRunner computes statistics, as used
moving average is constructed by calculating the
within the Analyze Model step.
arithmetic average of the n most recent data
points in the data set. The value of n is called the
moving average window. As the value of n Pre-analysis
increases, the "smoothness" of the moving aver- See Analyze Model.
age plot also does.
Project
Objective function
When you conduct an analysis in SimRunner,
An expression used to quantitatively evaluate a you create a project containing the model and the
simulation model’s performance. By measuring results from any tests you run.
various performance characteristics and weight-
ing them, an objective function is a single mea-
sure of how well a system performs. Run length
The time length for which SimRunner will run
SimRunner allows you to include many different
your model for each experiment.
performance characteristics in one objective
function. For example, if you want an objective
function to include a measure of total entities pro-
SimRunner 45
User Guide
Simulation
Simulation is the act of creating a model to repre-
sent an actual location (e.g., plant floor, bank
lobby, or emergency room) or abstract, logical
process.
Steady-state
The point at which the output of the model exhib-
its statistical regularity—the distribution of the
output is the same from one time period to the
next.
Upper bound
The highest numeric value of the input factor. In
other words, “no more than this.”
Warm-up Time
The amount of time the model must run for the
objective function to reach a steady state in the
context of non-terminating simulations.
Weighting
This value signifies the importance of maximiz-
ing or minimizing the variable. For example,
while it may be important to maximize the pro-
duction level, it may be more important to main-
tain current personnel levels.
46
SimRunner 47
User Guide
Bibliography
References
This manual referenced the following works. The
list below will allow you to find these works eas-
ily as you seek to increase your knowledge of
simulation and optimization.
Simulation
Harrell, C. R.; B. Ghosh; and R. O. Bowden.
2000. Simulation Using ProModel.McGraw-
Hill, Massachusetts.
Optimization
Fogel, D. 1992. Evolving Artificial Intelligence.
Ph.D. Thesis, University of California, CA.
Index
A F
About SimRunner 29 Factors
Activity capacities 18 input 44
Animation output 44
disable 20 File
menu 29
B
G
Bound
lower 44 General procedure 10
set 26 Getting to know SimRunner 13
upper 45
H
C
Help
Concepts and theories 7 menu 29
Help topics 29
D
I
Data
type 18, 43 Independent variables 43
Define Input factors 44
macros 18 limit 26
Disable animation 20 Interval width 44
Distribution parameters 19 Introduction
what is simrunner? 5
E
K
Exit 29
Export Optimization Data 27, 29 Keyboard 2
50
L P
Limit Pitfalls 11
input factors 26 Pre-analysis 44
Location capacities 18 Print 29
Log time 17 Product support 2
Lower bound 44 Project 44
new 29
open 29
M save 29
ProModel
product team 2
Macros 44
define 18
Maximize
up to a target R
example 17
MedModel Resource capacities 18
product team 2 RTI 19
Menus Run length 44
file menu 29 Run-time interface 19
help menu 29
options menu 29
Modeling S
services 3
Moving Average 44
Save
as 29
project 29
N ServiceModel
product team 2
New project 29 Simulation 45
Number of replications Statistical Advantage 43
test 23 Steady-state 45
Support, technical 2
Symbols and notation
O keyboard 2
text 2
Objective function 16, 44
formulate 26
Open project 29 T
Optimization 5, 44
concepts 26 Target
techniques 27 maximize up to 17
Options Technical support 2
menu 29 Text 2
Output Theories and concepts 7
factor 44 Time
log 17
SimRunner 51
User Guide
U
Upper bound 45
V
Variables 18
independent 43
W
Weighting 45
Where do I begin? 6
general procedure 10
pitfalls 11
52