Advanced Modelling in Operations Management (EBAMO5A)

Advanced Modelling and Simulation (EBAMS5A)

Conducting Simulation Studies

Presented by www.vut.ac.za
Asser Letsatsi Tau (Pr Eng Tech Cand, MSAIChE, SACAA, PhD (Operations) , Master RD (Operations), MEng (Chemical), PDBA, SCT 41 (Civil Eng)
1
 INTRODUCTION

• Now that you’ve gained some insight into the process of developing and analysing a simulation model, it’s time
that we stepped back and discussed the overall activities involved in a typical simulation study.
The system to be modelled may produce manufactured goods, fast food, or services. It could also be a system for
handling paperwork, call center, distribution center or system, or any other system that results in a product or service.
Conducting simulation studies could quite interesting operations research project. Questions around designing a
simulation study entails: what constitutes a successful simulation, formulation of the problem, the issue of using the
correct solution methodology for the problem, and defining simulation specification. Furthermore, the model
formulation and construction activities , and the ever-present verification and validation approaches are then defined.
Experimentation and analysis, and an overview of the reporting and documentation requirements for the simulation
project. Using Arena run-time capability one can disseminate their simulation study work.

2
A Successful Simulation Study
It might seem obvious that if you solve the problem or meet the objectives, you’ve achieved success; however, that is
not always the case. In most instances, the final pronunciation of success or failure will be made by the higher-level
management that is paying the bill—and, like it or not, they have a tendency to view the problem or objectives in a
different context.
• What defines a successful simulation project seldomly means a good simulation model development. It
more often means that the simulation study meet the objectives set forth by the decision makers. This
implies that it is important to understand which metrics they will use to measure objectives of the study.
For example:
If you’re asked to undertake a simulation study to redesign a current system, inquire further to gain a better
understanding of what is expected. If that inquiry results in a statement that indicates that management is interested in
finding out if it is possible to make the system perform better, find out what “better” means. If better means that you
are expected to reduce WIP (work in process) by 30%, reduce cycle times by 20%, increase resource utilizations by
15%, and meet all future customer due dates without any capital investment, you at least know that you’re facing an
impossible mission. We recommend that you elect not to accept the assignment. If, however, “better” means that the
primary measures are WIP, cycle times, resource utilizations, and customer due dates balanced against capital
investment, you at least have a fighting chance of designing a better system.
• Thus, just do not conduct the study without a customer in mind. You
should identify the decision
maker and attempt to define the metrics by which the project will be measured. – have the
customer of the simulation project in mind, it is their final call to make!!!
3
If you’re about to undertake the first simulation study for a company or facility, it’s critical that it result in a
success. If the first simulation study is labeled a failure, it is unlikely that a second study will ever be
started. If you have a choice of simulation projects, select one that is simple and is almost guaranteed to
result in a success. Once you have achieved several successes, you might be able to afford a failure (but
not a disaster).
There is also a peculiar dilemma frequently faced by the experienced simulation analyst. If you use
simulation as a standard tool through the design of a new system and the new system works as
advertised, what have you saved? If you had not used simulation, would the results have been any
different? You really don’t know. So the tool worked, or at least you think it did, but you have no
way of quantifying any savings resulting from the application of the tool.

The one approach that is often suggested, but seldom used, is not to use simulation for the design of
the next new system. The assumption here is that the resulting system will not perform as
advertised. After it’s clear that the system has problems, use simulation to show how the system
should have been designed. This would allow you to quantify the savings had you used simulation.
This is clearly an extreme approach—and not recommended by us—that’s based on numerous
assumptions. Of course, there is always the possibility that the system will perform just fine. (It can
happen!) This would leave the value of simulation in even greater doubt in the eyes of management.
You be aware that your definition of success may not be the same as that used by management. Although you may not be able
to control management’s evaluation, at least you should try to understand what measurements they will use in making their
evaluation.
4
Problem Formulation
• The first step in any problem-solving task is to define and formulate the problem.
In the real world, you are rarely handed some sheets of paper that completely define the problem to be solved. Most
often you (the analyst) will be approached by someone (the client) who thinks that you might be able to help them.
The initial problem is rarely defined clearly. In fact, it may not even be obvious that simulation is the proper tool. At
this point, you need to be able to open a dialog with the requester, or their designate, and begin asking a series of
questions that will ultimately allow you to define the problem completely.
Assume that you have at least a vague notion of what the problem is, and hopefully, a better idea of what the system
is. It might be a good idea to start with the system. Does the system currently exist, is it a new design, or has it not yet
been designed? Knowing this, you can try to bound the system for the purpose of the study. Is the system a single or a
small number of operations, a large department, the entire facility, or the entire company? Although it would be nice
if you could draw walls around the system so that there are no interactions with the other side of the wall (Think
system boundaries), this is not likely to be the case.
• Having established some initial boundaries, next try to define the performance metrics. There are really two kinds
of metrics with which you should be concerned. The first, and most obvious, are the performance metrics that
will be used to measure the quality of the system under study. The second, and maybe the most important, are the
performance metrics that will be used to measure the success of the study.
By this time, you should have a fairly good understanding of the system (and its size), the performance metrics, and
the expectations of the client. The next step is to select a solution methodology.

5
Solution Methodology
• Consider the cost of using a particular solution technique compared to the potential benefits of the eventual
solution.
Identifying the best solution methodology is not always an easy task. If you determine that a specific methodology
might give you the best answer, but you have never used that technique, this might not be the best time to
experiment. Therefore, you might want to rephrase the question: Given the solution methodologies that you feel
comfortable using, which will most likely give the most cost-effective solution?
• Sometimes the choice is obvious, at least to us. For example, if you’re being asked to perform a rough-capacity
analysis of a proposed system, and you’re given only mean values for all the system parameters and you are only
interested in average utilizations, it might be faster (and just as accurate) to use a calculator to determine the
answer. At the other extreme, you might be asked to find the set of optimal routes for a fleet of school buses or
garbage trucks. Although you could use simulation, there are other tools specifically designed to solve this
problem.
There are also instances where companies have devoted effort to develop generic simulation models that can be
altered quickly. These types of models are typically developed when there are many similar systems within an
organization. A generic simulation is then developed that can be used to model any of these systems by simply
changing the data.
Assuming that simulation is the correct technique for the problem. Definition of the system and the details
of the resulting simulation have to be embarked on.

6
System and Simulation Specification
Sometimes, halfway through the development of a simulation, conditions suddenly change, new information becomes
available, or you gain early insight that spawns new ideas. This might trigger the need to revisit the problem-
formulation phase or to alter the design of the model drastically. Thus, back up and revisit the problem whenever
conditions change.
The process of developing a specification can take many forms, depending on the size of the study, the relationship
between the analyst and client, and the ability of both parties to agree on the details at this early stage. If one
individual is playing both roles—client and analyst—this step might be combined with the model formulation and
construction phases.
• As you learn more about the system, start thinking in terms of how you might model these activities. Providing
early input that will result in only minor improvements can increase your credibility for the tasks to follow. If the
system is a new design, find out if there are similar existing systems that might be toured, to obtain a better
understanding of the overall process to be simulated, that you be explored. If the system exists only on paper,
review it. If there’s nothing on paper, develop a process flow diagram or a rough sketch of the potential system.
This should be carried out with the client so there is total agreement on the specifics of the system, from the
onset.
• With the defined system to be modelled, specifications needs to be developed for the potential model. There is no
magic formula for such a specification, but generally it should contain the following elements:

7
o Simulation objectives
o System description and modelling approach Animation exactness
o Model input and output
o Project deliverables
If you’re developing a specification for the first time, don’t expect to get all your answers during the initial meeting.
You might want to consider the 70/20/10 rule. Through experience, we have found that about 70% of the time the
client team will have the complete answer or the information required. About 20% of the time the team will not know
the answer, but they will know how to get it (for example, they may have to ask Cynthia Selepe, who works on the
night shift). The remaining 10% of the time they have no idea what the answer is or where they might find it (or there
are several competing answers).
• An assumption that it’s possible to define the complete study before you start building the actual model. There are
circumstances where this is not possible. These types of projects are open-ended in that the complete project is not yet
defined or the direction of the project depends on the results obtained in the early phases. Even though it may not be
possible to specify the entire project completely, the development of a specification still is often desirable.
Of course, it means that you may have to amend or expand the specification frequently. As long as both parties are
agreeable, there’s no reason to avoid these types of projects. There should be willingness to accept the fact that
direction of the project can change dramatically over its duration. These are often called spiral projects in that they
tend to spiral up into huge projects or spiral down into no project at all.
Now that the simulation have been specified, and it’s finally time to start the model.
8
Model Formulation and Construction
Spend some time formulating the model design. Some of the things that you want to take into consideration are the
data structure or constraints, the type of analysis to be performed, the type of animation required, and your current
comprehension of software. The more complex the system, the more important the formulation.
• Once you’ve formulated a modelling approach, considerations on what constructs is to be used to build the
model. This suggests that first choice should be modules from the Basic Process panel, followed by modules
from the Advanced Process and Advanced Transfer panels, and modules from the Blocks and Elements panel
only when required. This is a recommended approach for the novice simulation software such on Arena user.
Most experienced modelers prefer to start with the Advanced Process and Advanced Transfer panels and select
from the other panels when required. This allows one to create exactly the type of model required and gives the
maximum amount of flexibility.
Finally, ready to open a new model window and begin the model construction, which is probably the most fun part of
the entire study.
• If the model is small, all the required modules might placed on one window/sheet, fill in the required data and
hope that it works right the first time. (It rarely does.) If the system to be modelled requires a large, complicated
model, partitioning of the model building into phases might need to be explored. Selection of a portion of the
system and building of a model of that portion, including at least a rough animation. Once it is convincing that
it’s working correctly, next phase can be added. Continuation in this manner is required until the entire model
has been created. Thus, this approach makes model verification.

9
Verification and Validation
Once you have a working model, and sometimes even while you are building it, it is time to verify and then validate
the model.
• Verification is the task of ensuring that the model behaves as you intended; more colloquially, it’s known as
debugging the model.
Model verification entails the assessment of the correctness of the representation of real systems being modelled by
either inspection of computer codes, the running of tests and final performance checks on the statistical outcomes.
These further assist in increasing the modeller’s confidence that the model is constructed correctly which then
increases the confidence level of the model as monitored on Key Performance Indicators of interest (Marsudi &
Firda, 2018).
• Validation is the task of ensuring that the model behaves the same as the real system.
Validation is carried out through comparison of modelling results with data that is previously available. For example,
historic data can be available from database of an organization (Marsudi & Firda, 2018).
To validate a simulation model, you should compare the results from your model to the results from the real system
(in the case of an exist system).

10
Kasim et al. (2021) also indicated that validation of models is defined as a process of assurance of the simulation
model’s accuracy and animation. This is carried out using correct data and representation of actual system of
production. Furthermore, the difference between the simulation model output and the real output can be calculated by
using the following equation:

𝑠𝑖𝑚𝑢𝑙𝑎𝑡𝑖𝑜𝑛 𝑜𝑢𝑡𝑝𝑢𝑡−𝑎𝑐𝑡𝑢𝑎𝑙 𝑑𝑎𝑡𝑎

Difference (%) = actual data

• A margin of similarity of < 10% is acceptable as reported by Kasim et al. (2021).

Marsudi (2020) and Tau et al. (2022) also validated simulation results by carrying out simplified mathematical
calculations.
If the system does not yet exist, you’re in trouble right from the start. Even if the system does exist, such a
comparison, sometimes may be a challenging task.
It’s not uncommon for organizations to keep extensive metrics on past system performance, but often they do not
keep the information that tells you what the system was doing, making, or subject to during that time period. Even if
the data exist, they may be inaccurate or misleading.

11
Experimentation and Analysis
• There are several practical implications for consideration:
Ideally, before you start any analysis, you would design a complete set of experiments that you intend to conduct.
You would also decide on the types of analysis tools that you would be using. Although this may be ideal, it’s far
from reality. i.e. Scenario analysis as experiments on the simulated model.
In some cases, you just don’t have the luxury of sufficient time; in other cases, you don’t really know where you’re
going with the analysis until you get there.
• You might also consider structuring your experimentation based on the type of analysis that needs to be performed. For
the sake of this discussion, let’s identify three different types of analysis:
a) Candidate analysis is normally done during the early design phases of a system. The intension is to try identify the best
candidate systems from a much larger group of potential designs that merit additional study.
b) Comparative analysis would normally be the next logical step in selecting the final system design. Define finite set of
designs, and compare them to identify the best design. This type of analysis typically requires a more detailed model, but
only concerned with comparing one system to another.
c) Predictive analysis typically deals with only a few systems—often only one. This type of analysis requires inclusion of
all activities that will affect the system’s ability to achieve the predicted performance. At this point, best system has been
selected, and proceeding with recommendation to build it, provided that it meets the required objectives.

12
Presenting and Preserving the Results
• At this stage, the simulation modelling study has been completed, at least the initial study, and ready to go forth
with results.
In many instances, a written report is required. In other cases, meeting a group of decision makers and giving an
informal or formal presentation of the results is critical
• This may actually be the most important phase of the project because it may determine whether your results are
acceptable by the customer or the decision committee or nor.
Furthermore, there are a few obvious points to be made in report or presentation of results.
✓ First, ensure the correct questions have been addressed , and that concise answers are provided.
✓ Always, the equivalent of an executive summary which that states recommendations clearly and the major
reasons for them in one page (or slide) or less should be included.
✓ If possible, presenting numbers in absolute terms should be avoided; ranges or confidence intervals can be used.
✓ Finally, instructions can be made for simulating other alternatives if the decision markers are not convinced.

13
Disseminating the Model

During the course of a simulation study, simulation model and animation need to be shared with the client.
• If the clients have a copy of Arena, the latest model ( .doe ) file can be send to them, and they can view and alter
the model at any time.
Arena does have a run-time mode, which will allow the user to open, run, and view the animation and the results of
any model.
• It allows the user to edit any existing data and re-save the model.
• However, it does not allow the user to add or alter any logic or data modules.
If the run-time version is required, Rockwell Automation can be contacted for more information.

14
Steps In A Sound Simulation Study

15
Additional References

Kasim, A. A., Bareduan, S. A., Masood, I., Ho, F. H., & Abdullah, H. (2021). Capacity study of a food processing
company using arena simulation software. Research progress in mechanical and manufacturing engineering . 2, pp.
166 - 173. Malaysia: Universiti Tun Hussein Onn Malaysia Publisher’s Office.
Marsudi, M., 2020. Modeling and Simulation of Student Registration Process by Using ARENA. United States of
America, IEOM Society International, pp. 1349 - 1357.
Tau, A. L., Edoun, E. I., Mbohwa, C. & Pradhan, A., 2022. Product dispatch work process line improvement using
arena simulation and modeling: a case study. Australia, IEOM Society International, p. 375.

16
 …END…

[email protected]

17