Scribd
Upload
8 views2 pages

Validation

Validation for stuff IDK

Uploaded by

samw90524
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
8 views2 pages

Validation

Validation for stuff IDK

Uploaded by

samw90524
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

Simulation Course Notes Professor Linda Weiser Friedman

Validating Simulation Models

Diagram – Progress of a Simulation Study

Why Validate? 1) To produce a better model 2) to prove model credibility

Verification – Does the simulation model perform as intended? Is the conceptual model
implemented correctly? This would include debugging the program. (internal validation)

Validation – Is the simulation model an accurate representation of the actual system being
studied? Building the correct model. (external validation)

Validation is goal oriented. A model valid for one purpose may not be valid for another.

The concept of validation is one of degree; it is not necessarily an either / or notion.

Is it ever theoretically possible to establish that we have an absolutely valid model?

If 0  validity  , then as validity increases so does the development cost of the model; and
the value to the decision maker increases but at a decreasing rate.

benefit
The cost curve (benefit to cost ratio or BCR curve) would probably peak at less than the most
valid model money can buy.

How to validate simulation models

There are subjective and objective tests of validity.

 Face Validity. Does the model seem “reasonable” to people who are experts in that type of
system?
 Documentation will be a big help here. Make sure your model is well documented. Try
for self-documenting, as far as possible.
 ask experts
o Look carefully at the flow diagram. Walk through the logic flow
o Does the input data seem correct?
o Are the outputs as expected?
o Output many different statistics from the model; check output for
“reasonableness.”
 check with other model builders (why reinvent the wheel?)
 close observation of the real system
 theory

Validating Simulation Models 1


Simulation Course Notes Professor Linda Weiser Friedman

 Test the Assumptions of the Model


 Structural assumptions. How the system operates, eg, exactly how do customers enter a
queue, leave the queue, choose a server, etc.
 Data assumptions
o Try different values of parameters and see what happens to output stats
o Data fitting
o Sensitivity analysis

 Comparison of Input / Output Transformations. A computer simulation model is,


simplistically, an I/O transformation device. So, in a sense, is the real system itself.
 Turing Test. Can we determine which output (or, report) came from the real system and
which from the simulation.
 Compare the outputs of the real system and the model, using identical input streams.
“Predicting” the past.
o 2-sample tests of significance
o Can use non-parametric test (e.g., Kolmogorov-Smirnov) to compare the output
probability distributions.
 If no real system exists, then one can either use an analytical approximation, or
 Model the current system and validate before inserting proposed changes into the
model

 Field Tests. Can the model successfully predict events which have not yet occurred?

Validating Simulation Models 2

You might also like