Modeling & Simulation Lecture 7

Building Simulation Models

Instructor: Eng. Ghada Al-Mashaqbeh The Hashemite University Computer Engineering Department

Outline
Introduction. Validation and verification as parts of the simulation process. Guidelines for determining the level of model details. Verification techniques.

The Hashemite University

Building Simulation Model I

Goal: To build simulation models that are valid, credible, and representative of the real systems. We are talking about four important concepts:
Validation: is the process of determining whether a simulation model is an accurate representation of the system. Verification: is to determine the conceptual simulation model has been correctly translated into a computer program (debugging).
The Hashemite University 3

Building Simulation Model II

Credibility: a simulation study is credible if its results is accepted by the manger or other key persons to be correct (it is not necessarily to be valid).
What are the things that help in establishing model credibility?

Accreditation: is the official certification that states that a simulation model is acceptable for some purpose. For accreditation the simulation must be:
Valid, verified, and credible. High quality of the used data. Limitations and problems must be defined.
The Hashemite University 4

The Hashemite University

General Notes on Validation

Ease or difficulty of validation depends on the whether a version of the actual system is available or not. How? Validity of a model has no relation with cost. An expensive model does not that the model is valid and vice versa. Validity of a model is dependent on the purpose of the study. Validation of a model is not optional (if you have time or not!!).
The Hashemite University 6

Guidelines for determining the level of model details I

1. Define the specific issues to be investigated and the performance measures that will be used for evaluation. e.g. think of WLAN, Ethernet LAN, M/M/1 models 2. The entity moving through the simulation model need not be the entity moving in the actual system.

It is not necessary to model all the details of the real system in simulation. Determine the needed details based on the following directions:

Example: it might be better to model a box of parts in a manufacturing model as an entity rather than modeling each part as an entity. Example: model Ethernet packet rather than bit by bit in the packet
The Hashemite University 7

Guidelines for determining the level of model details II

3. Use Subject Mater Experts (SMEs) and sensitivity analysis to help determine the level of model detail.
SMEs are People who are familiar with the system where they are asked what components are likely to be added to the model Sensitivity analysis is used to determine what factors (components) that have a great impact on the performance measure.

4. Do not include excessive amount of model detail. Start by a moderate detail and add more details as needed.
e.g. for M/M/1 model start with a simple single server+queue+arrivals
The Hashemite University 8

Guidelines for determining the level of model details III

5. Do not have more detail in the model more than is necessary to address the issues of interest subject to the condition that the model must have enough details to be credible (so you must find a trade off). 6. The model detail should be consistent with the type of data available. -existing system= lots of data more details. -new systems= no much data less details 7. Include details as the Time and Money constraints allow. 8. If the number of factors is large. Perform sensitivity analysis to determine the important factors. There are some commercial software packages 9 The Hashemite University available for this purpose.

Verification of Simulation Computer Program Techniques I

Technique1: Use modules and subprograms e.g. switched Ethernet:
Switch routine * terminal routine
Start by a moderate simulation model then gradually increase complexity as needed.

Technique 2: Make a structured walk-through of

Technique 3: Run the simulation model under

the program by other persons as one may not able to criticize himself.

several settings of the input parameters. Try it first for known system performance to conduct some comparisons and so to check the obtained results.
The Hashemite University 10

Verification of Simulation Computer Program Techniques II

Technique 4: Trace the state of the simulated
system and compare with a hand simulation. It is advised to use an interactive debugger. Technique5: Run the model under simplified assumptions for which its true characteristics are known.
E.g. For switched Ethernet LAN: Use two terminals + low traffic () + fixed

Technique 6: If it is possible, use animation to

trace entities in the model.

Technique 7: Compute a sample mean and

E.g. show cars when simulating a traffic light.

sample variance for each simulation input probability and compare with the desired known mean and variance.
The Hashemite University 11

Verification of Simulation Computer Program Techniques III

Technique 8: Use commercial simulation packages carefully to reduce the amount of programming time.
Use the tool that is related to your field of interest. Make sure everything you need is supported in the tool. Make sure that the tool is well used and recommended by other researchers and have been trusted to provide valid results.
The Hashemite University 12

Techniques to increase model validity and credibility

Technique 1: Collect high quality information and data on the system. Technique 2: Interact with the manager on a regular basis. Technique 3: Maintain an assumptions of document and perform a structural walkthrough. Technique 4: Validate components of the model by using quantitative techniques. Technique 5: Validate output from the overall simulation model. Technique 6: Animation.
The Hashemite University 13

Technique 1: Collect high quality information and data on the system I

Make use of the existing information including:

A. Conversation with Subject Mater Experts (SMEs) Example: for a communication network, relevant people may include:
End users Network designers Technology experts System administrators Application architects Maintenance personnel Managers
The Hashemite University 14

Technique 1: Collect high quality information and data on the system II

B. Observations of the system:
Difficulties of Correct Data Collection from the real system: 1. Data are not representatives of what we need Example: the data collected in a military field test may not represent the actual combat field.

2. Data are not of the appropriate type or format. 3. Data may contain measurements, recording or rounding errors. 4. Data may be biased because of self interest. 5. Data may have inconsistent units. Km, miles, etc.
The Hashemite University 15

Technique 1: Collect high quality information and data on the system III
C. Existing Theory
-e.g. bank system = poisson Process.

D. Use previous data that have been

introduced from similar simulation studies. (if you have done it before)

E. Use the experience and intuition of the modelers to hypothesize how certain

components of a complex system operation (if it is unknown systems).

The Hashemite University

16

Technique 2: Interact with the manager on a regular basis.

To make the problem and the simulation model clearer. To follow the manager requirements and interests. Manager knowledge of the model increases the model validity and credibility.

The Hashemite University

17

Technique 3: Maintain an assumptions of document and perform a structural walk-through. Record all assumptions that have been taken in the simulation model including
Project goals. Detailed description of each subsystem. What simplifying assumptions were made and why. Summary of data such as mean, histogram, etc. Sources of important and controversial information.
The Hashemite University 18

Technique 4: Validate components of the model by using quantitative techniques.

Use sensitivity analysis to determine model factors that have a significant impact on the desired measures of performance. Examples include
The value of a parameter The choice of a distribution that the entity moving in the system The level of detail of subsystems What data are the most crucial to collect e.g: Suppose that you talk to a bank manager and estimated that teller queue to be 5 customers. You simulated the for queue size of 4,5,6,7 and found the system produces almost the same results. Then the output is not sensitive to this parameter.
The Hashemite University 19

Technique 5: Validate output from the overall simulation model.

To validate a simulation output compare the obtained results with:
An existing system (if available). An expert opinion: if the results obey the expert opinion then your model is said to have a face validity. With another model: but be careful to select a valid model.
The Hashemite University 20

Technique 6: Animation
Remember animation is useful for both technical and non-technical people!!!

The Hashemite University

21

Additional Notes
The lecture covers the following sections from the textbook:
Chapter 5:
Sections 5.1 5.4

The Hashemite University

22