Output Analysis for a

terminating simulators

Presented By :Saleem
Almaqashi
Roll No : 49
1
Introduction

One realization (run) does not necessarily give
the correct answer(s).
Variance exists in simulation results so we must
be cautious about how we interpret results
Output from our model {Y
1
, Y
2
, Y
3
, }
{Y
1
, Y
2
, Y
3
, } may not be independent;
{Y
1
, Y
2
, Y
3
, } may have different distributions,
depending on a number of different factors;
Estimators, confidence interval, and so on, must
be constructed.

2
3
Types of Simulations with Regard to Output
Analysis
Terminating
Non-terminating
Steady-state parameters
Steady-state cyclic parameters
Dynamic parameters
Transient parameters
Other
parameters
Simulation
Systems
Types of Simulations with
Regard to Output Analysis

Terminating: Parameters to be estimated
are defined relative to specific initial and
stopping conditions that are part of the
model
There is a natural and realistic way to
model both the initial and stopping
conditions
Output performance measures generally
depend on both the initial and stopping
conditions
4
Terminating: Examples
Natural event E that specifies the length of each
run.
Ex.1: Bank opens from 9 to 5, E = {8 hours of
simulation (9-5)}
Ex.2: Military exercise Red vs. Blue, E = {Either
Red or Blues wins}
Ex.3: Manufacturing 100 products (random
process), E = {Completion of the 100
th
product}
Ex.4: Reliability simulation of a car (quality
problems random), E = {First major quality problem
occurs}

5
Non terminating
Non terminating: There is no natural and realistic
event that terminates the model Interested in long-
run behavior characteristic of normal operation

If the performance measure of interest is a
characteristic of a steady-state distribution of the
process, it is a steady-state parameter of the model
.

Theoretically, does not depend on initial conditions
Practically, must ensure that run is long enough so
that initial-condition effects have dissipated
6
Non terminating Examples
Ex. 1 (Cyclic): An inventory system, say a retail
store, replenishments are cyclic (e.g., monthly
orders and arrivals of goods), but demand is
random. The performance measure can be
average monthly inventory level.
Ex . 2 (Dynamic ) : Airport security system
simulation (case study). Passenger volume varies
by hour of day, day of the week. Performance
measure passenger waiting time (to get through
security check point) is hourly.
Ex.3 (Transient ) : Transportation (case study).
Multiple-lane highway has one lane blocked (e.g.,
accidents). Cars need to merge and traffic slows
down. Performance measure: drive time during the
traffic jam.
7
Terminating vs. Steady state
simulation
8
9
Terminating Simulations
Statistics based on Observations
Let D
i
= delay of i
th
customer in queue
d = (the true value of) average delay in
queue
For m customer


In general
x
j
= measured performance from the run
= E[x]
Confidence Interval (of n runs):
mean) (true lim ) (
1
Estimate
1
m
D
d
m
D
m D
m
i
i
m
m
i
i
=

=
= =
) used (Prisker
) (
) (
2
2
2 / 1 , 1
I n
n S
t n x
x
n
o
o

10
Terminating Simulations
Replication Approach
Independent runs with different RN streams/seeds but
same I.C.s, also need same number of observations
for each run (conflict with statistics based on time-
persistent variables)
Ex: Run model 10 times with identical initial conditions
x
j
- 1.051 6.438 2.646 0.805 1.505
0.546 2.281 2.822 0.414 1.307
Each one is the average of 35 observations


Compute:

=
= =
35
1
35
) 35 (
j
j
j j
D
D x
172 . 3
1
) (
) 10 ( 982 . 1 ) 35 (
10
1
) 10 (
2
2
10
1
=

= = =

=
n
X x
S x X
j
j
j
11
Terminating Simulations
Replication Approach
Ex: (Continue)
From the statistics table: use o = 0.10 (t
9,0.95
= 1.833)
So, confidence interval of delay times



Interpretation
90% chance (probability) the true value is between
0.950 and 3.014 (not falls into) or 90% that 0.950
to 3.014 covers the true value
Better answer increased number of runs
y probabilit 90% a with 014 . 3 950 . 0
032 . 1 982 . 1
10
172 . 3
833 . 1 982 . 1
s s

d
12
Terminating Simulations
Measure of Precision (Accuracy)
Absolute vs. relative
Half width of actual C.I. absolute precision
Ratio of the half width to the magnitude of estimate
relative precision of C.I.
Determine how many runs are needed
Assumption: S
2
(n) is not changing (within each run)
Absolute Precision (|)



Relative Precision ()
}
) (
: min{ ) (
2
2 / 1 , 1
*
| |
o
s > =

i
n S
t n i n
i a
}
) (
) (
: min{ ) (
2
2 / 1 , 1
*

o
s > =

n X
i
n S
t
n i n
i
r
13
Terminating Simulations
Measure of Precision (Accuracy)
In the previous Ex:
Absolute precision (|) = 1.032
If we want C.I. = 1.982 0.5



Continue to make additional runs i = 37 (27 more runs)
Relative precision () = 1.032/1.982 = 0.521
(i.e., 90% of prob. value is within 52% of the true mean)
If we want = 0.15


Lead to i = 99 (89 more runs)
)) ( ( ) ( i.e.,
} 5 . 0
172 . 3
: 10 min{ ) 5 . 0 (
2 *
95 . 0 , 1
*
n S f n
i
t i n
a
i a
=
s > =

|
} 15 . 0
982 . 1
172 . 3
: 10 min{ ) 15 . 0 (
95 . 0 , 1
*
s > =

i
t
i n
i
r
14
Terminating Simulations
Sequential Procedure (to make multiple replications)
1. Select the initial replication n
0
> 2
2. Compute (based on a selected o)



3. Compute


4. If yes stop
If not n = n + 1 (one additional run)
and repeat steps 2, 3, and 4
n
n S
t n
n
) (
) , (
2
2 / 1 , 1 o
o o

=
precision relative - determined is ?
) (
) , (

o o
s
n X
n
15
Terminating Simulations
Sequential Procedure
In practice, need to obtain statistics at the
end of each replication (run)
Replicate 1:
Estimate: x
1
, S
1
2
(x)
clear stats - use new RN
streams
Estimate: x
2
, S
2
2
(x)

2:
n
0
> 2:


n:
Estimate: x
n0
, S
no
2
(x)
Compute X(n
0
), S
2
(n
0
)
Until

o
s

) (
) (
2
2 / 1 , 1
n X
n
n S
t
n
Conclusion
Statistical methods are used to
analyze the results of simulation
experiments.
Terminating: A simulation where there
is a specific starting and stopping
condition that is part of the model.

16





17