IE 403 Ch03 RNG RVG With Comments

Chapter 3
Random Number Generation /

Random Variate Generation
1
Manuel D. Rossetti, All Rights Reserved
Objectives
To be able to describe and use the basic
pseudo-random number generation methods
To be able to describe and use the common
random variate generation methods
2
Random Number Generation

The random numbers used in a simulation are not really
random!
Definition: A sequence of pseudo-random numbers, U(i), is
a deterministic sequence of numbers in [0,1] having the
same relevant statistical properties as a sequence of truly
random U(0,1) numbers. (Ripley 1987)
3
Linear Congruential Generator

Definition: A LCG defines a sequence of integers, ( R0 , R1 , ) between 0 and
m-1 according to the following recursive relationship:
Ri 1 aRi c mod m for i 0,1,2, where
R 0 iscalled the seed of the sequence

a is called the constant multiplier
c is called the increment
m is called the modulus
with m, a, c, R0 integers and a 0 , c 0 , m 0 , m a , m c m R0 , 0 Ri m 1

and a corresponding sequence of rational numbers defined by: U i Ri m
4
LCG Example
Consider an LCG with parameters m 8, a 5, c 1, R0 5 and compute the first nine Ri
and U i from the defined sequence.
The mod operator is defined as:
y
z y mod m z y m where x is the floor operator, i.e. the greatest integer x
m
For example, z 17 mod 3 z 17 3 17 3 5 2 . Thus, the mod operator returns the

17
3
integer remainder (including zero) when y m and y when y m ,
e.g. z 6 mod 9 z 6 9 6 9 0 6
6
9
5
Solution
Notice that the random R1 (5 R0 1) mod 8
numbers are fractions of 8 R2 (5 R1 1) mod 8
Notice that if one of the R3 (5 R2 1) mod 8
numbers is repeated, then
the sequence cycles
R4 (5 R3 1) mod 8
R5 6 U 5 0.75
Want to pick a, m, and c
to have long cycles and R6 7 U 6 0.875
dense coverage of (0,1) R7 4 U 7 0.5
R8 5 U 8 0.625
R9 2 U 9 0.25
6
Desirable Properties
Fast
Close to true U(0,1)
Long period before repeating
Does not degenerate to constant or zero
Ability to reproduce identical sequences of random
numbers
Ability to select subsequences
7
Random Number Streams

A seed, e.g. R1 =2, defines a R 0 5 R8
starting place in the cycle and thus

a sequence. R7 4
R1 2
Random number streams denote

sequences defined by different
seeds R6 7 R2 3
Want to divide up the cycle in to

separate streams to be able to use
independent random numbers for R5 6
R3 0
different tasks
R4 1
8
Arenas Generator
Combines two separate generators in order to
achieve a very large period
Combined multiple recursive generator, see
reference LEcuyer et al. (2002) in the text
Would take approximately 219 years into the
future before average desktop computers will
have the capability to exhaust the cycle of the
(generator) in a year of continuous computing
Cycle length of 3.1 1057 with approximately 1.8 1019 streams with stream lengths of
1.7 10 38 and sub-streams of length 7.6 10 22 numbering at 2.3 1015 per stream
9
Testing Random Number

Generators
Kolmogorov-Smirnov Test
Chi-Squared Test in 1-D and higher dimensions
Independence tests
Runs test
Autocorrelation test
Poker test
As well as other tests
Bottom line:
Testing is serious business
Use only tried and true generators
10
Generating Random Variates

Four basic strategies or methods:
Inverse transform or inverse CDF method
Convolution
Acceptance/Rejection
Composition
11
Inverse CDF Method

Theorem: Let X ~ F ( x) and define Y F (X ) , then Y is uniformly distributed on (0,1),
i.e P{Y y} y for 0 y 1
Proof: U
xi F 1 (ui )
PY y PF X y 1
PF 1 F X F 1 y
PX F 1 y ui
F F 1
y
y
Inverse CDF Algorithm

Step 1: Generate U i ~ Uniform(0,1)
Step 2: Return X i F 1 U i 0
xi X
12
Example Exponential Distr.
13
Discrete Example
R
xi 1 2 3 4 1.0
f xi 0.4 0.3 0.2 0.1 X=4 (0.9 < R 1)
F x i 0.4 0.7 0.9 1.0 .8 X=3 (0.7 < R 0.9)
0 if x 1 .6
X=2 (0.4 < R 0.7)
0.4 if 1 x 2
.4
F ( x) 0.7 if 2 x 3
0.9 if 3 x 4 X=1 (0 R 0.4)
.2
1 if 4 x
0
0 1 2 3 4
14
Discrete Inverse CDF

If U i is in the range Then X i is
0 U i 0.4 1 Involves a search
0.4 U i 0.7 2
0.7 U i 0.9 3
0.9 U i 1.0 4
Discrete Inverse CDF Algorithm

Generate ui uniform(0,1)
for i=1 to n
If ui F ( xi ) then return x
i
Loop
15
Convolution
The distribution of the sum of two or more random variables is called the convolution.
n
Let Yi ~ G( y) be independent and identically distributed random variables. Let X Yi ,

i 1
then the distribution of X is said to be the n-fold convolution of Y.
A binomial random variable is the sum of Bernoulli random variables.

A negative binomial random variable is the sum of geometric random
variables.
An Erlang random variable is the sum of exponential random variables.
A Normal random variable is the sum of other normal random
variables.
A chi-squared random variable is the sum of squared normal random
variables.
16
Convolution Example
Convolution Generation Algorithm
Step 1: Generate Yi ~ G( y) i 1,2, , n
n
Step 2: Return X Yi
i 1
Suppose X is Erlang with mean E[ X ] k . Then, X can be defined as the sum of k IID
exponential random variables each with mean .
Yi ~ exp, E[Yi ]
n n n n
X Yi ln(1 U i ) ln(1 U i ) ln
(1 U i )

i 1 i 1 i 1 i 1
17
Acceptance-Rejection
In the acceptance-rejection method, the PDF f (x) from which we desire to obtain a
sample is replaced by a proxy PDF h(x) that can be sampled from more easily. h(x) is
defined such that the selected samples from h(x) can be used directly to represent random
variates from f (x) . The PDF h(x) is based on the development of a majorizing function
for f (x) . A majorizing function, g (x) , for f (x) , is a function such that
g ( x ) f ( x) x .
g(x)

Let c g ( x)dx

Define h( x) g ( x) c
f(x)
x
18
Acceptance Rejection
Algorithm
Acceptance-Rejection Algorithm
Step 1: Generate W ~ h( x)
Step 2: Generate U ~ U (0,1)
Step 3: if U g (Y ) f (W ) accept X W and return, else reject W and return to step 1
19
Example
A simple method to choose a majorizing function is to
3

1 x 2
f ( x) 4
1 x 1
set g ( x) max f ( x) . As can be seen from the figure of
0 otherwise f (x) the maximum ( 3 4 ) occurs at 0 so that g ( x) 3 4 .
We need to find h( x) g ( x) c using g ( x) 3 4 .
1
0.75
1 1

3 3
0.8 c g ( x)dx dx
1 1 4 2
0.6
f ( x) Thus,
0.4
1
0.2 1 x 1
h( x ) 2
0 0 otherwise
0
1 0.6 0.2 0.2 0.6 1
1 x 1
Therefore, h(x) is a uniform distribution over the
range [-1,1], i.e. Uniform(-1,1).
20
Example
A simple method to choose a majorizing function is to
3

1 x 2
f ( x) 4
1 x 1
set g ( x) max f ( x) . As can be seen from the figure of
f (x) the maximum ( 3 4 ) occurs at 0 so that g ( x) 3 4 .
0 otherwise
We need to find h( x) g ( x) c using g ( x) 3 4 .
1
0.75
1 1

3 3
0.8 c g ( x)dx dx
1 1 4 2
0.6
f ( x) Thus,
0.4
1
0.2 1 x 1
h( x ) 2
0 0 otherwise
0
1 0.6 0.2 0.2 0.6 1
1 x 1
Therefore, h(x) is a uniform distribution over the
range [-1,1], i.e. Uniform(-1,1).
21
Algorithm
Steps 1 and 2 generate a random variate, W , from
Step 1: Generate U1 ~ U (0,1) h(x) . Notice that W is in the range [-1,1] (the same as
the range of X ) so that step 4 is simply finding the
Step 2: Set W 1 2U 1
height associated with W in terms of f . In step 3,
Step 3: Generate U 2 ~ U (0,1) 3
we generate U 2 . What is the range of U 2 ? The
Step 4: Set f
3
1W 2 4
4
range is 0, . Notice that this range corresponds to
3
3 4
Step 5: if U 2 f accept X W
4 the range of possible values for f . Thus, in step 5,
and return, else reject W and return
we are comparing a point between 0, to the
3
to step 1 4
candidate point, W s height, along the vertical axis. If
we are under f we accept the W , otherwise we reject
W and generate another candidate point. In other
words, if we generate a point under the curve we
accept it.
22
Cont. Example
Step 1: U1 = 0.25
Step 2: W = a + (b-a)U1(inverse transform for uniform distribution)
= -1 +2(2.5)
= -0.5
Step 3: f = 0.75(1-(-0.5^2))
= 9/16
Step 4: U2= some random number say 0.4
Step 5: Use that number and multiply by majorizing function
= u2*0.75 <= f
= 0.4*0.75 <= 9/16
= 0.3 <= 9/16 Therefore accept 23
Algorithm
Steps 1 and 2 generate a random variate, W , from
Step 1: Generate U1 ~ U (0,1) h(x) . Notice that W is in the range [-1,1] (the same as
the range of X ) so that step 4 is simply finding the
Step 2: Set W 1 2U 1
height associated with W in terms of f . In step 3,
Step 3: Generate U 2 ~ U (0,1) 3
we generate U 2 . What is the range of U 2 ? The
Step 4: Set f
3
1W 2 4
4
range is 0, . Notice that this range corresponds to
3
3 4
Step 5: if U 2 f accept X W
4 the range of possible values for f . Thus, in step 5,
and return, else reject W and return
we are comparing a point between 0, to the
3
to step 1 4
candidate point, W s height, along the vertical axis. If
we are under f we accept the W , otherwise we reject
W and generate another candidate point. In other
words, if we generate a point under the curve we
accept it.
24
Composition
n n
Write F x p i Fi x where p i 0 and p i

1 or equivalently for the
i 1 i 1
probability density function:

n
f x p f x
i i
i 1
The idea is to first randomly select the appropriate distribution using p i and
then generate an X from f (x) .
25
Example Bimodal Distribution

Histogram Divide the data into
two parts
450
400
Fit individual
Frequency
350
300
250
200 distributions to
150
100
50
each part
0
Use composition
4. 031 4
5. 433 8
8 92
7. 923 6
10 204 8 4
11 844 4 8
14 765 1 4
12 846 2
13 512 76
52 4
8
3. 629 5
9. 639 2
40 0
26
7
8
6
2
method to generate
03 2
85 7
20 5
36 6
52 6
6. 357
8
.1 19
.6 48
.3 40
.8 02
2. 872
6
0.
4
0
from overall
Bin
distribution.
26
Algorithm
Based on input data analysis:
Let p1 0.57 , X 1 ~ 0.45 Erlang (0.305,8) , p 2 0.43 , X 2 ~ 5.0 Gamma(1.07,1.79)
Algorithm
Step 1: Generate U ~ U (0,1)
Step 2: if U p1 , set X 0.45 ERLA(0.305 , 8)
Else set X 5.0 GAMM(1.07, 1.79)
where ERLA() and GAMM() are functions that generate Erlang and
Gamma random variates.
27

IE 403 Ch03 RNG RVG With Comments

Загружено:

Сведения о документе

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

IE 403 Ch03 RNG RVG With Comments

Загружено:

Авторское право:

Доступные форматы

Chapter 3

Random Number Generation /

Random Number Generation

Linear Congruential Generator

Ri 1 aRi c mod m for i 0,1,2, where

R 0 iscalled the seed of the sequence

with m, a, c, R0 integers and a 0 , c 0 , m 0 , m a , m c m R0 , 0 Ri m 1

The mod operator is defined as:

For example, z 17 mod 3 z 17 3 17 3 5 2 . Thus, the mod operator returns the

Random Number Streams

starting place in the cycle and thus

Random number streams denote

Want to divide up the cycle in to

Testing Random Number

Generating Random Variates

Inverse CDF Method

Inverse CDF Algorithm

Example Exponential Distr.

f xi 0.4 0.3 0.2 0.1 X=4 (0.9 < R 1)

F x i 0.4 0.7 0.9 1.0 .8 X=3 (0.7 < R 0.9)

Discrete Inverse CDF

Discrete Inverse CDF Algorithm

Let Yi ~ G( y) be independent and identically distributed random variables. Let X Yi ,

then the distribution of X is said to be the n-fold convolution of Y.

A binomial random variable is the sum of Bernoulli random variables.

Write F x p i Fi x where p i 0 and p i

probability density function:

Example Bimodal Distribution

Let p1 0.57 , X 1 ~ 0.45 Erlang (0.305,8) , p 2 0.43 , X 2 ~ 5.0 Gamma(1.07,1.79)

Вам также может понравиться