Академический Документы
Профессиональный Документы
Культура Документы
W
IE
EV
PR
UMI
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
W
IE
EV
PR
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Random Number Generators: MC Integration
and TSP-Solving By Simulated Annealing,
Genetic, and Ant System Approaches
by
Tathagata Samanta
W
Bachelor of Science
Applied Mathematics
Jadavpur University, India
IE
1997
Masters of Science
EV
Applied Mathematics
Jadavpur University, India
2000
PR
A Thesis
submitted to the
Florida Institute of Technology
in partial fulfillment of the requirements
for the degree of
Doctorate of Philosophy
in
Applied Mathematics
Melbourne, Florida
May 2006
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
UMI Number: 3206620
INFORMATION TO USERS
The quality of this reproduction is dependent upon the quality of the copy
W
submitted. Broken or indistinct print, colored or poor quality illustrations and
photographs, print bleed-through, substandard margins, and improper
IE
alignment can adversely affect reproduction.
In the unlikely event that the author did not send a complete manuscript
and there are missing pages, these will be noted. Also, if unauthorized
EV
UMI
UMI Microform 3206620
Copyright 2006 by ProQuest Information and Learning Company.
All rights reserved. This microform edition is protected against
unauthorized copying under Title 17, United States Code.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
© Copyright 2006 Tathagata Samanta
All Rights Reserved
W
IE
EV
PR
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
We the undersigned committee hereby recommends
that the attached document be accepted as fulfilling in part
the requirements for the degree of
Doctorate of Philosophy in Applied Mathematics
K. ■ ^
Syamal K. Sen, Ph D, /
Major Advisor
Professor, Department of Mathematical Sciences
Dissertation Advisor
W
IE
—
Michael D. Shaw, Ph.D.
Associate Professor and Associate Head,
EV
■<ZSU
Ravi P. AgarwalfPh.Dr
Professor, Department of Mathematical Sciences
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
ABSTRACT
W
Pseudo-random number generators (PRNGs) based on linear congruential, combined
IE
linear congruential, generalized feedback shift register, and subtract with borrow
EV
Faure, and Niederreiter are used for simulating Monte Carlo integration (MCI) and
Traveling Salesman Problem (TSP). MCI can be used for evaluation of integrals and
PR
integral equations, boundary value problems for partial differential equations in heat
option pricing. TSP occurs in VLSI design, network routing problems including
Integration based on MC methods and polynomial time algorithms for TSP are
randomized and so rely heavily on the random number (RN) generation. It is believed
that the solutions of the algorithms are sensitive with the quality of RNs which lead to
iii
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
the urge of developing different methods to produce RNs. This research tries to find
whether the choice of RNGs affects the final solution of the algorithm, the reason
behind that and rank the RNGs based on the solutions of the problem.
The performances of the RNGs have been compared with some available
statistical tests, viz., Chi-square test, Kolmogorov-Smimoff test, Spectral test, cycle
length and generation time. The performances of the RNGs are then compared based
on the solution of one, two, and three dimensional MCIs. The RNGs are also
implemented on three heuristic methods for TSP-solving, viz., 2-Opt and 3-Opt based
Simulated Annealing (SA), Genetic Algorithm (GA), and Ant System (AS) technique.
W
Their performances are compared statistically.
IE
This study finds that the QRNGs (more uniform but less random) performs
much better in terms of the accuracy of result than PRNGs (less uniform but more
EV
random) in MCIs. PRNGs are found to perform better than QRNGs in SA, while
choosing of PRNGs and QRNGs did not effect in the GA. In AS approach the QRNGs
PR
were found to be a little better than the PRNGs. It is often felt that the performances of
these generators are controversial mainly because of the experience that from one
problem to another problem of the same type the same generator behaves sometimes
better and sometimes worse than another generator. In addition, there was an attempt
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
TABLE OF CONTENTS
ABSTRACT.................................................................................................................iii
LIST OF FIGURES....................................................................................................viii
LIST OF TABLES........................................................................................................ x
ACKNOWLEDGEMENTS........................................................................................xii
Chapter
1 Introduction................................................................................................1
W
1.2 Available Test Methods......................................................................... 3
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2.2.1 Halton Method 18
3.1.2 Discrepancy................................................................................ 32
W
3.1.3 Spectral Test............................................................................... 34
IE
3.2 Numerical Results and Graphical Representation............................... 37
4.6 Remark................................................................................................ 71
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
5.3.1 Simulated Annealing.................................................................. 76
6 Conclusions............................................................................................. 114
Bibliography............................................................................................................. 120
W
IE
EV
PR
vii
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
LIST OF FIGURES
W
Figure 3.8 : Randomness of QRN Sobol..................................................................45
viii
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Figure 5.5 : Tour Lengths in SA and AS algorithms for 70 nodes......................... 104
Figure 5.7 : Tour Lengths in SA and AS algorithms for 150 nodes....................... 108
Figure 5.10 : Computational time of Pre- Generation versus Generation of RNs during
Figure 5.11: Computational time of Pre- Generation versus Generation of RNs during
W
IE
EV
PR
ix
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
LIST OF TABLES
Table 3.3 : Discrepancy measure for various RNs generated by each RNGs 40
W
Table 4.5 : No. of RNs needed for summation and probability based MCIs: 1-D, 2-D,
7 j ..................................................................................................................61
EV
Table 4.7 : PRNs : Expected accuracy, Computed accuracy, and Time complexity in
/, 62
PR
Table 4.8 : QRNs : Expected accuracy, Computed accuracy, and Time complexity in
/ , ............................................................................................................. 63
Table 4.9 : PRNs : Expected accuracy, Computed accuracy, and Time complexity in
I 2 ...................................................................................................................64
Table 4.10 : PRNs : Expected accuracy, Computed accuracy, and Time complexity in
I 2............................................................................................................. 65
Table 4.11 : QRNs : Expected accuracy, Computed accuracy, and Time complexity in
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 4.12 : PRNs : Expected accuracy, Computed accuracy, and Time complexity in
h ............................................................................................................. 67
Table 4.13 : PRNs : Expected accuracy, Computed accuracy, and Time complexity in
I3............................................................................................................. 68
Table 4.14 : QRNs : Expected accuracy, Computed accuracy, and Time complexity in
h ............................................................................................................. 69
Table 5.1 : Mean Tour Lenghts, Standard Deviation and Confidence Interval (95\%)
W
for TSP using SA and GA with 22 nodes................................................98
Table 5.2 : Mean Tour Lenghts, Standard Deviation and Confidence Interval (95\%)
IE
for TSP using SA and GA with 51 nodes................................................99
EV
Table 5.3 : Mean Tour Lenghts, Standard Deviation and Confidence Interval (95\%)
Table 5.4 : Mean Tour Lenghts, Standard Deviation and Confidence Interval (95\%)
PR
Table 5.5 : Mean Tour Lenghts, Standard Deviation and Confidence Interval (95\%)
Table 5.6 : Mean Tour Lenghts, Standard Deviation and Confidence Interval (95\%)
Table 5.7 : Mean Tour Lenghts, Standard Deviation and Confidence Interval (95\%)
for TSP using SA and GA with 380 and 1000 nodes............................ 110
xi
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
ACKNOWLEDGEMENTS
I would like to express the deepest gratitude to my advisor Professor S.K. Sen,
for continually injecting in me the spirit of research. Without his guidance and
assistance this research work would not have been possible. I owe him much for
sharing his insights and the freedom to pursue my own interest. I would also like to
thank my other committee members, Dr. M.D. Shaw, Professor R.P. Agarwal, and Dr.
D. Mitra for their valuable comments and time towards my thesis. I thank Professor V.
W
Lakshmikantham, Head, Department of Mathematical Sciences, Florida Institute of
for their vital tips in latex and Andrea Reese for proof reading my thesis. I would also
thank all the Math Graduate Students, Abhijit, Raka, Gopal, Soumyadipto, Priyobrata,
PR
Rohit, Sunil, and Ashim. Also, I am obliged to Penny Flynn and Sally Sellings for
law and my brother-in-law. I would also like to thank my grandfather (maternal uncle)
Nilachal Samanta. Their love, support and encouragement has been invaluable to me.
xii
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
C hapter 1
Introduction
W
In this dissertation, different Random Number Generators (RNGs) axe statis
tically compared and ranked based on the empirical results obtained solving
IE
Monte Carlo Integration (MCI) and Traveling Salesman Problem (TSP). Monte
Carlo (MC) simulations, randomized algorithms, statistical tests, and crypto
graphical applications all rely heavily on the RNGs. Most programming lan
EV
guages includes built-in RNG functions that prove inadequate for applications
where huge amount of random values need to be generated. Ideally, a random
number (RN) generator must not produce repetitive or cyclical values and so
PR
This work attempts ranking RNGs based on the accuracy and time com
plexity of computation of test MCIs and test TSPs. This is as opposed to a
direct ranking independent of any application of an RNG by the available statis
tical tests. A direct ranking of RNGs is only involved, in this context, obtaining
a measure of discrepancy (uniformity) and randomness for each generator. This
indirect ranking is more desirable since the primary interest of this work is the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
solution of a real world problem and not as such in the internal properties of
RNGs.
W
which lead to the urge of developing different methods to produce RNs. Various
researchers have invented techniques to test the uniformity and randomness of
IE
the RNGs, but are yet to produce a full-proof test or test suite by which all
RNGs can respectively be tested and ranked. This research tries to find whether
EV
the choice of RNGs affects the final solution of the algorithm. It provides an
alternative way to rank the available RNGs. Methods like this can be of great
benefit to choose the ideal RNG for specific simulations. It is often felt that
the performances of these generators axe controversial mainly because of the
PR
experience that from one problem to another problem of the same type the
same generator behaves sometimes better and sometimes worse than another
generator. In addition, there was an attempt to suggest pre-generation of RNs
to reduce time complexity of the algorithms.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1.2 A vailab le T est M eth o d s
Generation of RNs were done by tossing a coin, rolling dice or using a roullete
wheel. Over the last 70 years several methods to produce RNs were introduced.
The first method was suggested by John von Neumann, known as the Middle-
square method in 1946 and most recently, the Mersenne Twister was invented
by M. Matsomuto and T. Nishimura in 1997. Statistical testing is employed to
check the randomness and uniformity of a sequence of numbers, preferably a
large sequence generated by an RNG. Some standard test suites available are
as follows.
W
poker, coupon collector’s, permutation, run, maximum-of-t, collision, birthday
spacings, and serial correlation tests. Apart from the foregoing empirical tests
IE
there is the spectral test (empirical and theoretical), which is the most powerful
test for the quality of RNGs.
EV
Diehard Test Suite Developed by George Marsaglia [2] consist of the follow
ing set of tests, viz., birthday spacings, overlapping permutations, ranks of 31 x
31 and 32 x 32 matrices, ranks of 6 x 8 matrices, OQSO, DNA, count the l ’s in
a stream of bytes, count the l ’s in specific bytes, parking lot, minimum distance,
PR
random spheres, squeeze, overlapping sums, runs, and craps tests, monkey tests
on 20 -bit words, and monkey tests OPSO.
NIST Test Suite Developed by the Computer Security Division and the
Statistical Engineering Division at NIST [4] include the frequency, block fre-
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
quency, cumulative sums, runs, long runs, Marsaglia’s rank, spectral (based on
the Discrete Fourier Transform), nonoverlapping template matchings, overlap
ping template matchings, Maurer’s universal statistical, approximate entropy
(based on the work of Pincus, Singer and Kalman), random excursions (due to
Baron and Rukhin), Lempel-Ziv complexity, linear complexity, and serial tests.
The literature review for the MCI and TSP-solving with Simulated An
nealing, Genetic and Ant System approaches are in the respective sections.
W
With the advancement of technology various methodology are developed to solve
a problem. MC methods, one of the premier technique in simulation, are ran
domized in nature and hence uses RNs to simulate. It has applications in eval
IE
uation of integrals including path integrals, integral equations, boundary value
problems for PDE’s, ordinary differential equations with random walks, ordinary
EV
The second problem studied is the TSP. It refers to the problem of finding
the shortest path for a traveling salesman visiting all of N cities only once
starting at a city and returning to it in the end. The complexity of the TSP is
(N —1)! = x ( ^ ) N for the deterministic algorithm. For over half a century
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
its study has led the way to improved randomized algorithms which are always
polynomial time. The most popular ones are Simulated Annealing (with 2-Opt,
3-Opt, fc-Opt), Genetic algorithm, and Ant System approaches. The goal of
this study is to investigate the effect of the RNGs in the quality of result and
rank them according to that.
The work included in this thesis presents an overview of the existing RNGs
in Chapter 2 and in Chapter 3 discusses some important statistical tests for
uniformity and randomness and presented the results based on them. MCI
W
for single, double, and triple integrations with results are given in Chapter 4.
Chapter 5 provides introduction to TSP, its Integer Programming formulation,
IE
Held-Karp lower bound for solutions, and the different heuristic techniques,
viz., Simulated Annealing, Genetic Algorithm, and Ant System approach with
EV
results. Ranking of the RNGs based on the empirical results, observations and
remarks are included in Chapter 6 .
PR
All the work in this research has been performed on desktop computers with
Intel Pentium 4 Processor 3.1 GHz, 592MHz, 1 GB RAM and 2MB L2 Cache
machine running Windows XP. The programs are coded in Matlab and compiled
using Matlab7 and the plots are also generated using Matlab7. All concerned
Matlab programs are included to readily check/run for data considered here as
well as for data not considered here.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
C hapter 2
W
What is a random number generator? A random number generator (RNG) is
a physical or mechanical device designed to generate a sequence of numbers
that does not have any recognizable pattern and hence called random numbers
IE
(RNs). The motivation for this generation is from nature, for example, brown
ian motion (where motion of a particle is random), DNA structure (where the
EV
genetic patterns are random), and chaos theory (where small variation in the
initial condition generates randomness throughout the system).
• Sampling
• Numerical analysis
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
• Financial markets: Option derivatives
How do we get them? We can obtain RNGs broadly in two different ways.
W
optics process. SGI [6 ] has patented a method called lavarand for generating
random numbers using digital snapshots of lava lamps. The properties for these
IE
types of RNGs are slow, not portable, unpredictable, and no mathematical (i.e.,
theoretical) analysis is possible.
EV
• Must be reproducible.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2.1 P seu d o-ran d om N u m b er G enerators
W
the new RN will be a + r(b —a), where r is the RN in [0, 1).
IE
2.1.1 Linear C ongruential M ethod
EV
The method for the linear congruential generator LCG (a, c, m, x0) proposed by
Lehmer in 1948 [1], [7], [8 ], [10] is as follows. The sequence of RNs xn is given
by the recursion
PR
where the order k and the modulus m are positive integer and the coefficients
are integer in the interval [—(m —1), (m —1)]. The numbers xn of the sequence
are then usually scaled to the interval [0,1] by rn = xn/m . When c = 0, this
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.