You are on page 1of 113

1

1/3/17

What is Genetic
Algorithm ?

As the name Genetic


Algorithm (GA), itself
, suggests, it is
a class of stochastic methods,
modeled after evolutionary
mechanisms of nature.
a popular strategy to optimize
highly non-linear systems with a
large number of variables.
It is rated among the top ten
algorithms of the 21st century, that
has found the largest practical
applications

Father of Genetic
Algorithm !!
Prof.JOHN
HENRY
HOLLAND
Professor, Electrical
Engineering and
Psychology
University of
Michigan ,
Ann Arbor, USA
Presently

Books written by John


Holland

Another Pioneer in Complex


Adaptive Systems and
Evolutionary Computing
Professor at the
Department of
Industrial and Enterpri
se Systems Engineering
(IESE) at the
University of Illinois a
t Urbana-Champaign
David E Goldberg Director of Illinois
Genetic Algorithms

IN INDIA AT IIT
KANPUR

Prof. Kalyanamoy Deb


GENETIC ALGORITHM
LABORATORY

BIGGEST
ADVANTAGE of
GAinvolve
!! any
It does not
rigorous mathematics
except the use of Random
Numbers !!

Yet powerful enough to


solve very complicated
highly non-linear
optimization problems

Let us christen him,


George !

1/3/17

The Story of the


Shakespearean Monkey !!

WHAT DOES GEORGE DO ?

HE GOES ON , 24 X 7 , TYPING ON THE KEY BOARD,


HITTING THE VARIOUS KEYS RANDOMLY HE NEVER
STOPS !!

FOR SIMPLICITY, LET US ASSUME, THAT THE KEYBOARD


IS NOT THE CONVENTIONAL ONE, NORMALLY USED BY
US, BUT RATHER A MODIFIED ONE, WITH 26 KEYS FOR
THE ENGLISH ALPHABETS, 1 SPACEBAR AND 5
PUNCTUATION MARKS ( , . ! : ; ).
LET US ALSO ASSUME THAT THE SPACEBAR IS LONGER
THAN THE OTHER KEYS, ROUGHLY 3 TIMES LONGER,
AND HENCE OUR GEORGE, THE MONKEY, HAS THREE
TIMES MORE CHANCE OF HITTING THE SPACEBAR

NO DISTINCTION BETWEEN UPPER AND LOWER CASE


LETTERS

WHAT IS THE PROBABILITY OF GEORGE


HITTING A PARTICULAR KEY ?
Total No of Keys
34
Consisting of
Alphabets
26
Punctuation Marks
5
Space Bar
The probability
of hitting
3

Can George type the first line


of Shakespeares HAMLET ?
To be, or not to be, that is the question;
Whether it is nobler in the mind to
suffer
The slings and arrows of outrageous
fortune,
Or to take arms against a sea of
troubles,
The first
line
the above verse
And
byofopposing,
end has
them
42 characters, consisting of,
30 letters,
3 punctuation marks and
9 spaces

LET US LOOK AT THE PROBABILITY


OF THIS HAPPENING
If George starts typing, the chance hell
get the first character right is 1 in 34.
Since the probability hell get the second
character right, is also 1 in 34, he has a 1
in ( 34*34 ) (rule of joint probability)
chance of landing on both the first two
correct characters.
It follows that the probability, that George
will type the first line of the full
Shakespearean stanza is:
((1 / 34) 33) X ((3 / 34) 9)

George may get the first line


of Hamlet typed, but.
The chances are
1 in
1,066,653,157,510,057,196,835,3
54,658,
914,553,991,296,211,872,584,111
The above number, let us call it N , is
,802,
259,963
incredibly large , of the order of 10 (60) , but
Still 1/N is not

Some Probability Calculations


The probability of George getting it
wrong in one try = (1 1/N) 1
Suppose we want George to have a 1
out of 100 chance of getting it wrong
( or 99 out of 100 chance of getting it
correct ) over X # of tries. So we get
(1 1/N) X = 1/100, which can be
written as ,
X

Some More
Calculations
To solve for x, we take the natural
log of each side:
ln(((N-1)/N) X) =
ln(1/100)
or
X * ln((N-1)/N) = ln(1/100)
X = ln(100) / (ln(N/(N-1))
As N goes to infinity, ln(N/(N-1))
1/N, therefore:

X N * ln(100) 5 * N

HOW LONG IT WOULD TAKE FOR


GEORGE TO HIT IT RIGHT ?
Assume that George is hard working,
and punches the keyboard at
maximum possible speed
Let him type 20 lines ( = 840 characters )
per
second ,

This means that hell type 630,720,000


( 20 x 86400 x 365) lines per year.

The number of years it will take him is:


(N / 630,720,000) * ln(100) = 7.8 * 10 51
years

Thats a really a mind bogglingly

The Age of the


Universe is only 13.82
Billion Years !!!

WHAT IF ALL THE MONKEYS IN


THE WORLD WORKING
TOGETHER ??
Let all the monkeys in the world, say 1
billion in number, type simultaneously
How long will it take to have a 99% probability,
that one of the monkeys, getting it right and
become our Shakespearean Monkey ?
A few calculations will lead us to the conclusion
that this will take about 7.8 * 10 42 years

Still the problem


persists.
Even though billions of monkeys
working together in unison, has
reduced the time on a massive
scale, (of the order of 1 billion), we
still havent solved the problem of
hitting a Shakespearean monkey,
in an amount of time, that any
human could experience in his or
her lifetime.

6 Million Years Ago 0.2 Million Years Ago

20

1/3/17

Inspiration from
the Natural
Evolution
Process !!

THE HUMAN
EVOLUTION !!

Natures Starting
Point !!
( Mr Tracey Cools
gggg. Grandfather !! )
Civilisation Started

Mr. Tracey Cool,


Design Engineer,
Rolls Royce

EVOLUTION AS A PROCESS NATURES


OPTIMISATION

Fish 400 million


Years ago !!

A gold fish which came from China, priced at $8350, Is displayed


during the Fishlook 2003 exhibition in Jakarta. Indonesia, Singapore
and Malaysia have participated in this sevenday exhibition
to find the gold fish with the nicest colour and shape.

POLAR BEAR A REMARKABLE


ADAPTION TO SURROUNDINGS

POLAR BEARS HUDDLE


TOGETHER, TO ESACPE FROM
BEING HUNTED AND SAVE
THEMSELVES FROM A SCUDDING

Charles Darwin 1809 1882


The Evolutionary Theorist

DARWINIAN
HYPOTHESIS
It

is
neither
the
strongest nor the most
intelligent
of
the
species that survive,
but
the
one
most
responsive to change

CONSEQUENCES OF THE
DARWINIAN HYPOTHESIS
Primary Driving Mechanism of Evolution
is Natural Selection
Individuals better adapted to
environment will leave behind, on an
average, more off springs than their
less adapt colleagues.

Survival of the Fittest

A LITTLE BIT OF GA
HISTORY !!
Even

in the formative years of Electronic Computer


Revolution during the late 1950a there were attempts
at modeling biological evolution.

Genetic

algorithms were invented by John Holland in


the 1960s and were later developed by Holland and
his students and colleagues at the University of
Michigan during the 1970s.

Hollands

1975 book Adaptation in Natural and


Artificial Systems presented the genetic algorithms
as an abstraction of biological evolution and gave a
theoretical framework for adaptation under the
genetic algorithms.

GA DEVELOPMENTS DURING THE


1980S AND 1990S

1980s saw many a real-world applications


due to Internet revolution and increase in
power of Pcs
Pattern Recogonition
Flow Control Devices
Structural Optimisation
Micro-chips Design
Aerospace Applications
Micro-Biology
1990s saw commercial and business
applications
Stock Market Predictions
Scheduling and Planning

HOW DOES GENETIC EVOLUTION


OCCUR ?

29
1/3/17

GENETIC INFORMATION TRANSFER


The

information determining
the growth and development of
individuals of a species is
encoded in their genes.

In

sexual species, when two


individuals breed,
complementary portions of
their genetic material
(chromosomes) are passed on
to their off spring randomly .

INHERITANCE

Off springs must retain at least some of the


features that made their parents fitter than
average. Otherwise evolution is reset at every
generation

VARIABILITY

At any given time individuals with varying fitness


must co-exist in the population. Otherwise
natural selection has nothing to operate upon .

TWO ESSENTIAL
INGREDIENTS

A POPULATION OF RABBITS

FOX PREYS ON THEM

WHAT HAPPENS IN THE


RABBIT POPULATION ?
SOME RABBITS ARE FASTER AND
SMARTER
THEY ARE LESS LIKELY TO BE EATEN BY
THE FOX
MORE OF SUCH SMARTER ONES TEND TO
SURVIVE ON AN AVERAGE
THEY TEND TO BREED AMONG
THEMSLEVES AND ALSO AMONG LESS
SMARTER ONES

RABBIT STORY CONTINUES

IT RESULTS IN A GOOD MIX OF RABBITS


GENETIC MATERIAL.

OF COURSE, EVEN DUMB AND SLOWER


ONES SURVIVE BECAUSE OF LUCK.

THEY ALSO MATE AND BREED. THE OFFSPRINGS WILL, ON AN AVERAGE, BE


SMARTER AND FASTER THAN THEIR
PARENTS IN THE ORIGINAL POPULATION,
BECAUSE ACCORDING TO DARWINIAN
HYPOTHESIS, ONLY THE FITTEST SURVIVE
WITH MORE PROBABILITY

NATURES PRINCIPLE
BEST

INDIVIDUALS REPLICATE
AND GET PROPAGATED IN THE
POPULATION MORE AND MORE

AVERAGE
WORST

INDIVIDUALS STAY ON

INDIVIDUALS WHITHER
AWAY AND DIE A NATURAL
DEATH

THE LESSON LEARNT


SMART ONES
SURVIVE !!

THE GENOTYPE AND


PHENOTYPE OF
BIOLOGICAL
EVOLUTIONARY
PROCESS
37
1/3/17

THE GENOTYPE -- DEFINITION


Genotype is the information contained
within the DNA.
Genotypes can only be determined by
biological tests and not by observations.
Genotype is an inherited trait and
hereditary information, passed by the
parents determines genotype.The entire
genetic information about an organism is
contained in a genotype even those that
are not expressed visually or otherwise .

GENOTYPE EXAMPLES
The gene responsible for certain
diseases
The gene responsible for eye
color
The gene responsible for hair
color
The gene responsible for height
The gene responsible for certain
type of behaviour

THE PHENOTYPE
Phenotype is simply the expression of information
within a gene. It is the visible or observable
results of genes, combined with the environmental
influence on an organisms appearance or
behavior.
It is the expression of gene information,
observable with the senses (like the sound of a
birds chirping or the colour of a cats hair)
Phenotype is not an inherited trait only the
genes that may cause that phenotype are.

PHENOTYPE EXAMPLES

The size of a birds beak


The length of a foxs tail
The color of the stripes on a
cat
The size and shape of the
spots on a dogs back
An individuals shoe size

1/3/17

Can we breed
Monkeys with
required traits?

42

Can we breed monkeys to hit


at the Shakespearean Monkey,
our
traget
?
Based on the principles of
evolution, that we have just
now seen, can we breed
monkeys, that have the trait of
typing correctly the characters
of the Shakespearean verse ? In
this context,
The Genotype is their DNA
signatures
The Phenotype is the English
alphabet they type correctly.

Monkeys with DNA


Signatures
Assume

that DNA signatures


determine the ability of the monkeys
to type any of the 42 characters in the
text to be, or not to be , that is the
question ; , correctly

monkeys fitness is determined by


the number of correct characters he
types

By

correct characters , we mean the


characters that match the

Let us Start with a Population


of 100 Monkeys !!!

Monkey Breeding
Scenario

Monkeys with higher fitness qualify for


selection as parents and are allowed to breed

At birth, each monkey childs DNA is a


combination of two selected parent monkeys.
( crossover )

The children born out are expected, on an


average, to type more characters correctly
and improve their fitness

In addition, each individual character in the


childs DNA string , also has a small
probability of ( mutation ) i.e. of becoming
a random character, rather than one from the

EVOLUTION CYCLE AND THE


GENETICALLY ENHANCED MONKEYS

In the next generation, the children


become the parents and are again allowed
to go through the selection, crossover and
mutation processes
It is expected that the average fitness of
the population shows a steady, but rather
a very very slow, improving trend.
The monkeys become genetically
enhanced with generation after
generation and finally we fervently hope to
produce our darling , the Shakespearean
Monkey , which is The target

EVOLUTION CYCLE AND THE


GENETICALLY ENHANCED
MONKEYS

On the evolutionary scale, if we try to do this


biologically, let us assume ( even though this is
nearly impossible ) that, each generation of
monkeys lives, on an average, 10 years and that
200 generations produce a target monkey.
We have to wait for 2000 years to produce a
Shakespearean Monkey
This is still a long time for humans,
But it is very very less ( by 57 orders ) compared
to the
N , that we computed earlier.

Still the problem persists.


Even though we figured out the
genetic processes of selection, crossover and mutation and try to
implement them on a population of
monkeys , we still havent solved the
problem of hitting a Shakespearean
monkey, in an amount of time, that
any human could experience in a given
lifetime.

What do we do ?

COMPUTERS , COMPUTERS EVERYWHERE !!

Today

we have proliferation of
computers
They have, day by day , become
tinier and cheaper
possess unimaginable
processing power and memory
Parallel Processing Technology
enables still higher and higher
computing power and speed
Can we , then, simulate the
evolutionary cycle on the digital
computer, choosing an
appropriate DNA
representation
randomly simulate the three
processes of selection, crossover and mutation

Can We Simulate
the Evolutionary
Process on the
Computer ?
Can We Breed
Digital
Monkeys ?

Computer Simulation of the


Monkey Evolution
The Digital Monkey !

We have established that the monkeys DNA


determine individual monkeys pattern of typing.

We want at least one of our monkeys to type


the text To be, or not to be: That is the
question: That is the Shakespearean Monkey !!

Assume that there are 32 ( = 2 5 ) characters


including the spacebar on the keyboard ( instead
of 34, taken earlier ) and 42 characters in the
desired string.

How can we express the monkeys DNA as binary


information?

Binary Representation
of DNA
Each key on the monkeys typewriter
corresponds to a 32-bit number. In binary
representation this is a 5-digit string as
2 5 = 32. Thus
KEY # 1 = a or 00000
KEY # 2 = b or 00001
KEY # 3 = c or 00010
etc. etc. etc. up until
KEY # 32 = SPACE or 11111

THE BINARY STRING AND


ITS INTERPRETATION

Lets take the string:


0101100111101010100101010101011110000101
We can divide it in to 5-bit segments:
01011 00111 10101 01001 01010 10101 11100
00101
Which translates to a typing sequence of key #s:
11,7,21,9,10,21,28,5.
To start from 1 instead of 0, we add 1 to each
of these numbers.
12,8,22,10,11,22,29,6
And which translates to the phrase:

lhui ju;f

THE FULL TEXT AND THE


BINARY STRING

For us to get To be or not to be: That is the


question: out of the monkey, well need a
longer genome a ( 42 x 5 ) 210 -bit
character string.
For example:
10000001111011100000001010000001100
001110101100100010000000001010110000
101101000000111101101011111001011010
000100000010101011100100111001001100
011101100101011111010101001100011101
1101001110011111001100000110001
This translates into

The full text

16,
10,
16,
21,

7, 23, 0, 5, 0, 12, 7, 11,


24, 11, 8, 3, 27, 11, 28,
5, 11, 18, 14, 9, 17, 27,
9, 17, 27, 20, 28, 31, 6,

4, 8, 0,
22, 16,
5, 15,
1, 17

Which further interprets ( after


adding 1 ), as the following text:

qhxafamhleiakylid:l;wqqflsojr:fpvjr
:u; gbr

Pool of
Monkeys !!

Suppose we have a group of 5 monkeys, with the


following DNA sequences (expressed as a
character string after genome-phenome
conversion).

1. ,:drxkaitxsdp: f?ynjasqosupchujk,, rizhg:r


2. cklavicmemt :bbkyyn fjgd?y?k?dlgoyponkoa
3. lt;wv hmstx,cxy ,mo;vwwpbqkuibuhq;fq'ow
4. zqayapqfiszwrh ?isye: ghg:mllowxzcuyadly
5. ielcm,sc dnzm,ngeuviu:lu;zo ojxwpizvk;adqe

HOW DO WE GENERATE
THE BINARY STRINGS ?

Generate a random a number


between 0 and 1
If a is less than 0.5 consider the bit
as 0
If it is greater than or equal to 0.5
consider the bit as 1
There are no strict rules !!. You can
frame your own rules and follow. The
binary sequence has to be randomly
generated that is the only
requirement
Simple isnt it ?

Evolving the
Monkeys Next
Generation
The first step is the Selection of Parents
Each monkeys fitness is defined as the #
of characters in their DNA that match To
be, or not to be: that is the question:
Monkeys are selected in the order of
their ranking by fitness
Various selection methods are possible
each with its own advantages and
drawbacks.

SELECTION METHODS
ELITIST

SELECTION
FITNESS PROPORTIONATE SELECTION
ROULETTE WHEEL SELECTION
SCALING SELECETION
TOURNAMENT SELECTION
RANK SELECTION
GENERATIONAL SELECTION
STEADY STATE SELECTION
HIERARCHIAL SELECTION

SELECTION BY ROULETTE
WHEEL

GENERATE A RANDOM NUMBER r

If r < q(1) choose first individual ( q


(i) is the fitness of ith member of the
population pool )

If not choose ith individual such that


q(i-1) <= r <= q(i )

SELECTION BY
ROULETTE WHEEL
It

is likely that same chromosome may be


selected more than once but that will have a
high fitness ratio (a big chunk in the
roulette wheel).

This

is consistent with Natures operations.

Chromosomes with least fitness will also be


selected but with considerably less chance.

This

is also appropriate, as it introduces


variability and keeps the evolution process
progressing, preventing it from getting
struck at a particular generation.

Roulette Wheel Selection

PROBABILISTIC SELECTION
BASED ON FITNESS
Better

individuals are
preferred
Best is not always picked
Worst is not necessarily
excluded
Nothing is guaranteed
Mixture of greedy
exploitation and

Evolving the Monkeys


Next Generation
1. ,:drxkaitxsdp: f?ynjasqosupchujk,,
rizhg:r
Fitness = 0
2. cklavicmemt :bbkyyn fjgd?y?k?
dlgoyponkoa
Fitness = 1
3. lt;wv hmstx,cxy
,mo;vwwpbqkuibuhq;fq'ow
Fitness = 2
4. zqayapqfiszwrh ?isye:
ghg:mllowxzcuyadly
Fitness = 0

First 14 Characters of the Target string


t

o
b

15th to 28th Characters of the Target string


t
o

b
f

e
:

n
?

"

29th to 42nd Characters of the Target string


t
h
e

h
u
i
k

?
d
l
"

u
i
b
u

l
o
w
x

o
j
x
w

,
,

g
o

h
q

"
z

p
i

u
e
s
t
i
o
n

r
i
z
h
g
:

y
p
o
n
k
o
"

;
f
"
q
"
o
w

c
"
u
v
a
d
l

z
v
k
:
a
d
q

:
Target

r
#1

a
#2

#3

y
#4

e
#5

Fitness

Match Rows

40

34

40

11

Applying the Processes of


Selection, Cross- over and
Mutation
Total

Fitness = 5
1. % of Total Fitness = 0%
2. % of Total Fitness = 20%
3. % of Total Fitness = 40%
4. % of Total Fitness = 0%
5. % of Total Fitness = 40%

Applying the Processes of


Selection, Cross- over and
Mutation
Next,

we will select the parent monkeys


for breeding. Each monkeys chance at
being selected is equal to their % of total
Fitness. In this case, its as if we spun a
Roulette wheel with these possible
outcomes: 2,3,3,5,5 (since Monkeys 1 and
3 scored a zero, they are not eligible for
reproduction). Lets assume we randomly
select monkeys #3 and #5 to be our first
parents.

Schematic of Cross
over

Operators-Crossover and Mutation


Crossover:

Applied with high probability


Position for crossover on the two parent chromosomes
randomly selected
Offspring share characteristics of well-performing parents
Combinations of well-performing characteristics generated

Mutation:

Applied with low probability


Bit for mutation randomly selected
New characteristics introduced into the population
Prevents algorithm from getting trapped into a local optimum

Cross Over Operation


on Bit Strings

Mutation
MUTATIONis a genetic operator used to
maintain genetic diversity from one
generation of a population of
genetic algorithm chromosomes to the
next.
It is analogous to biological mutation.
Mutation alters one or more gene values
in a chromosome from its initial state. In
mutation, the solution may change
entirely from the previous solution.

MUTATION

Hence GA can come out with better


solution by using mutation.
Mutation occurs during evolution
according to a user-definable
mutation probability. This
probability should be set low.
If it is set too high, the search will
turn into a primitive random search
with no focus on target

Effective Mix of GA Operators has


a bearing on the Convergence to
Target

Using selection alone will


tend to fill the population
with copies of the best
individual from the
population
Using selection and
crossover operators will tend
to cause the algorithms to
converge on a good but suboptimal solution

CROSS-OVER OPERATION

Decide a cross-over probability


Pc

For each chromosome generate a


random number r

If r < Pc select that chromosome


for the cross-over operation

Take a pair of chromosomes from


among the selected pool

MUTATION
OPERATION

DONE ON A BIT BY BIT BASIS

DECIDE ON A MUTATION
PROBABILITY-- Pm

Beware:

Too less Pm variability is reduced


Too large Pm best chromosomes
are lost unnecssarily

MUTATION
Generate a random number r
If r < Pm mutate that bit ( 0 to 1

or 1 to 0 )
Now the new population is ready
for next evaluation of fitness
The above operations are done in a
cyclic manner many times ( many
generations )
The ultimate population is expected
to contain the best chromosome
( optimum solution ).

Cross Over or
Mutation ?
Cross

Over or Mutation which one


is important or absolutely
necessary ?

Decades

of debates have no
definite answer

However,

there is consensus that

it

all depends on the problem to


which GA is applied and

each

process is important in its


own right and is to be
implemented in the correct
measure

ELITISM -- GOOD INDIVIDUALS ARE


PRESERVED DURING ANY GENERATION

NICHE POOL OF BEST


INDIVIDUALS

MICRO-GA OPERATES ON SELECTED


INDIVIDUALS

SPECIAL ADVANCED GA
OPERATORS

Comparisons
Traditional . Vs. Evolution Based
Methods

Traditional ( Gradient Based )


best

initial guess

may

lead to local, not global


optimum

Nature (Evolution Based )


population
more

of guesses

likely to find global


optimuma better solution

MIMICKING NATURAL
EVOLUTION

NATURAL EVOLUTION PROCESS

Generation n

Initial Population
Fitness to Environment
Best ones having high fitness breed

randomly
Variability (at any time individuals of
varying
fitness coexist)
Next Generation n +1

MIMICKING NATURAL
EVOLUTION
Numerical Simulation
Generation 0
Select feasible sets of independent variables
initial population
These are coded as genes (strings of binary / real
numbers)
Rank them by the objective function and select the best
few
Allow then to breed by crossing their genes at random
positions
To introduce variability mutate the strings randomly

Next Generation ( iteration ? )

NATURE TO COMPUTER MAPPING

INDIVIDUAL OF THE SPECIES


POPULTION

POSSIBLE SOLUTIONS

POPULATION

SET OF SOLUTIONS

FITNESS TO ENVIRONMENT

OBJECTIVE FUNCTION

NATURE
CHROMOSOME
SELECTION
CROSSS-OVER
MUTATION
EVOLUTION

COMPUTER
CODING FOR INDEPENDENT
VARIABLES
SEARCH OPERATORS TO MIMIC
GENETIC BREEDING
OPTIMAL SOLUTION

Comparison
Nature .vs. Computer GA
Nature
not

very efficient

at

least a 20 year wait between generations

not

all mating combinations possible

Computer
efficient

Genetic algorithm
and fast

optimisation
mating
Parallel

complete in a matter of minutes

combinations governed only by fitness

Computation Possible for Complex


Problems

HOW GA OPERATES ?
Imagine an army of
parachutists dropping
onto the landscape of a
problem's search space,
with each one being
given orders to find the
highest peak. Small
mutations enable each
individual to explore its
immediate
neighborhood, while
selection focuses
progress, guiding the
algorithm's off-springs
uphill to more
promising parts of the
solution space. Crossover enables transfer
of information among

EXPLORATION AND
EXPLOITATION
RANDOM SEARCH EXPLORES THE SEARCH
SPACE - IGNORES EXPLOITATION OF POTENTIAL
REGIONS OF POSSIBLE SOLUTIONS

GRADIENT BASED METHODS DO NOT EXPLORE


THE SEARCH SPACE BUT EXPLOITS THE BEST
SOLUTION FOR IMPROVEMENT

GENETIC ALGORITHMS STRIKE A REMARKABLE


BALANCE BETWEEN EXPLORATION AND
EXPLOITATION

MINIMISATION
GA TEND TO MAXIMISE A GIVEN

FUNCTION
TO MINIMISE AN OBJECTIVE
FUNCTION DEFINE IT AS NEGATIVE OF
ORIGINAL
min f(x) = max { -f(x)}
We can also redefine the objective
function as
f(x) = 1/ {1+f(x) }

HANDLING NEGATIVE
FUNCTION VALUES
GA ASSUMES THAT OBJECTIVE
FUNCTION IS ALWAYS POSITIVE

TO ACCOUNT FOR NEGATIVE


VALUES WE ADD A CONSTANT C

max (f(x)) = max { f(x)+C}

DISADVANTAGES OF GA
BEING

STOCAHSTIC IN NATURE,
CONVERGNCE IS RATHER SLOW

PROBLEM

SPECIFIC SETTING OF
VARIOUS PROBABILITIES ( CROSS-OVER
MUTATION ETC ) THOUGH SOME
EMPIRICAL RULES ARE KNOWN

IDEAL

POPULATION SIZE IS NOT EASY


TO CHOOSE

NEEDS

LOT OF GENERATIONS BEFORE


GETTING NEAR TO THE OPTIMUM
SOLUTION

PROBABILISTIC STEPS
The

initial population is typically


random

Probabilistic

selection based on

fitness
- Best is not always picked
- Worst is not necessarily excluded
Random

picking of mutation and


crossover points

Often,

there is probabilistic

GENETIC ALGORITHMS
A FORMAL DEFINITION
The genetic algorithm is a
probabilistic
search
algorithm
that iteratively transforms a set
(called
a
population)
of
mathematical objects (typically
fixed-length
binary
character
strings), each with an associated
fitness
value,
into
a
new
population of offspring objects,
using the Darwinian principle of
natural
selection
and
using
operations that are patterned
after naturally occurring genetic

BASIC STEPS IN PROBLEM


SOLVING BY GENETIC ALGORITHM

CHROMOSOMAL REPRESENTATION OF
PARAMETERS

INITIAL POPULATION GENERATION

REPEAT IN A CYCLE
RANKNING
MATE

BY FITNESS

SELECTION

CROSS-OVER
MUTATION
NEW

POPULATION

TILL BEST DESIGN OPTIMAL SOLUTION


IS OBTAINED.

Typical
Chromosome

A POPULATION

Ranking by
Fitness

MATE
SELECTION

CROSSOVER

MUTATION

Best Target
Individual !!

Eureka !!!

THE GA CYCLE
RANKING BY FITNESS

POPULATION

MUTATION

BEST SOLUTION

PARENT SELECTION
CROSSOVER

Typical GA Operation
Overview and Flow Chart
Initialize population at random
Evaluate fitness of new
chromosomes
Good
Enough?

Yes

No

Select survivors (parents)


based on fitness
Perform crossover and
mutation on parents

Done

GA EXAMPLE

Maximise
f(x) = x 3 - 60 * x 2 + 900 * x +100
0 x 31

x can be represented using


5-bit binary digit string ( only
integer values )

ANALYTICAL SOLUTION

To find maximum or minimum df/dx = 0

3 x2 120 * x + 900 = 0

X = 30 and 10

To decide maximum or minimum d2f/dx2 = 0

6x 120 = 0

At x = 10 6x 120 = - 60
maximum = 4100

At x = 30 6x 120 = + 60 Hence y is
minimum =100

As GA maximizes it will identify the point x =


10

Hence y is

The Function
Plot

GA Example
Generate random individuals

Chromosome
P1
P2
P3
P4

Binary String
11100
01111
10111
00100
TOTAL
AVERAGE

f(x)

28
15
23
4

212
3475
1227
2804
7718
1929.50

GA Example

Choose parents, using roulette


wheel selection
Crossover point is chosen randomly
Mutation is applied randomly

GA Example Crossover
P3
P2

1
0

0
1

1
1

1
1

1
1

P4
P2

0
0

0
1

1
1

0
1

0
1

C1
C2

1
0

1
0

1
1

1
1

1
1

C3
C4

0
0

0
1

1
1

1
0

1
0

GA Example - After First Round of


Breeding
The average value has risen 1929 to 2933
P2 was the strongest individual in the initial
population. It was chosen both times but we have
lost it from the current population
We have a value of x=7 in the population which is
the closest value to 10 we have found analytically.

Chromosome
P1
P2
P3
P4

Binary String
11111
00111
00111
01100
TOTAL
AVERAGE

f(x)

31
7
7
12

131
3803
3803
3998
11735
2933.75

MOVEMENT OF POPULATION
( NPOP = 2 ) WITH GENERATIONS

Ge
n

P
1

P
2

Av

Max

1
7

3
1

1552

2973

1
7

2
5

1849

2973

1
7

0
8

3973

3972

0
8

0
9

4021

4069

0
9

1
0

4089

4100

20

1
0

1
4

3892

4100

FURTHER GENERATIONS

For this simple problem maximum was

identified in just 5 generations with 2


population members. Of course this is a
trivial problem.
Further examples and applications will show
the power of GA in solving more complex
problems involving several parameters.
But before that, let us brush up the basics of
optimisation as applied to engineering design
problems.