NATURAL EVOLUTION AS A

PROCESS OPTIMISATION
Dr. R. BALU,
Professor,
Department of Aeronautical Engineering,
Noorul Islam College of Engineering,
Kumaracoil, K K District, TamilNadu, 629180

Optimised Man!

Mr. Tracey Cool,

Rolls Royce
Traceys personal photograph collection

WR21 GAS TURBINE

ASSEMBLY

WR21 ENGINE HARDWARE

Marine Gas Turbine - WR21

WR21 Schematic

Spiral Recuperator

What Properties Make a Good

Heat Exchanger?

Effectiveness
maximised
How efficient the heat exchanger is

Volume
minimised
Smaller = lighter & cheaper

Pressure Drop Fraction

minimised

What Parameters Can Vary?

Parting

sheet thickness
Fin type
Fin thickness
Fin density
Layer height
Layer arrangement
Number of layers

Components of a Heat
Exchanger

Variable Parameters

Heat Transfer and Fluid Flow Service Handbook Sheet YM12 1992

Variable Parameters
Hot layer height
Cold layer height

hh

Hot layer fin thickness

Cold layer fin thickness

th

Hot layer fin density

Cold layer fin density
Number of layers

dh

hc
tc
dc
N

Limits on Variable Parameters

hh
hc
th
tc
dh
dc
N

3 - 20 mm
3 - 20 mm
0.2 - 0.7 mm
0.2 - 0.7 mm
150 m-1 - 1200 m-1
150 m-1 - 1200 m-1
50 - 250 layer pairs

What Properties Make a Good

Heat Exchanger?
Effectiveness
maximised
= (1-e(-NTU(1-C*)))/(1-C*e(-NTU(1-C*)))
C* 1
= NTU/(1-NTU)
C* = 1
Volume
Dimensionless: Vrat

minimised

Pressure Drop Fraction

minimised
10 % used as a limiting value here

Fitness

f/((1-)Vrat)

Contribution to chromosome

Variable

hh
hc

th
tc

dh
N
d
c
Whole system

Binary String

11 characters
11 characters
9 characters
9 characters
11 characters
11 characters
8 characters

70 character string

What our man has achieved ?

A genetic algorithm developed jointly by Rolls Royce and Rensselaer

Polytechnic Institute produced a high-performance marine engine design
that was

Four times better than a human-designed configuration

Much better than a configuration designed by an expert system by
successfully navigating a solution space containing more than 103870
possibilities.

the genetic algorithm discovered this solution on a typical engineering

desktop workstation after evaluating about 5000 possibilities in a typical
time frame of a day
Conventional methods for designing such turbines can take

up to five years and

cost over $2 billion;

Genetic Algorithm Parameters

70-character binary string for each gene

contains information on each of the 7
variables
50 binary strings in each population
100 generations
mutation rate of 0.1
elitism employed
fittest string is repeated in the next
generation

No. Of Theoretical Possibilities

70 character Binary string ( typical for

practical GA problems )
2 70 ~ 10 67 possibilities
No. of Nanoseconds in one billion years ~
10 9 x 365 x 86400 ~ 10 16

An unimaginable huge task !!

Fitness as Generations
Advance
100

Fitness

80
60
40
20
0
0

20

40

60

80

100

Generation

A four fold increase in fitness function !!

Initial and 100th Generation

Effectiveness

1
0.9
0.8
0.7
0.6
0.5
0

0.25

0.5

Volume Ratio

0.75

Variable Parameters Optimised

Hot layer height

Cold layer height
Hot layer fin thickness
Cold layer fin thickness
Hot layer fin density
Cold layer fin density
Number of layers

- 9.8 mm
- 4.7 mm
- 0.25 mm
- 0.25 mm
- 885 m-1
- 1095 m-1
- 230

Natures Starting Point !!

( Mr Tracey Cools gggg.
Grandfather !! )

NATURES OPTIMISATION

A gold fish which came from China, priced at $8350, Is displayed

during the Fishlook 2003 exhibition in Jakarta Indonesia, Singapore
and Malaysia are participating in the sevenday exhibition
to find the gold fish with the nicest colour and shape.

FISH MILLION YEARS AGO

NATURES CREATIONS

NATURES CREATIONS

POLAR BEAR AND ITS

SUUROUNDINGS

POLAR BEARS HUDDLE TOGETHER TO ESACPE FROM

BEING HUNTED OR SAVE THEMSELVES FROM A
SCUDDING SNOWSTORM

WHAT IS COVERED ?

OPTIMISTATION
CLASSICAL GRADIENT BASED METHODS
NATURAL EVOLUTION
NATURE TO COMPUTER MAPPING THE
GENETIC ALGORITHM
BASIC INGREDIENTS OF GA
WHY DO GAs WORK ? THE SCHEMATA
THEOREM
GA-170 GA DRIVER SOFTWARE
EXAMPLES OF APPLICATION
GA STRATEGIES - HOW TO MAKE THEM WORK
BETTER
CONCLUSIONS

THE ROMAN GODDESS OPS

WIFE OF SATURN
- THE GOD OF TIME
MOTHER OF
JUPITER
- THE GOD OF
POWER

A Fitting patroness of
science of
optimisation

Optimisation

To make the best or most effective use of

a situation or an opportunity

-- Oxford English Dictionary

OPTIMISATION

In a broader sense and in the context of

engineering design, it can be defined as a
process of finding or identifying the best from
among a collection of alternatives, without
explicitly evaluating or enumerating all of them.

In essence, it makes a desired quantity

maximum or an undesired quantity minimum,
when this quantity is a complex function of many
independent parameters or variables.

Optimisation Methods

Local Techniques

Global Techniques

Gradient Methods Random Walk

Quasi Newton Simulated Annealing

Simplex

Genetic Algorithms

OPTIMISATION AN EXAMPLE
Optimisation is at work at all times in our life, knowingly
or unknowingly
Adjusting the focusing knob of an optical instrument
( camera, binocular )
Objective function

------- Good Image

Variables
Focal length the lenses
Time of Exposure
Lighting in the environment

SOME MORE EXAMPLES

Other Examples:

Shape for the coffee pot handle

A nations weapons stock-pile
Mix of ingredients for strength of concrete
Process for refining petroleum
Shape of a flying object for minimum drag
Weight of Thermal Insulation for a
Re-entry Vehicle

Differential Calculus
Function of Single Variable
y = f (x)
max
y -

y y +

y +

y +

min

More Complex Functions

Directivity Pattern of
An Antenna

M500 183 ISSN 1350-8539

Where are we ?

I have found the

peak
I am also at the valley

I am at the Valley

Function of Several Variables

y = f ( x1, x2, x3, ..xi,..xN ) = f ( X )
x = (x1, x2, x3, ..xi,..xN )
Gradient ( First Order Derivatives )
f( x ) = ( f / x1, f / x2 , f / x3, f / xi, .. f / xN ) T = grad ( f )

Function of Several Variables

Hessian

Matrix ( Second Order

Derivatives )

2 f( x ) =
( 2 f / x12, 2 f / x1x2, 2 f / x1xN )
( 2 f / x1x2, 2 f / x22, 2 f / x1xN )

(.....)
(..)
( 2 f / xNx1, 2 f / xNx2, ..2 f / xN2 )

CONDITIONS FOR MAXIMA AND

MINIMA

f ( x ) = grad ( f ( x ) ) = 0
2 f ( x ) is positive definite for
minimum
is negative definite for
maximum
zero indicates inflection
point

Numerical Evaluation by Finite

Differences

First Order Derivatives

Simple Forward or Backward
Differences
Requires 2 function Evaluations for
each derivative
Totally 2N function Evaluations for
all derivatives

Numerical Evaluation by Finite

Differences
Second Order Derivatives
2f / xi2 needs 3 function Evaluations
2f / xi xn needs 4 function Evaluations
Totally 2N2 + 1 function Evaluations for all
terms of Hessian Matrix

GRADIENT SEARCH DIRECTION

At point x any

search direction d (t) is

a descent direction if f(x). d(t) 0.

The descent direction -f(x) gives the

maximum amount of descent. Hence it

is called as the steepest descent
direction.

GENERAL OPTMISATION
PROBLEM WITH CONSTRAINTS
Minimise f ( x ) subject to
gj(x)0,

j = 1, 2, 3, .. ...J

Inequality Constraints

hk( x ) = 0 ,

k = 1, 2, 3, ... K

Equality Constrints

xi(L) xi xi (U) I = 1, 2, 3,..N

A solution is feasible if and only if all the above are satisfied.
Any other solution is infeasible. Thus an infeasible point can
never be an optimum point.

KUHN-TUCKER
CONDITIONS
Lagrangian Multiplier Technique is used to convert the constrained
optimisation problem into an unconstrained problem by modifying
the objective function to include the constraints
f ( x ) uj gj(x) - vk hk(x) = 0
with
gj(x)0,

j = 1, 2, 3, .. ...J

Inequality Constraints

h k( x ) = 0 ,

k = 1, 2, 3, ... K

Equality Constrints

uj gj(x) = 0

j = 1, 2, 3, J

Only for Eq. constraints

uj 0

j = 1, 2, 3, ..J

STAGES IN HUMAN EVOLUTION

Stone Age - 2 MillionYears to Egyptian Dynasty

Paleolithic 15000 Years - Hunting, gathering.
No Technical Progress except perhaps Cave
Paintings
Mesolithic 8000 Years - Stone-Working Material /
Tools Beginning of Technological advancement
Neolithic - Bronze Age Innovations and Creativity
Stone/Wood/Metals Pyramids were built
Hellensitic - Greek Civilisation - 1 st Century BC
Middle Ages Roman Civilisation
Industrial Revolution
Modern Computer / Space Era 1950s Onward

DARWINIAN HYPOTHESIS

Charles Darwin On the Origin of Species by

means of Natural Selection ( 1859 )
Primary Driving Mechanism of Evolution is

Natural Selection

Individuals Better Adapted to environment will

leave behind , on an average, more off springs
than their less adapt colleagues.

Survival of the Fittest

GENETIC INFORMATION TRANSFER

The

information determining the

growth and development of individuals
of a species is encoded in their genes.
In sexual species, when two individuals
breed, complementary portions of their
genetic material
(chromosomes) are
passed on to their off spring randomly.

TWO ESSENTIAL INGREDIENTS

INHERITANCE
Off springs must retain at least some of the
features that made their parents fitter than
average. Otherwise evolution is reset at every
generation

VARIABILITY
At any given time individuals with varying
fitness must co-exist in the population.
Otherwise natural selection has nothing to
operate upon.

A POPULATION OF RABBITS

FOX PREYS ON THEM

RABBIT STORY
SOME RABBITS ARE FASTER AND SMARTER
THEY ARE LESS LIKELY TO BE EATEN BY
THE FOX
MORE OF SUCH SMARTER ONES TEND TO
SURVIVE ON AN AVERAGE
THEY TEND TO BREED AMONG
THEMSLEVES AND ALSO AMONG
LESS.SMARTER ONES

RABBIT STORY
IT RESULTS IN A GOOD MIX OF GOOD RABBITS
GENETIC MATERIAL
OF COURSE, EVEN DUMB AND SLOWER ONES
SURVIVE
BECAUSE OF LUCK, AND THEY ALSO MATE AND
BREED
THE OFF-SPRINGS WILL, ON AN AVERAGE, BE
SMARTER AND FASTER THAN THEIR PARENTS IN
ORIGINAL POPULATION BECAUSE ONLY THE
FITTEST SURVIVE WITH MORE PROBABILITY

NATURES PRINCIPLE
BEST INDIVIDUALS REPLICATE AND

GET PROPAGATED IN THE

POPULATION MORE AND MORE

AVERAGE INDIVIDUALS STAY ON

WORST

INDIVIDUALS WHITHER AWAY

AND DIE A NATURAL DEATH

SMART ONES SURVIVE

MIMICKING NATURAL
EVOLUTION
NATURE PROCESS
Generation

Initial Population
Fitness to Environment
Best ones having high fitness breed randomly
- Variability (at any time individuals of varying fitness coexist)

Next Generation n +1

MIMICKING NATURAL
EVOLUTION
MATHEMATICS

Generation 0
Select feasible sets of independent variables initial population
These are coded as genes (strings of binary / real numbers)
Rank them by the objective function and select the best few
Allow then to breed by crossing their genes at random positions
To introduce variability mutate the strings randomly

Next Generation ( iteration ? )

NATURE TO COMPUTER MAPPING

NATURE

COMPUTER

INDIVIDUAL OF THE SPECIES

POPULTION

POSSIBLE SOLUTIONS

POPULATION

SET OF SOLUTIONS

FITNESS TO ENVIRONMENT

OBJECTIVE FUNCTION

CHROMOSOME

CODING FOR INDEPENDENT

VARIABLES

SELECTION
CROSSS-OVER
MUTATION
EVOLUTION

SEARCH OPERATORS TO MIMIC

GENETIC BREEDING
OPTIMAL SOLUTION

GA HISTORY

1950 Modeling Biological Evolution

Even in the formative years of Electronic Computer

Revolution

1962 GEP BOX, GJ Fredman, WW Beldman -Evolutionary Algorithms for Optimisation and
Machine Learning
1965 Ingo Rechenberg , University of Berlin,
introduces evolution strategies, a method he
used to optimize real valued parameters for
devices. ( air foils )

GA HISTORY
Genetic algorithms were invented by John
Holland in the 1960s and were later developed by
Holland and his students and colleagues at the
University of Michigan in the 1960s and 1970s.
Hollands 1975 book Adaptation in Natural and
Artificial Systems presented the genetic
algorithms as an abstraction of biological evolution
and gave a theoretical framework for adaptation
under the genetic algorithms.

GA HISTORY
1980s saw many a real-world applications due to
Internet revolution and increase in power of Pcs

Pattern Recogonition
Flow Control Devices
Structural Optimisation
Micro-chips Design
Aerospace Applications
Micro-Biology

1990s saw commercial and business applications

Stock Market Predictions
Scheduling and Planning

PIONEERS OF GA
JOHN HOLLAND ( 1975 )
Digital Biology
Numerical Genetics

DAVID GOLDBERG ( 1985 )

MICHAELWICZ ( 1996 )
JOHN COZZA ( 1999)
Barbara Minsker ( 2003)
KALYANMOY DEB ( IIT KANPUR )

STARTING SHAPE

OPTIMISED SHAPE

DEFINITION OF THE GENETIC

ALGORITHM
PROF. JOHN COZZA ( STANFORD )
The genetic algorithm is a probabalistic search
algorithm that iteratively transforms a set
(called a population) of mathematical objects
(typically fixed-length binary character
strings), each with an associated fitness
value, into a new population of offspring
objects using the Darwinian principle of
natural selection and using operations that
are patterned after naturally occurring
genetic operations, such as crossover
(sexual recombination) and mutation .

BASIC
INGREDIENTS OF
GA

GENETIC RERESENTATION OF
INDEPENDENT VARIABLES
METHOD TO CREATE INITIAL
POPULATION
EVALUATION FUNCTION TO RATE
THE FITNESS ( OBJECTIVE
FUNCTION )
GENTIC OPERATORS TO EVOLVE THE
POPULATION TO NEXT GENERATION

The Genetic Algorithm Approach

Define limits of variable parameters

Generate a random population of
designs
Assess fitness of designs
Mate selection
Crossover
Mutation
Reassess fitness of new population

CHROMOSOMAL REPRESENTATION

BINARY STRING TO REPRESENT REAL

VALUES OF AN INDEPENDENT
VARIABLE X
MINIMUM VALUE OF X - XL
MAXIMUM VALUE OF X - XU
ACCURACY REQUIRED - d decimal
places
DOMAIN LENGTH 10d (XU -XL)
FIND N SUCH THAT
2 (N-1) < 10d (XU -XL) < 2N

CHROMOSOMAL
REPRESENTATION
THE BINARY STRING
(bn-1bn-2bn-3..b2,b1,b0) is equal to bi 2i i = 0,
1, 2,..N-1
Let this be X

The real number X = XL + X (XU -XL) / ( 2 N -1 )

String ( 000000.0000 ) corresponds to
XL

String ( 1111111..11111) corresponds

to XU

CHROMOSOMAL REPRESENTATION

EACH OF THE INPENDENT VARIABLE X (k) WILL BE

TRANSFORMED INTO BINARY STRING
OF LENGTH (m (i) ) as above.

IN THE COMBINED STRING, THE FIRST m1 BITS WILL

REPRESENT VARIABLE x1,, THE NEXT m2 BITS x2 AND
SO ON. THE TOTAL STRING LENGTH WILL BE mi

10010110101101110001010 0101100100010

x1

x2

x3

x4

TOTAL STRING LENGTH M = mk

x5
k =1,2,3..K

GENERATION OF INITIAL
POPULATION

GENERATE RANDOM NUMBERS BETWEEN 0

and 1. IF IT LIES BETWEEN 0 and 0.5 TAKE THE
BIT AS 0 AND IF IT LIES BETWN 0.5 and 1.0
TAKE THE BIT AS 1
GENERATE M RANDOM NUMBERS AND
CREATE THE FIRST SET OF INDEPENDENT
VARIABLES WITHIN THEIR ALLOWABLE RANGE
SIMILARLY GENRATE NPOPSIZE SETS. THIS
CONSTITUTES THE INITIAL POPULATION FOR
THE PROBLEM.
NPOPSIZE CAN BE TYPICALLY AS LOW AS 10
AND AS HIGH AS 100

DECODING AND FITNESS

EVALUATION
DECODE THE BINARY STRING TO
REAL VALUES OF VARIABLES x(i)
i=1,2,3.k , FOR EACH SET OF
THE POPULATION
Vi ( x1i, x2i, x3i, .xki) i = 1,2,3.NPOPSIZE
Evaluate the Objective function for each of v(i)
f(i) = obj(v(i)) I = 1,2,3..NPOPSIZE

FITNESS EVALUATION
Find Total OBJECTIVE FUNCTION F = f (i)
i=1,2,3.NPOPSIZE
Calculate the probability of selection
according to its own fitness

p(i) = f(i) / F

Find the cumulative probability q(i) = p(j) j =

1,2,3,,I

SELECTION METHODS

ELITIST SELECTION
FITNESS PROPORTIONATE SELECTION
ROULETTE WHEEL SELECTION
SCALING SELECETION
TOURNAMENT SELECTION
RANK SELECTION
GENERATIONAL SELECTION
STEADY STATE SELECTION
HIERARCHIAL SELECTION

SELECTION ROULETTE
WHEEL

GENERATE A RANDOM NUMBER r

If r < q(1) choose V(1)
If not choose V(i) such that
q(i-1) < r < q(i )
It is likely that same chromosome may be
selected more than once but that will have a
high fitness ratio ( a big chunk in the
roulette wheel ). This is consistent with
Natures operations.

Mate Selection Roulette

16%
16%

0%
0%
25%
25%

7%
7%
3%
3%
11%
11%
38%
38%

PROBABILISTIC SELECTION BASED

ON FITNESS
Better individuals are preferred
Best is not always picked
Worst is not necessarily excluded
Nothing is guaranteed
Mixture of greedy exploitation and

adventurous exploration

CROSS-OVER
OPERATION
Decide a cross-over probability Pc
For each chromosome generate a
random number r
If r < Pc select that chromosome for
the cross-over operation
Take a pair of chromosomes from
among the selected pool

CROSS - OVER
Generate a random integer in the range [ 1,2
.m-1 ] Let it be pos
Let the binary representation of the pair selected
be
V(1) = ( b1,b2,b3,..bpos, bpos+1, ..bm) and
V(2) = ( c1,c2,c3,..cpos,cpos+1,.cm )

CROSS-OVER OPERATION
Cross Over Operation is done to the
right of bit bpos and the new strings are
now
V(1) = ( b1,b2,b3,..bpos, cpos+1, ..cm)
V(2) = ( c1,c2,c3,..cpos, bpos+1,.bm )

MUTATION OPERATION
DONE

ON A BIT BY BIT BASIS

DECIDE ON A MUTATION
PROBABILITY-- Pm
Beware:
Too less Pm variability is reduced
Too large Pm best chromosomes
are lost unnecssarily

MUTATION
Generate a random number r
If r < Pm mutate that bit ( 0 to 1 or 1 to 0 )
Now the new population is ready for next
evaluation of fitness
The above operations are done in a cyclic
manner many times ( many generations )
The ultimate population is expected to
contain the best chromosome
( optimum solution ).

SPECIAL GA OPERATORS

ELITISM -- GOOD INDIVIDUALS ARE

PRESERVED DURING ANY GENERATION

NICHE POOL OF BEST INDIVIDUALS

MICRO-GA OPERATES ON SELECTED

INDIVIDUALS

Comparisons

Traditional . Vs. Evolution Based Methods

Traditional
best guess
may lead to local, not global optimum

Nature
population of guesses
more likely to find a better solution

Comparison
Nature .vs. Computer GA

Nature

not very efficient

at least a 20 year wait between generations
not all mating combinations possible

Computer Genetic algorithm

efficient and fast

optimisation complete in a matter of minutes
mating combinations governed only by
fitness

MINIMISATION
GA

TEND TO MAXIMISE A GIVEN

FUNCTION
TO MINIMISE AN OBJECTIVE
FUNCTION DEFINE IT AS
NEGATIVE OF ORIGINAL
min f(x) = max { -f(x)}
We can also redefine the
objective function as
f(x) = 1/ {1+f(x) }

HANDLING NEGATIVE FUNCTION

VALUES

GA ASSUMES THAT OBJECTIVE

FUNCTION IS ALWAYS POSITIVE

TO ACCOUNT FOR NEGATIVE

VALUES WE ADD A CONSTANT C

max (f(x)) = max { f(x)+C}

HANDLING CONSTRAINTS WITHIN GA

FRAMEWORK

gi(x) 0

i=1,2,3.m

hj(x) = 0 j=1,2,3,.n
Redefine f(x) as
f(x) = f(x) + r [ gi(x) ] where

r is penalty coefficient and is the penalty

function
Some constraints may be violated depending
on r

EXPLORATION AND
EXPLOITATION
RANDOM SEARCH EXPLORES THE SEARCH
SPACE - IGNORES EXPLOITATION OF
POTENTIAL REGIONS OF POSSIBLE
SOLUTIONS
HILL CLIMBING DOES NOT EXPLORE THE
SEARCH SPACE BUT EXPLOITS THE BEST
SOLUTION FOR IMPROVEMENT
GENETIC ALGORITHMS STRIKE A
REMARKABLE BALANCE BETWEEN
EXPLORATION AND EXPLOITATION

ADVANTAGES OF GA

BASED ON SIMPLE PRINCIPLES OF

NATURAL EVOLUTION NO GREAT
MATHMATICS
DOES NOT REQUIRE COSTLY EVALUATION
OF FIRST OR SECOND DERIVATIVES OF
OBJECTIVE FUNCTION
ALWAYS TEND TO FIND THE GLOBAL
MINIMUM OR MAXIMUM DOES NOT GET
BOGGED DOWN IN LOCAL HILLS AND
VALLEYS

ADVANTAGES OF GA

VERY USEFUL FOR PROBLEMS

WITH TOTALLY UNKNOWN
CHARACTERISICS OF OBJCTIVE
FUNCTION HAVING LARGE
NUMBER OF PARMETERS
NOT AFFECTED BY LOCAL
DISCONTINUITIES
CAN HANDLE NOISY AND
MULTIMODAL FUNCTION SPACE

DISADVANTAGES OF GA

NOT BASED ON SOUND

THEOETICAL CONSIDERATIONS
EXCEPT THAT ON SCHEMATA
TEMPLATES
DIFFICULTY IN FINDING EXACT
GLOBAL MAXIMA OR MINIMA
LARGE NUMBER OF FUNCTION
EVALUATIONS

DISADVANTAGES OF
GA
BEING STOCAHSTIC IN NATURE,
CONVERGNCE IS RATHER SLOW
PROBLEM SPECIFIC SETTING OF
VARIOUS PROBABILITIES ( CROSSOVER MUTATION ETC ) THOUGH
SOM EMPIRICAL RULES ARE
KNOWN
IDEAL POPULATION SIZE IS NOT
EASY TO CHOOSE

HOW GA OPERATES ?
Imagine an army of
parachutists dropping onto
the landscape of a
problem's search space,
with each one being given
orders to find the highest
peak. Small mutations
enable each individual to
explore its immediate
neighborhood, while
selection focuses
progress, guiding the
algorithm's offsprings
uphill to more promising
parts of the solution space.
Cross-over enables
transfer of information
among the parachutists.

GENETIC ALGORITHM
Generatio
n 0
Individual
s

Fitness

Generation
1
Offspring

011

$3

111

001

$1

010

110

$6

110

010

$2

010

CHROMOSOME (GENOME) OF THE

GLOBAL OPTIMUM
OPTIMUM SOLUTION
1

THE SEARCH SPACE

1
2
3
4
5
6
7
8

000
001
010
011
100
101
110
111

Alphabet size K=2, Length L=3

Size of search space: KL = 2L = 23 =8

GENERATION - 0
Generation 0
1
011
3
2
001
1
3
110
6
4
010
2
Total
Worst
Average
Best

DARWINIAN FITNESS
PROPORTIONATE SELECTION
Generation 0

1
011
2
001
3
110
4
010
Total
Worst
Average
Best

3 .25
1 .08
6 .50
2 .17
12
1
3.00
6

Mating pool

011
110
110
010

3
6
6
2
17
2
4.5
6

PROBABILISTIC SELECTION BASED ON

FITNESS

MUTATION OPERATION
Parent chosen probabilistically based on

fitness

Parent
010

Mutation point chosen at random

Parent
One offspring

--0

Offspring
011

AFTER MUTATION OPERATION

Generation 0

Mating pool

Generation 1

011

.25 011

001

.08 110

110

.50 110

010

.17 010

Total

12

17

Worst

Average

3.00

4.5

Best

---

011

CROSSOVER OPERATION

2 parents chosen probabilistically

based on fitness
Parent 1 Parent 2
011

110

CROSSOVER (CONTINUED)
Interstitial point picked at random

Fragment 1 Fragment 2
0111 2 remainders

Remainder Remainder
1
2
--1
--0

2 offspring produced by crossover

Offspring 1 Offspring 2
111

010

AFTER CROSSOVER OPERATION

Generation 0

Mating pool

Generation 1

011 3

.25 011

111

001 1

.08 110

010

110 6

.50 110

010 2

.17 010

Total

12

17

Worst

Average

3.00

4.5

Best

AFTER REPRODUCTION OPERATION

Generation 0

Mating pool

011 3

.25

001 1

.08

110 6

.50 110

010 2

.17

Total

12

17

Worst

Average

3.00

4.5

Best

Generation 1

--- 110

GENERATION 1
Generation 0 Mating
pool
1
011 3
.25 011 3
2
001 1
.08 110 6
3
110 6
.50 110 6
4
010 2
.17 010 2
Total
12
17
Worst
1
2
Average 3.00
4.5
Best
6
6

Generation 1
2
2
-----

111
010
110
011

7
2
6
3
18
2
4.5
7

PROBABILISTIC STEPS

The initial population is typically

random
Probabilistic selection based on fitness
- Best is not always picked
- Worst is not necessarily excluded

Random picking of mutation and

crossover points
Often, there is probabilistic scenario as
part of the fitness measure

GENETIC ALGORITHMS
The genetic algorithm is a probabilistic search
algorithm that iteratively transforms a set
(called a population) of mathematical objects
(typically fixed-length binary character
strings), each with an associated fitness
value, into a new population of offspring
objects using the Darwinian principle of
natural selection and using operations that
are patterned after naturally occurring
genetic operations, such as crossover
(sexual recombination) and mutation.

BASIC STEPS IN PROBLEM SOLVING

BY GENETIC ALGORITHM

CHROMOSOMAL REPRESENTATION OF
PARAMETERS
INITIAL POPULATION GENERATION
REPEAT IN A CYCLE

RANKNING BY FITNESS
MATE SELECTION
CROSS-OVER
MUTATION
NEW POPULATION

TILL BEST DESIGN OPTIMAL

SOLUTION IS OBTAINED.

Typical Chromosome

A Population

Ranking by Fitness

Mate Selection

Crossover

Mutation

Best Design

The GA Cycle

BASIC QUESTIONS
HOW DURING GA - CYCLE
OPERATIONS, WHICH ARE RANDOM IN
NATURE, THE BEST SOLUTION ( BEST
CHROMOSOME) IN EACH GENERATION
ALWAYS SHOWS A TENDENCY TO
MOVE TOWARDS INCREASING FITNESS
( OPTIMUM SOLUTION ) ?
HOW DOES GA MOVE TOWARD
OPTIMUM WITHOUT EVALUATING ALL
POSSIBLE SOLUTIONS ?

GENETIC ALGORITHM DRIVER

GA170
DEVELOPED

BY M.CAROLL CORNELL
UNIVERSITY AEROSPACE ENGG
DEPT
VERSION 1.7 RELEASED IN 2001
CODED IN FORTRAN
USER SPECIFIED SUBROUTINE FUNC

GA170 GA DRIVER

MAIN

ga.inp

CODE
DECODE
CROSOVR
EVALOUT
INPUT
INITIAL
MUTATE
NEWGEN
NICHE
POSSIBL
RAN3
RESTART
SELECT / SELECTN
SHUFFLE
GAMICRO

FUNC

PARAMS
OUTPUT

SINGLE VARIABLE
FUNCTION

LOW x Value the curve is a Rectangular Hyperbola

High x Value the curve is a parabola
Minimum occurs at x = 3.0 and f(x) = 27.0

LISTING OF SUBROUTINE

c###################################
c SINGLE VARIABLE OPTIMISATION PROBLEM x*x+54/x
c####################################
subroutine func(j,funcval)
implicit real*8 (a-h,o-z)
save
include 'params.f'
dimension parent(nparmax,indmax)
dimension iparent(nchrmax,indmax)
c
dimension parent2(indmax,nparmax),iparent2(indmax,nchrmax)
common / ga2 / nparam,nchrome
common / ga3 / parent,iparent
x1=parent(1,j)
funcval=-(x*x+54/x)
return
end
c##########################################

GA CONVERGENCE

HIMMELBLEAU FUNCTION
2
2
2
2
f x x1 x 2 11 x 2 x1 7

###########################################
c TWO VARIABLE OPTIMISATION PROBLEM
c##########################################
subroutine func(j,funcval)
implicit real*8 (a-h,o-z)
save
include 'params.f'
dimension parent(nparmax,indmax)
dimension iparent(nchrmax,indmax)
c
dimension parent2(indmax,nparmax),iparent2(indmax,nchrmax)
common / ga2 / nparam,nchrome
common / ga3 / parent,iparent
x1=parent(1,j)
x2=parent(2,j)
funcval=(x1**2+x2-11)**2+(x2**2+x1-7)**2
return
end
c##########################################

SURFACE PLOT OF HIMMELBLEAU FUNCTION

MINIMUM OCCURS AT X1=3 and X2=2 and f(x1,x2)=

HIMMELBLAU FUNCTION MINIMISATION

################# Generation 1 ##############
#
Binary Code
Param1 Param2 Fitness
1 00011110001100100000001010 0.590 1.256 -111.63504
2 11010110010100110110010001 4.186 2.120 -77.56325
Average Values:
2.388 1.688 -94.59914
Average Function Value of Generation= -94.59914
Maximum Function Value
= -77.56325
Number of Crossovers
= 25
Elitist Reproduction on Individual 2
################# Generation 10 #############
#
Binary Code
Param1 Param2 Fitness
1 10010110001100100110011001 2.934 1.500
-4.09933
2 11010110011100100110011001 4.189 1.500 -65.04394
Average Values:
3.561 1.500 -34.57164
Average Function Value of Generation= -34.57164
Maximum Function Value
= -4.09933
Number of Crossovers
= 26
Elitist Reproduction on Individual 1

################# Generation 50 ################

#
Binary Code
Param1 Param2 Fitness
1 10010110001000010001011011 2.932 0.681 -15.94875
2 10010110001000110001011111 2.932 1.933
-0.32739
Average Values:
2.932 1.307
-8.13807
Average Function Value of Generation= -8.13807
Maximum Function Value
= -0.32739
Number of Crossovers
= 30
%%%%%%% Restart micro-population at generation 50 %%%%%%%

################# Generation 100 ############

#
Binary Code
Param1 Param2 Fitness
1 10010111001000110001101101 2.952 1.942
-0.19523
2 11010111001101110101101101 4.203 4.598 -463.59571
Average Values:
3.578 3.270 -231.89547
Average Function Value of Generation= -231.89547
Maximum Function Value
= -0.19523
Number of Crossovers
= 24
Elitist Reproduction on Individual 1
################# Generation 200 ############
#
Binary Code
Param1 Param2 Fitness
1 10010111100110110011010101 2.961 2.005
-0.05147
2 11101010010101001100101111 4.577 2.998 -210.69526
Average Values:
3.769 2.502 -105.37337
Average Function Value of Generation= -105.37337
Maximum Function Value
= -0.05147
Number of Crossovers
= 24
Elitist Reproduction on Individual 2
%%%%%%% Restart micro-population at generation 299 %%%%

################# Generation 300 #############

#
Binary Code
Param1 Param2 Fitness
1 10010111100110110011111101 2.961 2.030
-0.04723
2 01111101001111001111001110 2.446 3.095 -28.93732
Average Values:
2.704 2.562 -14.49227
Average Function Value of Generation= -14.49227
Maximum Function Value
= -0.04723
Number of Crossovers
= 28
Elitist Reproduction on Individual 2
################# Generation 400 #############
#
Binary Code
Param1 Param2 Fitness
1 10011010000000110010110101 3.008 1.986
-0.00359
2 11011010100001111010110101 4.268 4.799 -556.22319
Average Values:
3.638 3.392 -278.11339
Average Function Value of Generation= -278.11339
Maximum Function Value
= -0.00359
Number of Crossovers
= 30
Elitist Reproduction on Individual 2

CONVERGENCE OF GA FOR HIMMELBLEAU FUNCTION

POPULATION MOVEMENT IN X1-X2 PLANE

EXAMPLE FUNCTIONS

F (x1 , x2 ) = 21.5 + x1 sin( 4 x1) + x2 sin ( 20 x2 )

2 variables x1 , x2
Range of x1 -3.0 to 12.1
Range of x2 4.1 to 5.8
Required Precision 4 decimals
Range of x1 must be divided into 15.1 x 10000 points
Range of x2 must be divide into 1.7 x 10000 points
N for x1 is 18 as 2 18 > 15100
N for x2 is 15 as 2 15 > 17000
Total string length required is 18 + 15 = 33
Cross over Probability Pc = 0.25
Mutation Probability Pm = 0.01
NPOPSIZE = 20

SAMPLES LISTING OF
FUNC
C

SUBROUTINE FUNC ( j,funcval )

C
C

C
C
C

IMPLICIT REAL*8(A-H,O-Z)
SAVE
INCLUDE PARAMS.f
DIMENSION parent(nparmax,indmax)
DIMENSION iparent(nchrmax,indmax)
common/ga2/nparam,nchrome
common/ga3/parent,iparent
Define constants
pi=4.*dtan(1.D0)
Redefine parameters from parent array
x1=parent(1,,j)
x2=parent(2,,j)
t1=dsin(4.*pi*x1)
t2=dsin(20.*pi*x2)
compute func val
funcval=21.5+t1*x1+t2*x2
add penalty if any
return
end

F(x1,x2)=21.5+x1sin(4x1)+x2sin(20x2)
THIS FUNCTION HAS ROUGHLY 660 PEAKS AND VALLEYS !!

BHEESHMAS LEGENDARY BED OF ARROWS !!

SAMPLE CHROMOSOME
AND FITNESS
010001001011010000111110010100010
---------- x1 ------------

--------------x2 -------------------

DECODOING

x1 = 1.052426 x2 = 5.755330

Fitness f( x1, x2 ) = 20.252640

INITIAL POPULATION

FITNESS VALUES

V(2)=7.580015

V(15)=30.0603

V(15) is the strongest and v(2) is the weakest chromosome

TOTAL FITNESS = eval(vi) = 387.776822

SELECTION PROBABILITY

CUMULATIVE PROBABILITY

ROULETTE WHEEL SELECTION

POPULATION RANKED FOR

SELECTION

CROSS OVER OPERATION

If ( r < 0.25 ) select the chromosome for cross over operation

POPULATION AFTER
CROSSOVER

MUTATION OPERATION

POPULATION AFTER MUTATION

FITNESS OF 1 GENERATION
st

V(11)=33.35187

TOTAL FITNESS = eval(vi) = 447.049688

AFTER 1000 GENERATIONS.

FITNESS OF POPULATION
AFTER 1000th GENERATION

V(11)=35.47793

GLOBAL MAXIMUM VALUE = 38.827553

X1=11.6237 x2=5.7264

AVERAGE AND BEST FUNCTION

VALUES

CONVERGENCE OF
PARAMETERS

ANTENNA DESIGN

ANTENNA DESIGN
The problem (Altshuler and Linden 1998) is to
determine the x-y-z coordinates of the 3dimensional position of the ends (X1, Y1, Z1,
X2, Y2, Z2, , X7, Y7, Z7) of 7 straight wires so
that the resulting 7-wire antenna satisfies
certain performance requirements
The first wire starts at feed point (0, 0, 0) in the
middle of the ground plane
The antenna must fit inside the 0.5 cube

ANTENNA GENOME
X1

Y1

Z1

X2

Y2

Z2

+0010

-1101

+0000

+0010

-0111

+0010

105-bit chromosome (genome)

Each x-y-z coordinate is represented
by 5 bits (4-bit granularity for data
plus a sign bit)
Total chromosome is 3 7 5 = 105
bits

ANTENNA FITNESS
Antenna is for ground-to-satellite

communications for cars and handsets

We desire near-uniform gain pattern 10
above the horizon
Fitness is measured based on the
antenna's radiation pattern. The
radiation pattern is simulated by
National Electromagnetics Code (NEC)

ANTENNA FITNESS
Fitness is sum of the squares of the

difference between the average gain

and the antenna's gain
Sum is taken for angles between -90
and +90 and all azimuth angles from
0 to 180
The smaller the value of fitness, the
better

GRAPH OF ANTENNA FITNESS

U. S. PATENT 5,719,794

Altshuler, Edward
and Derek Linden

2-STRUSS BEAM
2P

t
d

SECTION C-C

C
C
2B

PARAMETERS RANGES
AND CONSTRAINTS

CONSTANT PARAMETERS
LOAD P 33000 UNITS
HALF SPAN B 30 UNITS
THICKNESS 0.1 UNITS
MAXIMUM ALLOWABLE
STRESS 100,000 UNITS
ELASTIC MODULUS 3 x 10 7
UNITS
DENSITY 0.3 UNITS

INDEPENDENT VARIABLES
DIAMETER OF PIPE d
1 d 3
HEIGHT OF THE STRUSS H
10 H 35

CONSTRAINTS

THE BEAM SHOULD NOT FAIL IN

BUCKLING
B y
Yield Stress should be less than
allowable stress
y al
YIELD STRESS MUST BE LESS THAN
ALLOWABLE STRESS
OPTIMISE THE WEIGHT OF THE BEAM

STRESS FORMULAE
YIELD STRESS y
y = P(B2+H2)1/2 / { H t d }
BUCKLING STRESS b
b = {2E ( d2 + t2 ) / {8 ( B2 + H2 )}
W = 2 d t ( B 2 + H2 )

LISTING OF SUBROUTINE

################################
c 2-STRUSS BEAM
c###############################
subroutine func(j,funcval)
implicit real*8 (a-h,o-z)
save
include 'params.f'
dimension parent(nparmax,indmax)
dimension iparent(nchrmax,indmax)
c
dimension parent2(indmax,nparmax),iparent2(indmax,nchrmax)
common / ga2 / nparam,nchrome
common / ga3 / parent,iparent
c Constants
P=33000.
b=30.
t=0.1
SIGM=100000.
E=3.E+07

LISTING FOR BEAM PROBLEM

rho=0.3
pi=4.0d0*datan(1.d0)
d=parent(1,j)
h=parent(2,j)
hh=b*b+h*h
W=2.*pi*rho*d*t*hh**0.5
SIGY=P*hh**0.5/(pi*t*h*d)
SIGB=pi*pi*E*(d*d+t*t)/(8.*hh)
CONST1=SIGB-SIGY
CONST2=SIGY-SIGM
funcval=-W
c Penalty for constraint violations
if(const2.gt.0.) funcval=-W-1.E-04*CONST2
if(const1.lt.0.) funcval=-W+1.E-04*CONST1
return
end
c#############################################

OPTIMUM SOLUTION
CONJUGATE

GRADIENT
d = 2 Units H =19 Units W =
13.4 Units

Genetic
d

Algorithm

= 1.9 Units H = 18.9 units W =

12.3 Units

TYPICAL
APPLICATIONS OF GA

Machine Learning
Speech Recognition
Criminal Suspect Detection
Music Composition
Training of Neural Networks
Aircraft Design
Turbine Blade Design
Protein Structure Determination
Bidding Strategies
Image Processing

A genetically optimized three-dimensional

truss with improved frequency response .
Used for satellite boom which can be assembled in

SUCCESS OF GA SIMULATIONS

ROBUST VARIABLE REPRESENTATION

PROPER FITNESS EVALUATION MEASURE

TOLERATE RANDOM CHANGES

INCREASE SHOULD LEAD TO BETTER
SOLUTION

SELECTION STRENGTH AND TYPE

MUTATION AND CROSS-OVER
PROBABILITIES
POPULATION SIZE

GUIDELINES FOR SELECTION OF

GA PARAMETERS

DEPENDS ON PROBLEM BEING

SOLVED
BASED ON EXPERIENCE AND
DATA BASE ACCUMULATED
( BARBARA MINSKER (2003))
HELP TO ACHIEVE
CONVERGENCE QUICKLY WITH
LESS COMPUTATIONAL EFFORT

GA PARAMETERS
NUMBER OF FUNCTION
EVALUATIONS N = G x P

G Number of Generations to Converge

P Population Size

G is found to be approximately
G2L

, L is the binary string length

Optimum Population Size is

P 1.4 L

Hence N 3 x L 2

GA PROBABILITIES VALUES Pc
and Pm

Cross Over Probability Pc

Mutation Probability Pm

Pc = ( S-1) / S where S is the number of

parents chosen for mating For S=2, Pc = 0.5

Pm = 1 / P 1 / L

GA STRATEGY
Run the smallest population size (N).

After some generations, double the

population size ( 2N ) and inject some
of the best individuals of ( N ) into the
population ( 2N )
This is found to result in 70 %
reduction in number of generations to
converge.

GA STRATEGY
If the function evaluation is costly
then GA may take a very long time
If G = 200 and P = 20 the N = 4000
If we are dealing with simple airfoil
computations using CFD, each run
may take say 2 hours on a parallel
machine ( 4 or 5 Processors ).
Then 4000 evaluations may take
8000 hours i.e. ( an year !! )

GA STRATEGY
As GA proceeds, number of function

evaluations are made and a huge data

base is being created.
Can we make use of the already
available data to evaluate the function
by some other quicker methods ? So
that after say 50 or 60 generations we
use this quicker method for objective
function evaluation
NEURAL NETWORKS can be used
effectively for such purpose.

Conclusions

GAs are powerful optimisers based on

simple rules of natural evolution
They can be applied beneficially for problems
where nature of objective function is
complex and unknown
They can be combined with sophisticated
softwae tools, to produce best or optimum
aerodynamic configuration designs
They give rise to data bases, which can be
further used, for optimisation tasks, based
on neural networks