Evolutionary Optimization Techniques

Guided By
Krishnanand K.R.
Presented By:
Ramanand K.R.
S1S2 - EEE
MEA Engineering College
Krishnanand K.R.
Senior Research Associate
SOA University, Bhubaneswar
Overview of GA
Introduction To Bio-Inspired EAs
Contents
Conclusion
Test Functions
Applications
Introduction
Evolutionary Computation is the heuristic subset of Computational
intelligence. Evolutionary Optimization is concerned with obtaining the
optimal result of the system studied, using phenomena related to evolution.
Bio-inspired evolutionary optimization techniques simulate biological
phenomena through iterative stochastic algorithms to optimize
Bio-inspired Evolutionary Algorithms (EAs) for Optimization
phenomena through iterative stochastic algorithms to optimize
mathematically modeled systems.
Evolutionary optimization can be used for solving easily various complex
engineering problems with high computational burden and hence is a very
handy tool for an engineer.
Introduction
Bio-inspired Evolutionary Algorithms (EAs) for Optimization
Based on Biological Processes
Stochastic Search Methodology
Used for Global Optimization (finding global minimum or maximum)
1
Initialize The
Population
2
Evaluate The
Function
Population
Introduction
(
(
(
(
(
(

MN M1
2N 21
N 1 12 11
... ...
... ... ... ...
... ... ... ...
... ...
....
x x
x x
x x x
General Steps
3
Reproduction
Till Terminating
Condition
4
Mutation
Population
M Population size
N Number of Variables
X Deciding Variables
Y Fitness Value
(
(
(
(
(
(

M
2
1
Y
...
...
Y
Y
Introduction
The biological phenomena followed by the evolutionary algorithms vary from
simulating the behaviour of chromosomes to advanced organisms like flocks
of birds.
Genetic Algorithm was the earliest among the evolutionary computation.
There are many EAs available today like Particle Swarm Optimization (PSO),
Evolutionary Algorithms & Conventional Methods
There are many EAs available today like Particle Swarm Optimization (PSO),
Differential Evolution (DE), Shuffled Frog Leaping (SFL), Bacteria Foraging
Optimization (BFO), Artificial Bee Colony (ABC), Invasive Weed Optimization
(IWO) etc.
Conventional methods of optimization of non-linear problems consists of
dynamic programming, nonlinear programming, lagrange relaxation method
and various such gradient based methods.These methods are unsuitable
often due to their heavy computation requirements for large scale problems.
Introduction
Property Evolutionary Methods Conventional Methods
Search space Population of potential solutions Trajectory by a single point
Motivation Natural selection and social adaptation Mathematical based properties (like gradient)
Applicability Domain independent, generic in nature Applicable to a specific problem domain
Point Transition Probabilistic Deterministic
Evolutionary Algorithms & Conventional Methods
Point Transition Probabilistic Deterministic
Prerequisites An objective function to be optimized Auxiliary knowledge such as gradient vectors
Initial guess Automatically generated by the algorithm Provided by user
Flow of Control Mostly parallel Mostly serial
CPU time Generally high Less (for small problem size)
Results Global optimum more probable Local optimum, dependant on initial guess
Advantages Global search, parallel, speed Convergence proof
Drawbacks No formal convergence proof Local convergence, high computational cost
Genetic Algorithm (GA)
Overview of GA
It was invented by Prof. John Holland at the university of Michigan in 1975.
This algorithm is based on the natural evolution of any species through
crossover and mutation of chromosomes.
The chromosomes are a set of fixed number of genes and they have fitness
values of their own. values of their own.
Binary Coded Genetic Algorithm is most commonly used.
A Typical Chromosome
1 0 1 1 1 0 0 1 0 1 0 0 0 0 1 1 1 0 1 1 0 1 0 1 1 0 1 1 0 1
Overview of GA
Steps in GA
1) Choose the initial population randomly as per the
search space, i.e. assign values to the matrix
according to the assumed lower and upper limits of
the X values.
(
(
(
(
(
(
(

2N 21
N 1 12 11
... ...
... ... ... ...
... ... ... ...
... ...
....
x x
x x
x x x
2) A function Y = f(x
1
, x
2
, x
3
,, x
N
) represents a
mathematically modeled system based on its input-
output relations. Evaluate the fitness of the
population so as to obtain the Y values.
(

MN M1
... ... x x
(
(
(
(
(
(

M
2
1
Y
...
...
Y
Y
Overview of GA
Steps in GA
3) Select a few pairs of individuals or chromosomes to reproduce.
4) Breed new generations through crossover and mutation.
E.g. Two parent individuals cross over to produce a child and the child
undergoes mutation (possibly becoming genetically more superior).
0 1 1 0 0 1 0 0 1 1 0 1 1 0 1 0 0 0 1 0 1 1 0 1 0 1 0 1 1 0
1 0 1 1 1 0 0 1 0 1 0 0 0 0 1 1 1 0 1 1 0 1 0 1 1 0 1 1 0 1
1 0 1 1 1 0 0 1 0 1 0 0 0 0 1 0 0 0 1 0 1 1 0 1 0 1 0 1 1 0
1 0 1 0 1 0 0 1 0 1 0 1 0 0 1 0 0 0 1 0 1 1 0 0 0 1 0 1 1 0
After Cross-Over
After Mutation
Overview of GA
Steps in GA
4) Evaluate the new population thus generated and reinsert the new generation
in to the existing population according to their superiority. This is based on the
Darwinian principle of Survival of the Fittest.
5) Continue selection, crossover, mutation and reinsertion until the termination
condition. The termination condition can be a specific number or iterations or condition. The termination condition can be a specific number or iterations or
until a specific value (which is considered near to the optimum) is achieved.
Since it is an iterative method, the best individual at each iteration is
preserved. This is called Elitist Strategy. The best among the elites is the
desired optimal solution.
Applications
EAs can be applied to any optimization problem due to their generic nature.
They have wide applications in Engineering and Business. Any system design
problem, allocation problem, profit maximization problem or scheduling problem
can be formulated as an optimization problem.
Sample Applications
In Electrical Engineering, the Economic Load Dispatch (ELD) problem is a In Electrical Engineering, the Economic Load Dispatch (ELD) problem is a
complex problem of fixing the power generation done by the thermal generating
units at a certain time, so as to minimize the cost of production. The cost of
power generation is formulated as a function of power generated by each unit,
subject to many practical constraints. EAs can be used to find the optimal cost.
In Electronics, parameter estimation of a filter to achieve a particular frequency
filtering capability can be done by EAs.
Test Functions
To test the efficacy of a stochastic algorithm with no proof of convergence,
some statistical methods are adopted.
The algorithm is tried with certain benchmark functions whose optimal point
and the solution (that gives such an optimal point) are known. The extent to
which such mathematical functions are optimized is observed statistically
using many trial runs.
The actual dimension and the search domain of a problem can be very large.
The 3D plots of a few 2D functions are only presented, within a limited
search space. All the following functions are minimization problems.
A few such test functions are presented here.
Test Functions
1) Ackely Function
( )
1
N
x 2 cos
N
2
i
x
b.
N 2 1
e a e a.e ) x ,...., x , F(x
N
i
i
N
i
+ + =

-15 x
i
30 Global minimum f(x)=0 at all x
i
=0
Test Functions
2.a) Griewank Function
1 ) i cos(x
4000
x
) x ,...., x , F(x
N
1 i
0.5
i
N
1 i
2
i
N 2 1
+

=

=
=
-600 x
i
600 Global minimum f(x)=0 at all x
i
=0
Test Functions
2.b) Griewank Function (Same function, Plot for a larger search space)
-600 x
i
600 Global minimum f(x)=0 at all x
i
=0
Test Functions
3) Levy Function
-10 x
i
10
Global minimum f(x)=0 at all x
i
=1
Test Functions
4) Michalewics Function
( )

|
|

\
|
|
|

\
|
=
=
N
1 i
20
2
i
i N 2 1

x i
sin x sin ) x ,...., x , F(x
0 x
i

Global minimums : N=2, f(x)=-1.8013; N=5, f(x)=-4.687658; N=10, f(x)=-9.66015
0 x
i

Test Functions
) .x 10.cos(2. x 10.N ) x ,...., x , F(x
i
N
1 i
2
i N 2 1
+ =

=
5) Rastrigin Function
-5.12 x
i
5.12
Global minimum f(x)=0 at all x
i
=0
Test Functions
6) Schwefel Function

=
|

\
|
=
N
1 i
N 2 1
x sin x N 418.9829 ) x ,...., x , F(x
-500 x
i
500
Global minimum f(x)=0 at x
i
=1
Test Functions
7) Shubert Function
( ) ( )

= =
|

\
|
+ + =
N
1 j
N
1 i
j N 2 1
x 1 i i cos i ) x ,..., x , F(x
-10 x
i
10 Global minimum f(x)= -186.7309 for 18 different (x
1
,x
2
) Values
Test Functions
8) Sphere Function

=
=
N
1 i
2
i N 2 1
x ) x ,...., x , F(x
-5.12 x
i
5.12
Global minimum f(x)=0 at all x
i
=0
Test Functions
9) Zakharov Function
4
N
1 i
i
2
N
1 i
i
N
1 i
2
i N 2 1
) 0.5.i.x ( ) 0.5.i.x ( x ) x ,...., x , F(x

= = =
+ + =
-5 x
i
10
Global minimum f(x)=0 at all x
i
=0
Conclusion
Evolutionary Optimization techniques, being generic in nature, can be applied to
a variety of practical engineering problems.
These algorithms are flexible and can form a hybrid algorithms to give better
performance.
The mathematical test functions give statistical credibility to the optimization The mathematical test functions give statistical credibility to the optimization
capability of the evolutionary optimization techniques.
EAs can be used on any real life system for superior optimization as compared
to the conventional techniques.
References
[1] Ioannis G. Damousis, Anastasios G. Bakirtzis, and Petros S. Dokopoulos, Network-Constrained Economic Dispatch
Using Real-Coded Genetic Algorithm, IEEE Trans. On Power Systems, Vol. 18, No. 1, February 2003, pp 198-205.
[2] C. Jiejin, M. Xiaoqian, L. Lixiang and P. Haipeng ,Chaotic particle swarm optimization for economic dispatch
considering the generator constraints, Energy conver. and Management 48, 2007, pp. 645-653.
[3] Dervis Karaboga, Bahriye Basturk, A powerful and efficient algorithm for numerical function optimization: artificial bee
colony (ABC) algorithm, Springer Science, pp. 459471, 2007.
[4] D. Karaboga, B. Basturk, On the performance of artificial bee colony (ABC) algorithm , Applied Soft Computing,
Elsevier Science , pp. 687697, 2008.
[5] A.R.Mehhrabian, C.Lucas, A novel numerical Optimization algorithm inspired from weed colonization, vol-1, issue-4, [5] A.R.Mehhrabian, C.Lucas, A novel numerical Optimization algorithm inspired from weed colonization, vol-1, issue-4,
pp355-366, Ecological Informatives, Elsevier Science, 2006
[6] Emad Elbeltagi, Tarek Hegazy, Donald Grierson, Comparision Among Five Evolutionary-based Optimization
Algorithms, Elsevier Science, Advanced Engineering Informatics, pp. 43-53, 2005.
[7] Krishnanand K.R., B.K. Panigrahi, Santanu Kumar Nayak, P.K. Rout, Comparative Study of Five Bio-Inspired
Evolutionary Techniques, 2009 World Congress on Nature & Biologically Inspired Computing, IEEE, pp.1231-1236.
[8] Santanu Kumar Nayak, B.K. Panigrahi, Krishnanand K.R., P.K. Rout, Application of Artificial Bee Colony to Economic
Load Dispatch Problem with Ramp Rate Limits and Prohibited Operating Zones, 2009 World Congress on Nature &
Biologically Inspired Computing, IEEE, pp.1237-1242.
[9] Krishnanand K.R., Santanu Kumar Nayak, B.K. Panigrahi, V. Ravikumar Pandi, Priyadarshini Dash,Classification of
Power Quality Disturbances Using GA Based Optimal Feature Selection ,LNCS Springer Berlin/Heidelberg, Volume
5909, ISBN: 978-3-642-11163-1, Pages: 561-566)
www.themegallery.com