Department of Biochemical and Chemical Engineering

Process Dynamics and Operations Group (DYN)

Evolutionary Algorithms
Maren Urselmann
Prof. Dr.-Ing. S. Engell
Process Dynamics and Operations Group
TU Dortmund, Germany

Lecture | 14.07.2011

Process Dynamics
and Operations

Literature and Slides

Literature:
A.E. Eiben, J.E. Smith: Introduction to
Evolutionary Computation, Springer,
2007

Slides of this lecture are developed on

the basis of the proposed slides for an
evolutionary computation course by
A.E. Eiben and J.E. Smith

Lecture | 14.07.2011

Process Dynamics
and Operations

Evolutionary Algorithms
Evolutionary Algorithms (EA) are inspired by the
process of biological evolution
Biology delivered inspiration and terminology
EVOLUTION

PROBLEM SOLVING

Environment

Problem

Individual

Candidate Solution

Fitness

Quality (objective fct value)

Fitness chances for survival and reproduction

Quality chance for seeding new solutions
Lecture | 14.07.2011

Process Dynamics
and Operations

Darwinian Evolution
All environments have limited resources
Lifeforms have the basic instinct to reproduce
Some kind of selection is inevitable
Natural selection = survival of the fittest
Competition for those resources
Individuals that are better adapted to the environment
(fitter individuals) have increased chances to
survive/reproduce
Over time Natural selection causes a rise in the fitness
of the population
Lecture | 14.07.2011

Process Dynamics
and Operations

Natural Genetics
The information required to build a living organism is
coded in the DNA of that organism
Genotype (DNA inside) determines phenotype
Small changes in the genotype lead to small changes in
the organism (e.g., height, hair colour)
Sexual reproduction:
Recombination = Crossover
- Genes of offspring are build by parts of the maternal and by
parts of the paternal genes

Mutation
- Some of the genetic material changes very slightly
(replication error)
- The child might have genetic material information not
inherited from either parent
Lecture | 14.07.2011

Process Dynamics
and Operations

General Scheme of EAs

Lecture | 14.07.2011

Process Dynamics
and Operations

General Scheme of EAs II

(1) INITIALIZE population with random candidate solutions;
(2) EVALUATE each candidate;
(3) Repeat until (TERMINATION CONDITION is satisfied)
(1) SELECT parents;
(2) RECOMBINE pairs of parents;
VARIATION Operators
(3) MUTATE the resulting offspring;
(4) EVALUATE new candidates;
(5) SELECT individuals for the next generation;

Lecture | 14.07.2011

Process Dynamics
and Operations

Representation
Candidate solutions (individuals) exist in phenotype
space
They are encoded in chromosomes, which exist in
genotype space
Encoding : phenotype genotype (not necessarily one to one)
Decoding : genotype phenotype (must be one to one)

Chromosomes contain genes

In order to find the global optimum, every feasible solution must
be represented in genotype space

Lecture | 14.07.2011

Process Dynamics
and Operations

Evaluation (Fitness) Function

Represents the requirements that the population should
adapt to
a.k.a. quality function or objective function
Assigns a single real-valued fitness to each phenotype
which forms the basis for selection
Typically we talk about fitness being maximized
Some problems may be best posed as minimization
problems, but conversion is trivial
Lecture | 14.07.2011

10

Process Dynamics
and Operations

Population
Holds (representations of) possible solutions
Usually has a fixed size and is a multiset of genotypes
Selection operators usually take whole population into
account i.e., reproductive probabilities are relative to
current generation
Diversity of a population refers to the number of
different fitnesses / phenotypes / genotypes present
(note not the same thing)
Lecture | 14.07.2011

11

Process Dynamics
and Operations

Parent selection mechanism

Assigns variable probabilities of individuals acting as
parents depending on their fitnesses
Usually probabilistic
high quality solutions more likely to become parents than
low quality
but not guaranteed
even worst in current population usually has non-zero
probability of becoming a parent

This stochastic nature can aid to escape from local

optima
Lecture | 14.07.2011

12

Process Dynamics
and Operations

Variation Operators
Role is to generate new candidate solutions
Usually divided into two types according to their arity
(number of inputs):
Arity 1 : mutation operators
Arity >1 : Recombination operators
Arity = 2 typically called crossover

There has been much debate about relative importance

of recombination and mutation
Nowadays most EAs use both
Choice of particular variation operators is representation
dependant
Lecture | 14.07.2011

13

Process Dynamics
and Operations

Recombination
Merges information from parents into offspring
Choice of what information to merge is stochastic
Most offspring may be worse, or the same as the
parents
Hope is that some are better by combining elements of
genotypes that lead to good traits

Lecture | 14.07.2011

14

Process Dynamics
and Operations

Mutation
Acts on one genotype and delivers another
Element of randomness is essential and differentiates it
from other unary heuristic operators
Importance ascribed depends on representation and
dialect
May guarantee connectedness of search space and
hence convergence proofs

Lecture | 14.07.2011

15

Process Dynamics
and Operations

Survivor Selection
a.k.a. replacement
Most EAs use fixed population size so need a way of
going from (parents + offspring) to next generation
Often deterministic
Fitness based : e.g., rank parents+offspring and take best
Age based: make as many offspring as parents and
delete all parents

Sometimes do combination (elitism)

Lecture | 14.07.2011

16

Process Dynamics
and Operations

Initialization and Termination

Initialization usually done at random,
Need to ensure even spread and mixture of possible
genes
Can include existing solutions, or use problem-specific
heuristics, to seed the population

Termination condition checked every generation

Reaching some (known/hoped for) fitness

Reaching some maximum allowed number of generations
Reaching some minimum level of diversity
Reaching some specified number of generations without
fitness improvement

Lecture | 14.07.2011

17

Process Dynamics
and Operations

Example: The 8 Queens Problem

Place 8 queens on an 8x8 chessboard in such a way

that they cannot check each other

Lecture | 14.07.2011

18

Process Dynamics
and Operations

The 8 Queens Problem: Representation

Phenotype:
a board configuration

Genotype:
a permutation of
the numbers 1 - 8

Obvious mapping

1 3 5 2 6 4 7 8
Lecture | 14.07.2011

19

Process Dynamics
and Operations

The 8 Queens Problem: Fitness Evaluation

Penalty of one queen:
the number of queens she can check.
Penalty of a configuration:
the sum of the penalties of all queens.
Note: penalty is to be minimized
Fitness of a configuration:
inverse penalty to be maximized
Lecture | 14.07.2011

20

Process Dynamics
and Operations

The 8 Queens Problem: Mutation

Small variation in one permutation, e.g.:

swapping values of two randomly chosen positions
Offspring

Parent individual

1 3 5 2 6 4 7 8

1 3 7 2 6 4 5 8

Lecture | 14.07.2011

21

Process Dynamics
and Operations

The 8 Queens Problem: Recombination

Combining two permutations into two new permutations:
choose random crossover point
copy first parts into children
create second part by inserting values from other parent:
in the order they appear there
beginning after crossover point
skipping values already in child

1 3 5 2 6 4 7 8
8 7 6 5 4 3 2 1

Lecture | 14.07.2011

1 3 5 4 2 8 7 6
8 7 6 2 4 1 3 5

22

Process Dynamics
and Operations

The 8 Queens Problem: Selection

Parent selection:
Pick 5 parents and take best two to undergo crossover

Survivor selection (replacement)

When inserting a new child into the population, choose an
existing member to replace by:
Sorting the whole population by decreasing fitness
Enumerating this list from high to low
Replacing the individual with the lowest fitness by the
child

Lecture | 14.07.2011

23

Process Dynamics
and Operations

The 8 Queens Problem: Summary

20

Note that this is only one possible set of choices of

operators and parameters
Lecture | 14.07.2011

24

Process Dynamics
and Operations

The 8 Queens Problem: Results

Lecture | 14.07.2011

25

Process Dynamics
and Operations

Typical Behavior of an EA
Phases in optimizing on a 1-dimensional fitness landscape
Early phase:
quasi-random population distribution
Mid-phase:
population arranged around/on hills
Late phase:
population concentrated on high hills
Lecture | 14.07.2011

26

Process Dynamics
and Operations

Best fitness in population

Typical run: progression of fitness

Time (number of generations)

Typical run of an EA shows so-called anytime behavior

Lecture | 14.07.2011

27

Process Dynamics
and Operations

Evolutionary Algorithms
EAs fall in the category of generate & test algorithms
They are stochastic, population-based algorithms
Variation operators (recombination and mutation) create
the necessary diversity and thereby facilitate novelty
Selection reduces diversity and acts as a force pushing
quality

Lecture | 14.07.2011

28

Process Dynamics
and Operations

Different Types of EAs

Historically different flavours of EAs have been
associated with different representations

Binary strings : Genetic Algorithms

Real-valued vectors : Evolution Strategies
Finite state Machines: Evolutionary Programming
LISP trees: Genetic Programming

These differences are largely irrelevant, best strategy

choose representation to suit problem
choose variation operators to suit representation

Selection operators only use fitness and so are

independent of representation
Lecture | 14.07.2011

29

Process Dynamics
and Operations

Evolution Strategies

Lecture | 14.07.2011

30

Process Dynamics
and Operations

Evolution Strategies

Developed: Germany in the 1970s

Early names: I. Rechenberg, H.-P. Schwefel
Today: ES belong to the class of EAs
Typically applied to:
numerical optimization

Attributed features:
fast
good optimizer for real-valued optimization
relatively much theory

Special:
self-adaptation of (mutation) parameters
Lecture | 14.07.2011

31

Process Dynamics
and Operations

Introductory Example
Task: minimimize f : Rn R
Algorithm: ES with two members using

Vectors from R directly as chromosomes

Population size =1
No recombination
Always Mutation
One child per generation (=1)
Greedy selection ( (+) selection)
No self-adaptation of the step size

Lecture | 14.07.2011

32

Process Dynamics
and Operations

Introductory example: scheme

(1) INITIALIZE population with a random candidate
solution: t:=0; xt=<x1t, x2t, , xnt>
(2) EVALUATE the candidate f(xt);
(3) Repeat until (TERMINATION CONDITION is satisfied)
(1) zi = *N(0,1) , for i=1,,n;
(2) MUTATION: yit = xit + zi;
(3) EVALUATE the offspring f(yt);
(4) SELECT the individual with better fitness for the next
generation
(1) IF f(xt) < f(yt) THEN
(2) ELSE

xt+1 = xt;
xt+1 = yt;

(5) t=t+1;
Lecture | 14.07.2011

33

Process Dynamics
and Operations

Introductory example: mutation mechanism

z values drawn from normal distribution N(,)
Expected value is set to 0
Standard deviation is called mutation strength

The one
dimensional case
Illustration of the
normal distribution *N(0,1)

Lecture | 14.07.2011

34

Process Dynamics
and Operations

Self-adaptation
Pecularity of the ES is the self-adaptation
Representation of the individuals is extended by one or
more strategy parameters:
E.g.

x = <x1,,xn, >

The strategy parameters are used to control certain

statistical properties of the genetic operators (especially
the mutation)
They can evolve during the evolution process
Inclusion of the strategy parameters in the individuals
genome selection and inheritance together with the
individual

Lecture | 14.07.2011

35

Process Dynamics
and Operations

Self-adapted mutation
Idea:
is part of the chromosome x1,,xn,
is also mutated into

Order is important:
first
then x x = x + N(0,1)

Rationale: new x , is evaluated twice

Primary: x is good if f(x) is good
Secondary: is good if the x it created is good

Lecture | 14.07.2011

36

Process Dynamics
and Operations

Evolution strategies: properties

Prerequisits for self-adaptation:
> 1 to carry different strategies
> to generate offspring surplus, e.g., 7
Survivor selection with high selection pressure

Parent selection:
With uniform random distribution
Each individual has the same probability to be selected

Lecture | 14.07.2011

37

Process Dynamics
and Operations

Recombination
Creates one child
Acts per variable / position by either
Averaging parental values,
Intermediate recombination

Selecting one of the parental values

Discrete recombination

From two or more parents by either:

local
Using two selected parents to make a child
Selecting two parents for each position anew global

Lecture | 14.07.2011

38

Process Dynamics
and Operations

Survivor selection
Basis of selection is either:
The set of children only: (,)-selection
The set of parents and children: (+)-selection
The set of parents with an age < generations and
children: (,,)-selection

(+)-selection is an elitist strategy

(,)-selection can forget
Often (,)-selection is preferred for:
Better in leaving local optima
Using the + strategy bad values can survive in x, too
long if their host x is very fit

Selective pressure in ES is very high ( 7 is the

common setting)
Lecture | 14.07.2011

39

Process Dynamics
and Operations

Uncorrelated mutation with one

Chromosomes: x1,,xn,
Mutation:
= exp( N(0,1))
xi = xi + N(0,1)

Mutation of strategy parameter

Mutation of object parameters

Typically the learning rate 1/ n

boundary rule: < =
minimal mutation strength =

Lecture | 14.07.2011

40

Process Dynamics
and Operations

Mutants with equal likelihood

Parent individual

defines the radius

Circle: mutants having the same chance to be created

Lecture | 14.07.2011

41

Process Dynamics
and Operations

Uncorrelated mutation with n step sizes

Chromosomes: x1,,xn, 1,, n
Mutation:
i = i exp( N(0,1) + Ni (0,1))
xi = xi + i Ni (0,1)

Two learning rate parmeters:

overall learning rate
coordinate wise learning rate
1

2n

and

1
2 n

And i < i =

Lecture | 14.07.2011

42

Process Dynamics
and Operations

Mutants with equal likelihood

Parent individual

i define the axes

Ellipse: mutants having the same chance to be created

Lecture | 14.07.2011

43

Process Dynamics
and Operations

Correlated mutation (CMA-ES)

Parent individual

Rotation angles are needed

Rotated ellipse: mutants having the same chance to be created

Lecture | 14.07.2011

44

Process Dynamics
and Operations

Covariance Matrix Adaptation - ES

Chromosomes: x1,,xn, 1,, n ,1,, k
where k = n (n-1)/2
I.e. for each pair of coordinates (i,j), i,j {1,,n}, ij one
rotation angle exists

and the covariance matrix C is defined as:

cii = i2
cij = 0 if i and j are not correlated
cij = ( i2 - j2 ) tan(2 ij) if i and j are correlated

Lecture | 14.07.2011

45

Process Dynamics
and Operations

Correlated mutations
The mutation mechanism is then:
i = i exp( N(0,1) + Ni (0,1))
j = j + N (0,1)
x = x + N(0,C)
x stands for the vector x1,,xn
C is the covariance matrix C after mutation of the
values

1
2n

and

and 5

2 n

i < i = and
| j | > j = j - 2 sign(j)

Lecture | 14.07.2011

46

Process Dynamics
and Operations

Mutants with equal likelihood

Parent individual

i define the axes

k define the rotation angles

Rotated ellipse: mutants having the same chance to be created

Lecture | 14.07.2011

47

Process Dynamics
and Operations

ES technical summary tableau

Representation

Real-valued vectors

Recombination

Discrete or intermediary

Mutation

Gaussian perturbation

Parent selection

Uniform random

Survivor selection

(,) or (+) or (,,)

Specialty

Self-adaptation of mutation step

sizes

Lecture | 14.07.2011

48

Process Dynamics
and Operations

Performance of methods on problems

EAs as problem solvers:

Goldbergs 1989 view

Special, problem tailored method

Evolutionary algorithm

Random search

Scale of all problems

Lecture | 14.07.2011

49

Process Dynamics
and Operations

Michalewicz 1996 view

Performance of methods on problems

EA 4

EA 2

EA 3

EA 1

P
Scale of all problems
Lecture | 14.07.2011

50

Process Dynamics
and Operations

EC and Global Optimization

Global Optimization: search for finding best solution x*
out of some fixed set S
Deterministic approaches
e.g. box decomposition (branch and bound etc)
Guarantee to find x* , but may run in super-polynomial
time

Heuristic Approaches (generate and test)

rules for deciding which x S to generate next
no guarantees that best solutions found are globally
optimal

Lecture | 14.07.2011

51

Process Dynamics
and Operations