A brief introduction to evolutionary algorithms EAs

Golden Rule - 1
y Every EA solution should be viewed as based on the

following generic hypothesis

y The optimum value(s) of ----- can be found using a

---- evolutionary approach.

One of the most important parts of an assignment will be your analysis and presentation of your results. EAs are stochastic algorithms, and therefore, your results with the same approach may be different from someone else s. Always, clearly and carefully analyze your results.

Golden Rule - 2
y There must be a reason for every choice made in your

EA

y Parameter value choices y Stopping criteria y Decision to re-run the EA y Decision to mutate and re-run the EA

Golden Rule - 3
y The most important part of an EA is the encoding used y Reduce size of the search space y Eliminate the possibility of illegal solutions y Eliminate local optimums y If you fail to find the best encoding, you solution may be deficcient y Corollary y Most of the work in developing an EA lies in the problem analysis and design, not in the implementation

Parameters a value for a parameter, you y If you have chosen

must have a reason for that choice state it.
y When in doubt as to a good reason or method to choose

a parameter value, try a few different values and use the best as determined empirically as the reason. Here is an example of the wrong way to do it:
y y

I set the normal mutation rate to 0.1. What is normal ? Is there an abnormal value? Why 0.1? Is 0.1 a probability per bit or per chromosome for mutation?

Stopping Criteria
y In many (most) cases, when executing an EA, you will

not know the optimum solution. Thus, the question arises as to why you stopped the EA. Always give your stopping criteria and why you used it. An example of a bad choice is:
y Stopped at 2000 generations and the convergence rate

was 0.99

Say what?

Bottom Line - Summary

y This is a computer SCIENCE class. Many of your home

works will be experiments act like scientists, i.e.,

y Be systematic, i.e. have a plan (hypothesis), follow it,

and document what you did (keep a record for the writeup) y A picture (plot or table) is worth a thousand words (abstraction) y Everything that happens has a reason. If the occurrence is of significance, then why did it happen?

BASICS
y Evolutionary Algorithm (EA) vs Evolutionary Computation (EC)
y EA and EC are somewhat synonymous y EC is simply a computer-based

implementation of an EA

Evolutionary
y What makes an algorithm evolutionary? y Has a population y Generates a new population from the old population generations/biological y The more fit a solution (individual), the more likely it will be used in or influence the generation of the new population
y

Fit implies the presence of a fitness measure

y The search process, while influenced by fitness has a

component of randomness stochastic/mutation

10

REMEMBER!!
y Whenever a stochastic process is used to solve a

problem, if the correct or optimal solution is unknown, you should not report the result of a single run. Always report results as an average of several runs, or the best of several runs.

11

AI & Evolutionary Computation

y Evolutionary Computation y Machine learning optimization and classification paradigms based on evolutionary (biological) mechanisms

12

AI & Evolutionary Computation

yEC is about self-organization
y Simple processes that lead to

complex results, e.g. a genetic algorithm (GA) is simple y The whole is > the sum of its parts y There is no conservation of simplicity Wolfram
13

AI & Evolutionary Computation

y EC field consists of
y Genetic Algorithms y Genetic Programming y Evolutionary Strategies (small

perturbations of best solution to search) y Particle Swarm Optimization y Ant colony optimization y Differential Evolution y ..
14

Applications
y Consider the following two examples of optimization

problems. By consider, I mean think of y Writing a computer program to solve each y The time for the program to run y How you will know when you have the optimum solution? y Optimization of some function of n variables that is not everywhere differentiable with some set of constraints on those n variables y The traveling salesperson problem

15

Application Function Optimization

y Let s say that we have some function

F(x,y,z) = 2x + 3y/x2 - 2z2/(x-y)2 and there are bounds on the ranges of allowed values for x, y, and z An optimization problem might be to find the values of x, y, and z which gives the minimum value of F.

16

How
y do I write the program? y long will it take to execute? y will I know when I have the optimum? y do I determine the relative quality of a solution y If one set of values for F gives a value less than all of

the other values that I have tested, is it the optimum?

17

Application TSP Optimal path

y For a given table of costs find the minimum cost

transit that visits every city once and only once

y NP- complete y Grows as O(n!) y Can be solved using dynamic programming in O(2n n )

and space O(2n n2 ) (better in time but worse in space y For 30 cities there are ~2.7X1032 possibilities y exhaustive search anyone?

18

How
y do I write the program? y long will it take to execute? y will I know when I have the optimum? y do I determine the relative quality of a solution

19

Examples
y The preceding 2 examples illustrate some of the

problems associated with computerizing an algorithm

y Time to solve (TSP) y Fitness (F(x,y,z) and TSP) y What is an optimum? y How do we compare one solution with another? y Illegal solutions

how do we know if a solution is illegal (parameter out of bounds or multiple visits to the same city) and what next? y How do we code it?
y

Are there parts of one coded solution that can be used in another solution? IMPORTANT!
20

Robustness and Reusability

y One of the positives of EAs is their robustness. y An EA used to solve one type of problem, may be useful in solving a very different problem with little change to the algorithm y e.g. Matlab s Optimtool

21

Searching the Search Space

y Optimization problems can be thought of as search

problems.
space.
y

y Each solution occupies a position in the search

The parameters and functions of the solution represent the search space

y The real question is how do we search or travel in

this space? y Thought of in another way, one can ask, given a current location in the space, how do we decide
y y

Are we at the solution ? If we re not there, how do we decide where to go next?

22

Searching the Search Space

Are we at the solution ? If we re not there, how do we decide where to go next?

y There are many approaches to answer these

questions. We will emphasize the EC approaches. y In the search process, we would like a technique such that each time we try a solution, we gain useful information that can be used in the choice of the next solution y This eliminates a completely random approach
23

Example
y For the problem 4+x = 10, what does a value of x = 10 tell us? y The error = ? y The sign of the error tells us which way to go y The magnitude of the error tells us how far to go
24

Optimization
y Optimization can be thought of as occurring in two

(search) spaces concurrently

y Parameter space

Space of legal parameter values y Fitness space y Space of values of fitness or objective function, e.g. for a given set of parameter values, how fit is the function
y
25

SGA
y An SGA (Simple Genetic Algorithm) uses a biological

or evolutionary approach to traverse the search space.

y It was developed in the 70 y Chromosomes are encoded as binary strungs

26

Define Chromosome Encoding And Fitness Schemes Scheme Generate initial population of chromosomes Compute Chromosome Fitness

Yes
Stopping Criteria met

Done

No
Mate Chromosomes Perform Crossover Apply Mutation Perform Replacement
27

Warning!
y In order to get people started, we must fix the

techniques used. This is only a very small snapshot of GAs let alone EC. However, this will give you sufficient background to complete the assignment.

28

The Problem
y In order to have something tangible to refer to, we

will deal with the following optimization problem y The problem is to optimize the following function: (-1<=X<=2) (find the maximum)

F(X ) ! x*
To an accuracy of 0.01

* T * x)  .

29

30

Chromosomes and Encoding

y GAs work from an encoding of the parameter space. y They also work with a population of solutions rather

than a single solution

31

Define the Chromosome Encoding Scheme

y Chromosome: A sequence of genes in which each gene

represents an encoding for a particular parameter value y Gene: (Often) a binary string (of bits) which encodes a particular parameter value y Encoding Scheme (must be reversible):
y It is a translation (formula) such that given a gene (string

of bits), one can determine the represented parameter value.

32

Chromosome Encoding Scheme

y For this problem, since there is only one parameter, y y y y

thus, the chromosome is made up of a single gene. The gene value range is given in the problem statement as -1<=X<=2 Since we want an accuracy of 0.01, the gene must represent (2-(-1))/0.01~300 values. Therefore a gene must be 9 bits, i.e. 29 =512 8 bits won t work because there would only be 256 different values.
33

Chromosome Encoding Scheme

y Defining an encoding scheme also defines a decoding

scheme Value(gene) = -1+(gene value)*(3/511) e.g. y if gene = 000000111 y gene value = 7 ( i.e. decimal equivalent of binary 000000111) y Value(gene)= -1+7*3/511 = -0.9589041 .
34

Generate initial population of chromosomes

y An initial population is simply a randomly

generated set of chromosomes (possible solutions) y Since a chromosome can be viewed as a binary string, generating a random chromosome simply means generating a random string of 1 s and 0 s for each chromosome of the population

35

Generate initial population of chromosomes

y IMPORTANT NOTE y In the flow chart, nothing is said about the size of the

population. There is no algorithm for determining the size of the population, only a few rules of thumb y Generally
y as the search space size increases, the population size

will (should) increase y As the complexity of the fitness calculation function increases the population size will (should) decrease
36

Generate initial population of chromosomes

y For binary chromosomes of fixed length n, the size

of the search space is simply 2n y In this problem, the search space size is relatively small
y Since there are only 9 bits 29 = 512.

y thus we will use a population size of n=10 y The initial population will be generated as 10 random binary strings, each of length 9

37

Compute Chromosome Fitness

y In this simple problem, since we want the X that

gives the maximum F(X), we can just use F(X) for the fitness calculation y Thus, for this problem, we would compute the F(X) (fitness) for each of the 10 randomly generated X s

38

Compute Chromosome Fitness

y If the first chromosome in the population was

000000111 y X=000000111=>7 Value(X)= -1+7*3/511 = -0.9589041

F X ! x sin 10 T x  1 0 ! 3 51

39

Mate Chromosomes
y There are several schemes used to determine which

chromosomes to mate. y Generally some stochastic process is used to determine which two chromosomes to mate y A technique called Roulette Wheel is a fairly common technique
y Its positive is that it is simple to implement y Its negative is that it may not maintain diversity in the

population

40

Maintaining Population Diversity

y With roulette wheel mating, over time, as some

chromosomes get more fit, they will tend to overwhelm the less fit chromosomes and thus be the only ones selected for mating. y In a worst case scenario, you may get to a single or duplicate chromosome for the entire population. When this happens, the only searching that occurs will be through mutation.

41

Roulette Wheel Selection

y Compute the relative fitness of each chromosome as a

fraction of the total fitness. y e.g. if there are 3 chromosomes of fitness C1=10, C2=20, and C3=30. y Relative fitness R1=10/60=0.167, R2=20/60 = 0.333, R3=30/60=0.5
y Note

this is sort of like another probability density function

42

Roulette Wheel Selection

y Using these three relative fitness values, now picture a

roulette wheel with a 1 unit circumference which is divided as follows

y 0 <= C1 <0.167, 0.167<= C2 <0.497, 0.497<= X<1.0
y

(Yes I realize it is not exact. This is because of round off errors)

43

Roulette Wheel Selection

y In this example of 3 chromosomes, to select two mates, we generate 2 random numbers, let s say 0.1 and 0.5 are our random numbers.

y Thus, for this roulette wheel, the mating

chromosomes are chromosome 1 and chromosome 2, because 0.1 falls in the chromosome 1 s region on the roulette wheel and 0.5 is in chromosome 2 s region
y This process is continued until all mating pairs are selected

44

Roulette Wheel Selection

y With a population of 10, we need 5 mating pairs. y Each mating pair will generate 2 offspring y What if both mates are the same?

y Select another mate for one of the

duplicates y OR y Mate these two- mutation may make changes.

45

Perform Crossover
y For each mating pair of chromosomes
y Select a random location (bit location) at

which to perform crossover y Swap the bits of the two chromosomes from the crossover point to the end of the chromosome

46

Apply Mutation
y For every bit of every chromosome generate a random

number from 0 to 1 y If the random value is < mutation rate, then flip the bit; otherwise, leave it unchanged

y Note that for a long chromosome (several bits), even a

small mutation rate will generally change at least one bit

47

Question
y If a chromosome is 40 bits in length, and the mutation

rate is 0.02, what is the probability that at least one bit will change under mutation? y P( at least 1 bit change) = 1 P(no changes) y P(no changes) = 0.9840 = 0.446 y Thus, the probability of at least one bit change is 0.554

48

Perform Replacement
y At this point we have two populations
y Population A is the original population, i.e. the parents y Population B is the offspring population, i.e. the

children y The goal is to come up with a population that

y y y

Is diverse Is an improvement At least keeps the best of population

49

Perform Replacement
y Some possible approaches
y Combine (sort) population A&B and then

keep only the best half y Replace parent(s) only if child is better y Simply replace parents with children, but keep best of population to date as a separate chromosome
50

Stopping Criteria met?

y In order to answer this question, one must establish

the criteria for stopping. y Let s assume that we will stop when the best of the population has remained the same for at least 5 generations

51

Continue
y Now simply continue by going back and computing

the fitness of each chromosome (it s probably already computed, since we did that for mate selection), y With each new generation, continue until the stopping criteria has been met

52

Terminology
y Chromosome is made up of (can be divided

into) genes. One can think of a gene as encoding a trait. The different possible settings for a gene are called alleles. Each gene is located (generally) at a particular locus (position) on the chromosome. The complete collection of genetic material (all chromosomes for a biological entity taken together) is called the genome. Two individuals that have the same genomes are said to be of the same genotype.
53

Terminology
y Search space: Collection of candidate solutions y Similarity or Distance between two

chromosomes
y For binary

it s often the number of changes that must occur in A to make it the same as B(Hamming distance) y For non-binary Euclidean distance or Euclidean distance squared

D!

a b

 a b

 an  bn
54

What s best?
y A GA consists of y Encoding y Initial population:
y y

Population generation technique Population size

y Mate selection algorithm y Crossover technique (single point, tw0o-point, uniform) y Mutation scheme and rate y Replacement scheme y Fitness function y Stopping criteria
55

max or min

What s best? Encoding

yGenerally
y A scheme that does not allow for illegal

solutions
y A scheme that minimizes the search

space size y Binary

56

yIt s all about the encoding

scheme and the fitness function!

57

Search Space Size

y How big is the search space for this problem? y For a 5x5 grid, with a choice of three different colors for

each square in the grid, there would be how many possible colorings? y The goal is to generate a coloring scheme such that no two grids that share a side have the same color.

58

GA Example Coloring a grid with a GA

y For this problem, there are ~8.47X1011 (i.e. 325 ) possible

colorings of which about 1000 are correct. y What s the probability of randomly generating a legal coloring?
y ~103 /1011 = 10-8 => like rolling the same number on a roll

of the die 10 times in a row.

y What encoding scheme should we use? y What fitness function should we use?
59

Other Mate Selection Schemes

y Most of the mate selection schemes try

to: y Reduce stochastic errors, or y Retain best solution(s) to date

(elitist), or y Maintain a diverse population of solutions, or y Speed the mate selection process
60

Mate Selection Reduce Stochastic Errors

y Are stochastic errors inevitable?
y Consider 4 chromosomes with relative fitness of

.25, .3, .4, and .05 y Probabilistically, chromosome 2 (0.3 relative fitness) should be chosen 1.2 times out of the 4 choices. y Obviously, this can t be how can you get .2 selections?

61

Mate Selection Reduce Stochastic Errors

y We can t remove the fractional part, but what about the integer part of the relative fitness?
y Can we insure that for chromosome 2, we

always select it at least 1 time? y If we do that, then what about the fraction part?
62

Remainder stochastic sampling

y This method better mirrors the expected outcomes, i.e.

lessens possible stochastic errors y For each string i for which fi /f >=1

/* f i is the fitness of chromosome i /* f is the average fitness of the population

place the integer portion of this number of chromosomes in the next generation mate set
y Use the fractional parts of each fitness for the

probabilities of a roulette wheel to select the remaining chromosomes y Randomly pair chromosomes
63

Elitist Schemes
y For some schemes, one could fail to choose the best

chromosome in the population

y How could this happen with roulette

wheel selection? y How could this happen with remainder stochastic sampling?
y

Could not happen

64

Elitist Schemes
y Take the best of population to date and copy it into the next generation y Means that over time the population size will grow y Only replace a parent with a child if the child is more fit
y This is hill climbing because average

fitness of population always increases or stays the same

65

Tournament Selection for Diversity

y A popular reproduction scheme to maintain

diversity is called tournament selection It works as follows y Randomly select two (or n) solutions and the best goes into the mating pool. y Selection is systematic, i.e. allow each solution to be selected exactly twice
y Best solution in population will win twice, etc. y Any solution will have 0, 1, or two copies in the pool

66

Tournament Selection
y Once the pool is created

y Randomly select a pair to mate and

continue until all pairs have been selected

y Deb and Goldberg claim this technique is at least as

good as any other method in terms of computational complexity and speed of convergence

67

Crossover
y Mate selection doesn t create any new solutions y Mate selection only creates duplicates y Creation of new solutions (search of solution space) is done by crossover and mutation

68

Crossover
y Single Point Crossover (SPC) with equal length

chromosomes

For a binary chromosome of length n bits, generate a random number x such that 1< x = n For SPC, swap bits x through n of the two chromosomes (bits are numbered 1 through n)

69

2-Point Crossover
y Like single point except select two points in each

chromosome, e.g. bits 10 and 25. Then swap bits 10-25 of the two chromosomes

70

Uniform Crossover
y In uniform, each bit of the children is a random

sample from the parents For k=1 to number of bits in chromosome

For each bit k of the two offspring, generate a random number r if r<0.5 then bit k of parent ONE goes to bit k of child ONE bit k of parent TWO goes to bit k of child TWO else bit k of parent TWO goes to bit k of child ONE bit k of parent ONE goes to bit k of child TWO
71

Uniform Crossover
y Pro
y Better maintains diversity
y

Generally some elitist scheme is used for child replacement of parent

y Con
y Can tend to lose good chromosomes

72

Mutation
y Crossover does most of the searching, even though

mutation does some y Mutation s main job is to maintain or introduce diversity y Sometimes the mutation rate is increased when population diversity gets low Stir things up

73

Mutation
y Mutation is generally performed after crossover,

bit-by-bit, on each of the new chromosomes. Given: chromosome length = n and probability of mutation of pm

For I = 1 to n generate a random number X if X <= pm complement bit I of the chromosome end for

74

Mutation
y A problem with the bit-by-bit mutation method is

that it requires the generation of a random number for every bit of every chromosome y Goldberg(1989) suggested a mutation clock operator to reduce the complexity of the mutation operation
y After the first bit is mutated, the location of the next

bit to mutate is created as a random function, i.e. skip some number of bits

75

Mutation
y A question that sometimes arises is what happens to

mutation if rather than a binary chromosome we have say a ternary chromosome?

y Each chromosome element is a 0, 1, or 2 y We can t simply complement y Can we mutate say a 2 to a 2 now? y What s the difference between randomly

choosing any one of the three possible values versus only one of the two different values?

76

Next Generation
y Once a complete generation has been created

the old population must in some way be merged with the new. y To replace all of old with new is called generational replacement y Remember, this replacement is not done until a new population is created. Don t replace old chromosomes as new ones are generated.The other replacement schemes utilize some form of elitism as we have already discussed
77