METAHEURISTICS

Introduction
Some problems are so complicated that are not
possible to solve for an optimal solution.
In these problems, it is still important to find a good
feasible solution close to the optimal.
A heuristic method is a procedure to find a very good
feasible solution of a considered problem.
Procedure should be efficient to deal with very large
problems, and is an iterative algorithm.
Heuristic methods usually fit a specific problem rather
than a variety of problems.

409

Introduction
For a new problem, OR team would need to start from
scratch to develop a heuristic method.
This changed with the development of metaheuristics.
A metaheuristic is a general solution method that
provides:
a general structure;
guidelines for developing a heuristic method for a
particular type of problem.

410

Nature of metaheuristics
Example: maximize
f (x ) 12 x 5 975 x 4

subject to 0

28000 x 3 345000 x 2 1800000 x

31

Function has three local optima.

The example is a nonconvex programming problem.
f(x) is sufficiently complicated to solve analitically.
Simple heuristic method: conduct a local
improvement procedure.

411

Example: objective function

412

Local improvement procedure

Starts with initial trial and uses a hill-climbing
procedure.
Example: gradient search procedure, bisection method,
etc.

Converges to a local optimum. Stops without reaching

global optimum (depends on initialization).
Typical sequence: see figure.
Drawback: procedure converges to local optimum. This
is only a global optimum if search begins in the
neighborhood of this global optimum.

413

Local improvement procedure

414

Nature of metaheuristics
How to overcome this drawback?
What happens in large problems with many variables?
Metaheuristic: solution method that orchestrates the
interaction between local improvement procedures
and a process to escape from local optima in a robust
way.
A trial solution after a local optimum can be inferior to
this local optimum.

415

Solutions by metaheuristics

416

Metaheuristics
Advantage: deals well with large complicated
problems.
Disadvantage: no guarantee to find optimal solution
or even a nearly optimal solution.
When possible, an algorithm that can guarantee
optimality should be used instead.
Can be applied to nonlinear or integer programming.
Most commonly is applied to combinatorial
optimization.

417

Most common metaheuristics

Tabu search
Simulated Annealing (SA)
Genetic Algorithms (GA)
Ant Colony Optimization (ACO)
Particle Swarm Optimization (PSO), etc.

418

Common characteristics
Derivative freeness: methods rely on evaluations of
objective function; search direction follows heuristic
guidelines.
Intuitive guidelines: concepts are usually bio-inspired.
Slowness: slower than derivative-based optimization
for continuous optimization problems.
Flexibility: allows any objective function (even
structure of data-fitting model).

419

Common characteristics
Randomness: stochastic methods use random
numbers to determine search directions; may be
global optimizers given enough computation time
(optimistic view).
Analytic opacity: knowledge based on empirical
studies due to randomness and problem-specific
nature.
Iterative nature: need of stopping criteria to
determine when to terminate the optimization
process.

420

Traveling Salesman Problem

421

Traveling Salesman Problem

Can be symmetric or asymmetric.
Objective: route that minimizes the distance (cost,
time).
Applications: truck delivering goods, drilling holes
when manufacturing printed circuit boards, etc, etc.
Problem with n cities has (n 1)!/2 feasible routes.
10 cities: 181 440 feasible solutions
20 cities: 6 1016 feasible solutions
50 cities: 3 1062 feasible solutions

422

Traveling Salesman Problem

Some TSP problems can be solved using branch-andcut algorithms.
Heuristic methods are more general. A new solution is
obtained by making small adjustments to the current
solution.
Example:
Sub-tour reversal adjusts sequence of visited cities by
reversing order in which subsequence is visited.

423

Solving example
Sub-tour reversal 3-4 of
1-2-3-4-5-6-7-1
1-2-4-3-5-6-7-1

424

Sub-tour reversal algorithm

Initialization: start with any feasible solution.
Iteration: from current solution consider all possible
ways of performing a sub-tour reversal. Select the one
that provides the largest decrease in traveled distance.
Stopping rule: stop when no sub-tour reversal improve
current trial solution.
Local improvement algorithm: does not assure
optimal solution!

425

Example
Iteration 1: starting with 1-2-3-4-5-6-7-1 (Distance =
69), 4 possible sub-tour reversals that improve
solution are:
Reverse 2-3:
Reverse 3-4:
Reverse 4-5:
Reverse 5-6:

1-3-2-4-5-6-7-1
1-2-4-3-5-6-7-1
1-2-3-5-4-6-7-1
1-2-3-4-6-5-7-1

Distance = 68
Distance = 65
Distance = 65
Distance = 66

426

Example
Iteration 2: continuing with 1-2-4-3-5-6-7-1 only 1 sub-tour
reversal leads to improvement; reverse 3-5-6: 1-2-4-6-53-7-1 with Distance = 64.
Algorithm stops. Last solution is final solution.
This solution is not the optimal!

427

10

Tabu Search
Fred Glover, 1977
Includes a local search procedure, allowing nonimprovement moves to the best solution.
Referred to as steepest ascent/mildest descent
approach.
To avoid cycle in local optimum, a tabu list is added.
Tabu list records forbidden moves, knows as tabu
moves.
Thus, it uses memory to guide the search.
Can include intensification or diversification.
428

Basic tabu search algorithm

Initialization: start with a feasible initial solution.
Iteration:
1. Use local search to define feasible moves in
neighborhood.
2. Eliminates moves in tabu list, unless it results in a
better solution.
3. Determine which move provides best solution.
4. Adopt this solution as next trial solution.
5. Update tabu list.

Stopping rule: stop using fixed number of iterations,

fixed number of iterations without improvement, etc.
429

11

Questions in tabu search

1. Which local search procedure should be used?
2. How to define the neighborhood structure?
3. How to represent tabu moves in the tabu list?
4. Which tabu list should be added to the tabu list in

each iteration?
5. How long should a tabu move remain in the tabu list?
6. Which stopping rule should be used?

430

Ex: minimum spanning tree problem

Problem without constraints: solved using greedy
algorithm.

431

12

Added constraints
Constraint 1: link AD can only be included together
with link DE.
Constraint 2: at most one of the three links AD, CD
and AB can be included.
Previous solution violates both constraints.

Applying tabu search:

Charge a penalty of 100 if Constraint 1 is violated.
Charge a penalty of 100 if two of the three links in
Constraint 2 are included. Increase penalty to 200 if all
three links are included.

432

Tabu search implementation

1. Local search procedure: choose best immediate

neighbor not ruled out by tabu status.

2. Neighborhood structure: immediate neighbor is the
one reached by adding a link and deleting one of the
other links in the cycle.
3. Form of tabu moves: list links not to be deleted.
4. Addition of a tabu move: add chosen link to tabu list.
5. Maximum size of tabu list: two (half of total links).
6. Stopping rule: three iterations without improvement.

433

13

Solving example
Initial solution: solution of unconstrained version
Cost = 20 + 10 + 5 + 15 + 200 (why?) = 250

Iteration 1: Options to add a link are BE, CD and DE.

Add

Delete

BE

CE

Cost
75 + 200 = 275

BE

AC

70 + 200 = 270

BE

AB

60 + 100 = 160

CD

AD

60 + 100 = 160

CD

AC

65 + 300 = 365

DE

CE

85 + 100 = 185

DE

AC

80 + 100 = 180

DE

AD

75 + 0

= 75

Minimum
434

Application of tabu search

Add DE to network.
Delete AD from network.
Add DE to tabu list

435

14

Iteration 2
Options to add a link are AD, BE and CD.
Add

Delete

BE

CE

100 + 0 = 100

Cost

BE

AC

95 + 0 = 95

BE

AB

85 + 0 = 85

CD

DE*

60 + 100 = 160

CD

CE

95 + 100 = 195

AD

DE*

(Tabu move)

AD

CE

85 + 100 = 185

AD

AC

80 + 100 = 180

Minimum

*A

tabu move; only considered if result in better solution

than best trial solution found previously.

436

Iteration 3
Options to add a link are AB, AD and CD.
Add

Delete

AB

BE*

Cost
(Tabu move)

AB

CE

100 + 0 = 100

AB

AC

95 + 0 = 95

AD

DE*

60 + 100 = 160

AD

CE

95 + 0

= 95

AD

AC

90 + 0

= 90

CD

DE*

70 + 0

= 70

CD

CE

105 + 0

= 105

Minimum

*A

tabu move; only considered if result in better solution than

best trial solution found previously.

437

15

Iteration 3 and optimal solution

438

Traveling salesman problem example

1. Local search procedure: choose best immediate
2.
3.
4.
5.
6.

neighbor not ruled out by tabu status.

Neighborhood structure: immediate neighbor is the
one reached by making a sub-tour reversal (add and
remove two links of current solution).
Form of tabu moves: list links such that a sub-tour
reversal would be tabu if both links are in the list.
Addition of a tabu move: add two chosen links to
tabu list.
Maximum size of tabu list: four links.
Stopping rule: three iterations without improvement.
439

16

Solving problem
Initial trial solution: 1-2-3-4-5-6-7-1 Distance = 69
Iteration 1: choose to reverse 3-4.
Deleted links: 2-3 and 4-5
Added links (tabu list): 2-4 and 3-5
New trial solution: 1-2-4-3-5-6-7-1 Distance = 65

Iteration 2: choose to reverse 3-5-6.

Deleted links: 4-3 and 6-7 (OK since not in tabu list)
Added links: 4-6 and 3-7
Tabu list: 2-4, 3-5, 4-6 and 3-7
New trial solution: 1-2-4-6-5-3-7-1 Distance = 64
440

Solving problem
Only two immediate neighbors:
Reverse 6-5-3: 1-2-4-3-5-6-7-1 Distance = 65. (This
would delete links 4-6 and 3-7 that are in the tabu list.)
Reverse 3-7: 1-2-4-6-5-7-3-1 Distance = 66

Iteration 3: choose to reverse 3-7.

Deleted links: 5-3 and 7-1
Added links: 5-7 and 3-1
Tabu list: 4-6, 3-7, 5-7 and 3-1
New trial solution: 1-2-4-6-5-7-3-1 Distance = 66

441

17

Sub-tour reversal of 3-7 (Iteration 3)

442

Iteration 4
Four immediate neighbors:
Reverse 2-4-6-5-7: 1-7-5-6-4-2-3-1 Distance = 65
Reverse 6-5: 1-2-4-5-6-7-3-1 Distance = 69
Reverse 5-7: 1-2-4-6-7-5-3-1 Distance = 63
Reverse 7-3: 1-2-4-5-6-3-7-1 Distance = 64

Iteration 4: choose to reverse 5-7.

Deleted links: 6-5 and 7-3
Added links: 6-7 and 5-3
Tabu list: 5-7, 3-1, 6-7 and 5-3
New trial (final) solution: 1-2-4-6-7-5-3-1 Distance = 63
443

18

Sub-tour reversal of 5-7 (Iteration 4)

444

Simulated Annealing
Kirkpatrick, Gelatt, Vecchi, 1983
Suitable for continuous and discrete optimization
problems.
Effective in finding near optimal solutions for largescale combinatorial problems such as TSP.
Enables search process to escape from local minima.
Instead of steepest ascent/mildest descent approach
as in tabu search, it tries to search for the tallest hill.
Early iterations take steps in random directions.

445

19

Simulated Annealing
Principle analogous to metals behavior when cooled at
a controlled rate.
Value of objective function analogous to energy in
thermodynamic systems:
At high temperatures, it is likely to accept a new point
with higher energy.
At low temperatures, likelihood of accepting a new
point with higher energy is much lower.

Annealing or cooling schedule: specifies how rapidly

the temperature is lowered from high to low values.

446

Simulated Annealing
Let
Zc = objective function value for current trial solution.
Zn = objective function value for candidate of next trial
solution.
T = tendency to accept Zn if not better than current
solution.

447

20

Simulated Annealing
Move selection rule: among all immediate neighbors
of current solution, select one randomly. To maximize,
accept or reject candidate as follows:
If Zn
If Zn

Zc, always accept candidate,

Zc, accept candidate with following probability:

Prob{acceptance} e x , where x

Zn

Zc

T
(For minimization problems reverse Zn and Zc.)
If candidate is rejected, repeat for another immediate
neighbor.

448

Probability of accepting solutions

x=

Zn - Zc
T

Prob{acceptance} = e x

0.01

0.990

-0.25

0.779

-1

0.368

-3

0.050

-5

0.007

Larger T: probability of accepting is higher.

Simulated annealing starts with large T and gradually decreases
it as the iterations proceed.
A temperature scheduling should be chosen.
Implementation of move selection rule: compare random
number between 0 and 1 to the probability of acceptance.
449

21

Simulated annealing algorithm

Initialization: start with a feasible initial trial solution.
Iteration: Use the move selection rule to select next
trial.
If none of immediate neighbors of current solution are
accepted, the algorithm is terminated.

Check the temperature scheduling: decrease T if a

certain number of iterations have been performed.
Stopping rule: stop after a predetermined number of
iterations (or if there are no accepted solutions). Best
trial solution at any iteration is the final solution.

450

Questions in simulated annealing

1. How should the initial solution be selected?
2. What is the neighborhood structure that specifies

which solutions are immediate neighbors?

3. What device should be used in the move selection
rule to randomly select one of the immediate
neighbors?
4. What is an appropriate temperature scheduling?

451

22

Traveling salesman problem example

1. Initial trial solution: any feasible solution. Can be

chosen randomly or it can be 1-2-3-4-5-6-7-1.

2. Neighborhood structure: is one reached by a sub-tour
reversal, as described previously.
3. Random selection of an immediate neighbor: has only
some restrictions.
4. Temperature schedule: five iterations for each of the
five values T1 = 0.2Zc (Zc is objective function for initial
solution), Ti+1 = 0.5Ti, i = 1, 2, 3, 4.

452

Solving problem: iteration 1

Initial trial solution: 1-2-3-4-5-6-7-1, Zc= 69, T1= 13.8.
Sub-tour to reverse can begin between the second slot
(city 2) and the sixth slot (city 6).
Can have equal probabilities to start in any of the five
slots.

After choosing beginning slot, choose end slot with

equal probability.
Suppose that 3-4 was chosen to reverse, Zn= 65.
Solutions are not always feasible, if algorithm chooses
to reverse 2-3-4-5, solution 1-5-4-3-2-6-7-1 is not
feasible.

As Zn= 65 < Zc= 69, this is the next trial solution.

453

23

Solving problem
Suppose that Iteration 2 results in reversing 3-5-6, to
obtain 1-2-4-6-5-3-7-1, with Zn= 64.
Suppose now that Iteration 3 results in reversing 3-7,
to obtain 1-2-4-6-5-7-3-1, with Zn= 66. As Zn > Zc:
Prob{acceptance} e

Zn Zc
T

e 2/13.8
0.865

One application of SA gave best solution 1-3-5-7-6-4-2-1

at iterations 13 and 15 (out of 25) with distance = 63.

454

Nonlinear programming application

Problems of the type:

Maximize

f ( x1 ,

, xn )

subject to
Lj

xj

U j , for j 1,

,n

1. Initial trial solution: any feasible solution. Can be

xj = Lj + (Uj Lj)/2.
2. Neighborhood structure: any feasible solution, see
random selection of immediate neighbor.

455

24

Nonlinear programming application

3. Random selection of an immediate neighbor: Set

Uj

Lj

reset x j

xj

N (0,

), for j 1,

,n

where N(0, j) is a normal distribution.

4. Temperature schedule: five iterations for each of the
five values T1 = 0.2Zc, Ti+1 = 0.5Ti, i = 1, 2, 3, 4.

456

Example
Initial solution: x = 15.5, Zc = f(15.5) = 3 741 121, and
T1 = 748 224. = (31 - 0)/6 = 5.167.
Iteration 1: x = 15.5 + N(0, 5.167) = 15.5 7.5 = 8.
Zn = f(8) = 3 055 616. As
Zn

Zc
T

3055616 3714121
748224

Prob{acceptance} e

0.916

0.916

0.400

After 25 iterations, gives optimal value of x = 20.031

(optimal is x = 20).

457

25

GENETIC
ALGORITHMS

Genetic Algorithms
Motivation
What evolution brings us?
Vision
Hearing
Smelling
Taste
Touch
Learning and reasoning

Can we emulate the evolutionary process with todays

fast computers?
459

26

Genetic Algorithms
Introduced by John Holland in 1975.
Randomized search algorithms based on mechanics of
natural selection and genetics.
Principle of natural selection through survival of the
fittest with randomized search.
Search efficiently in large spaces.
Robust with respect to the complexity of the search
problem.
Use a population of solutions instead of searching only
one solution at a time: easily parallelized algorithms.
460

Basic elements
Candidate solution is encoded as a string of characters
in binary or real. Bit string is called a chromosome.
Solution represented by a chromosome is the
individual. A number of individuals form a population.
Population is updated iteratively; each iteration is
called a generation.
Objective function is called the fitness function.
Fitness value is maximized.
Multiple solutions are evaluated in parallel.

461

27

Definitions
Population: a collection of solutions for the studied
(optimization) problem.
Individual: a single solution in a GA.
Chromosome: (bit string) representation of a single
solution.
Gene: part of a chromosome, usually representing a
variable characterizing part of the solution.

462

Definitions
Encoding: conversion of a solution to its equivalent bit
string representation (chromosome).
Decoding: conversion of a chromosome to its
equivalent solution.
Fitness: scalar value denoting the suitability of a
solution.

463

28

GA terminology
Generation t

population

y
individual

solution

fitness

(2,0)

(1,1)

(0,3)

(1,2)

(1,1)

gene
chromosome
464

Basic genetic algorithm

Initialization: Start initial population of solutions, e.g.
randomly. Evaluate the fitness for each individual.
Iteration:
1. Select some members of population to become parents.
2. Cross genetic material of parents in a crossover

operation. Mutation can occur in some genes.

3. Take care of infeasible solutions, by making them
feasible.
4. Evaluate fitness of new members, including the clones.

Stopping rule: stop using fixed number of iterations,

fixed number of iterations without improvement, etc.
465

29

Genetic algorithm
Define Initial Population

Parents

Fitness Function

Increment Generation
Assess Fitness
Children

Best Individuals
Selection

Mutation
Crossover
Genetic Algorithm
466

Termination criteria
Number of generations
(restart GA if best solution is not satisfactory).
Fitness of best individual.
Average fitness of population.
Difference of best fitness (across generations).
Difference of average fitness (across generations).

467

30

Reproduction
Three steps:
Selection
Crossover
Mutation
In GAs, the population size is often kept constant.
User is free to choose which methods to use for all
three steps.

468

Roulette-wheel selection
individuals fitness
p = 0.16
p = 0.23
p = 0.11
p = 0.07
p = 0.19
p = 0.24

01100
10001
11010
00111
11000
10110

34
48
23
15
41
50

selection

01100
10001
10001
11000
10110
10110

Sum = 211

Cumulative probability: 0.16, 0.39, 0.50, 0.57, 0.76, 1.00

469

31

Roulette-wheel selection

1
16%

1
2
23%

6
24%

2
3
4
5

5
19%

4
7%

3
11%

470

Tournament selection
Select pairs randomly
Fitter individual wins
deterministic
probabilistic
constant probability of winning
probability of winning depends on fitness

It is also possible to combine tournament selection

with roulette-wheel.

471

32

Crossover
Exchange parts of chromosome with a crossover
probability (pc is about 0.8)
Select crossover points randomly
One-point crossover:
0

crossover point
0

1
472

N-point crossover
Select N points for exchanging parts
Exchange multiple parts
Two-point crossover:
crossover points
0

0
473

33

Uniform crossover
Exchange bits using a randomly generated mask
Uniform crossover:
0

mask

474

Mutation
Crossover is used to search the solution space
Mutation is needed to escape from local optima
Introduces genetic diversity
Mutation is rare (pm is about 0.01)
Uniform mutation:
0

mutated bit
475

34

GA iteration
10010110
01100010
10100100
10011001
01111101
...
...
...

Elitism

Selection

Crossover

Mutation

10010110
01100010
10100100
10011101
01111001
...
...
...

reproduction
Current
generation

Next
generation

476

Spaces in GA iteration
fitness operators
gene-space
generation N

01100
10001
11010
00111
11000
10110

(de)coding

problem-space

fitness-space

12
17
26
7
24
22

34
48
23
15
41
50

fitness function

geneticoperators

generation N+1

01011
10111
11001
00011
11010
01010
477

35

Encoding and decoding

Chromosomes represent solutions for a problem in a
selected encoding.
Solutions must be encoded into their chromosome
representation and chromosomes must be decoded to
evaluate their fitness.
Success of a GA can depend on the coding used.
May change the nature of the problem.
Common coding methods:
simple binary coding
gray coding (binary)
real valued coding (requires special genetic operators)
478

Handling constraints
Explicit: in the fitness function
penalty function
barrier function
setting fitness of unfeasible solutions to zero
(search may be very inefficient due to unfeasible
solutions)

Implicit: (preferred method) with special encoding

GA searches always for feasible solutions
smaller search space
adhoc method, may be difficult to find

479

36

Questions in genetic algorithms

1. What is the encoding scheme?
2. What should the population size be?
3. How should the individuals of the current population

be selected to become parents?

4. How should the genes of the children be derived
from the genes of the parents?
5. How should mutations occur in the genes of the
children?
6. Which stopping rule should be used?

480

Example: peaks function

Maximization of the peaks function using GA
z

f ( x, y )
3(1 x )2 e

x 2 ( y 1)2

10

x 3
x
5

y5 e

x2 y 2

1
e
3

( x 1)2 y 2

481

37

Example: peaks function

Derivatives of the peaks function:

482

Example: settings
Search domain: [-3,3] [-3,3]
8-bit binary coding
Search space size = 28 28 = 65536
Each generation with 20 individuals
Fitness value = value of peaks function minimum
function value across population
One-point crossover scheme: 1.0 crossover rate
Uniform mutation: 0.01 mutation rate
Elitism of the best 2 individuals across generations
30 generations
483

38

Example: peaks function

GA process:

Initial population

5th generation

10th generation

484

Example: performance profile

10

Fitness

Best
Average
Poorest
-5
0

10

15

Generations

20

25

30

485

39

Nonlinear programming example

1. Encoding scheme: integers from 0 to 31. Five binary

genes are needed.

Example: x = 21 is 10101 in base 2.

Note that no infeasible solutions are possible.

2. Population size: 10 (problem is simple).

3. Selection of parents: select randomly 4 from the 5

most fit, and 4 from the 5 least fitted.

Select 3 pairs of parents, that will produce 6 children.

Elitism: four best solutions are clones in next
generation.

486

Nonlinear programming example

4. Crossover operator: When a bit is difference in the

two parents, select 0 or 1 randomly (uniform dist.)

Parents are 0x01x. Children can be 01011 and 00010.

5. Mutation operator: mutation rate is 0.1 for each

gene.
6. Stopping criteria: stop after 5 consecutive
generations without improvement.
Fitness value is just the objective function value in
this example.
6 generations were enough in this example.

487

40

Traveling salesman problem example

1. Encoding scheme: exactly as before.
Example: 1-2-3-4-5-6-7-1.
Initial population: generated randomly, using possible
links between cities.
2. Population size: 10 (problem is simple).
3. Selection of parents: select randomly 4 from the 5

most fit, and 4 from the 5 least fitted.

Select 3 pairs of parents, that will produce 6 children.

Elitism: four best solutions are clones in next
generation.

488

Traveling salesman problem example

4. and 5. operators: follows an algorithm:
1. Options for next link: links from current city not in childrens

tour that are used by parents.

2. Selection of next link: randomly with a uniform distribution.
3. Mutation: if a mutation occurs replace last link with any
other possible one, unless it is impossible.
4. Repetition: when there is still more than one link to include
in child, go to step 1.
5. Completion: add last city and complete tour in links exist. If
not, solution is infeasible and all process should be
repeated.

6. Stopping criteria: 5 iterations without improvement.

489

41

Traveling salesman problem

Example: P1: 1-2-3-4-5-6-7-1
P2: 1-2-4-6-5-7-3-1
Child: From P1 can be 1-2 or 1-7. From P2 can be 1-2 or
1-3. 1-2 has 50% probability to be chosen. Why?
Suppose 1-2 was chosen. At next step links 2-3 (P1) and
2-4 (P2) can be chosen. Suppose it is 2-4.
Child is 1-2-4. And so on!

Note that problem is too simple, and solutions

(individuals) can be repeated in the population.
See this example in Hilliers book.

490

ANT COLONY
OPTIMIZATION

42

Ant Colony Optimization

Marco Dorigo, 1992
Ant Colony Optimiziation is the most used method of
the Artificial Life algorithms (Wasp, Bees, Swarm).
Applications: Traveling Salesman Problem, Vehicle
Routing, Quadratic Assignment Problem, Internet
Routing and Logistic Scheduling.
There are also some applications of ACO in clustering
and data mining problems.

492

Ant Colony Optimization

Ants can perform complex tasks:
nest building, food storage.
garbage collection, war.
foraging (to wander in search of food).

There is no management in an ant colony

collective intelligence.

They communicate using:

pheromones (chemical substances), sound, touch.

Curiosities:
Ant colonies exist for more than 100 million years.
Myrmecologists estimate that there are around 20,000 species of ants.

493

43

The foraging behaviour of ants

How can almost blind animals manage to learn the shortest
route paths from their nests to the food source and back?

a) - Ants follow path between the

Nest and the Food Source

b) - Ants go around the obstacle following one

of two different paths with equal probability

c) - On the shorter path, more

pheromones are laid down

d) At the end, all ants follow the shortest

path.

Fotos: http://iridia.ulb.ac.be/~mdorigo/ACO/RealAnts.html
494

Artificial ants
Artificial ants move in graphs
nodes / arcs
environment is discrete

Food Source
Destination

As the real ants, they:

choose paths based on pheromone
deposit pheromones on the travelled paths
The environment updates the pheromones

Artificial ants have more abilities:

Nest
Source

they can see (heuristic ) [former visibility]

they have memory (feasible neighbourhood N) [former tabu list ]
495

44

Mathematical framework of real ants

j =3

Choose trail

13

i =1

pijk

Ant 1, t=0

f ( ij )

12
j =2

13

i =1

Deposit pheromone

Ant 1, t=1

12
j =2

j =3

Environment (time)
updates pheromones

13

i =1

Ant 2, t=2
12

ij

(t 1)

ij

(t ) (1

ij

j =2

496

Mathematical framework
Choose node
ij

pijk

ij
ij

, if

ij

0, otherwise

Update Feasible Neighbourhood

N\ j

Pheromone update
(l 1) (l ) (1
k
ij

Q f , if (i, j )
0, otherwise

k
ij

)
S

Initialization
Set ij = 0
For l =1: Nmax
Build a complete tour
For i = 1 to n
For k = 1 to m
Choose node
Update N
Apply Local Heuristic
end
end
Analyze solutions
For k = 1 to m
Compute fk
end
Update pheromones
end

497

45

Notation for ants in TSP

cij cost for transversal from city i to city j;
ij pheromone in edge (i,j);
ij amount of pheromone deposited in edge (i,j);
ij = 1/cij local heuristic;
pij probability that ant k in city i visits city j;
N set of cities still to be visited by ant k in city i;
evaporation coefficient;
parameters that determine relative importance
of pheromone versus heuristic;

498

Traveling Salesman Problem

n cities (5)
Complexity: (n1)! / 2
2
2

5
2

17

499

46

Solution using the nearest city heuristic

Step #1

Step #2
2

Step #3

Step #4

2
2

2
17

500

Solution using the nearest city heuristic

Step #5
2

The final solution is

obviously non-optimal

17
2

This heuristic can give the

optimal solution if it is
given a proper initial node

2
17

2
5
2
2

501

47

ACO in Travelling Salesman Problem

m ants
n cities
=1 d

vs
ACO balances the heuristic information with the
experience (pheromone) information
502

Iteration l = 1, ant m = 1
All paths have the same pheromone intensity 0=0.5.
Pheromone trail and heuristic information have the same
weight = 1, = 1, = 0.1.
An ant is randomly placed.
The probability to choose is, in this case, based only on
heuristic information:
p12=30%
p13=14%
p14=26%
p15=30%

Step #1

Ant m = 1 chooses node 5.

503

48

Iteration l = 1, ant m = 1
Step #2

32%

Step #3

53%

22%

47%
46%

Step #5

Step #4
100%

f1

2 2 2

17 12.36
504

Iteration l = 1, ant m = 2
All paths have the same pheromone intensity 0=0.5.
Pheromone trail and heuristic information have the same
weight = 1, = 1, = 0.1.
An ant is randomly placed.
The probability to choose is, in this case, based only on
heuristic information:
p12=30%
p13=14%
p14=26%
p15=30%

Step #1

Ant m = 1 chooses node 2.

505

49

Iteration l = 1, ant m = 2
Step #3

Step #2
39%
27%

34%

35%
65%

Step #5

Step #4

100%

f2

5 2 2 10.47
506

Iteration l = 1, pheromone update

The final solution of ant m=1 is
D=12.36. The reinforcement
produced by this ant m=1 is 0,08.

The final solution of ant m=2 is

D=10,47. The reinforcement
produced by ant m=2 is 0,095!

507

50

Updating pheromone matrix

The pheromone update can be done following different approaches:
Considering the pheromone dropped by every ants

l 1

0 .5 0 .5 0 .5 0 .5 0 .5

0.08

0.095

0 .5 0 .5 0 .5 0 .5 0 .5

0.08

0.095

0 .5 0 .5 0 .5 0 .5 0 .5

0.08

0.095

0 .5 0 .5 0 .5 0 .5 0 .5

0.08

00.95

0 .5 0 .5 0 .5 0 .5 0 .5

0.08

0.095

Considering the pheromone dropped by the best ant of the present iteration

l 1

0 .5 0 .5 0 .5 0 .5 0 .5

0.095

0 .5 0 .5 0 .5 0 .5 0 .5

0.095

0 .5 0 .5 0 .5 0 .5 0 .5

0.095

0 .5 0 .5 0 .5 0 .5 0 .5

00.95

0 .5 0 .5 0 .5 0 .5 0 .5

0.095

Considering the pheromone dropped by the best ant in all iterations

(after iteration N = 1, this is the same as the previous approach)

508

Iteration l = 2, ant m = 1
The pheromone trails have different intensities.
Pheromone trail and heuristic information have the same
weight = 1, = 1, = 0.1.
An ant is randomly placed.
The probability to choose is:
p41=19%
p42=26%
p43=23%
p45=32%

Step #1

Ant m = 1 chooses node 5.

509

51

Iteration l = 2, ant m = 1
Step #2

46%

32%

Step #3

71%

22%

29%

Step #5

Step #4
100%

f1

2 2 2

5 10.47
510

Iteration l = 2, ant m = 2
The pheromone trails have different intensities.
Pheromone trail and heuristic information have the same
weight = 1, = 1, = 0.1.
An ant is randomly placed.
The probability to choose is:
p21=26%
p23=29%
p24=26%
p25=19%

Step #1

Ant m = 2 chooses node 3.

511

52

Iteration l = 2, ant m = 2
Step #2

Step #3

24%
37%
24%
63%

52%

Step #4

Step #3

100%

f2

5 2 2 2 10.47
512

Iteration l = 2, pheromone update

The final solution of ant

m=1 and m=2 is D=10,47.

The reinforcement
produced by each ant is
0,095.

513

53

Updating pheromone matrix

Considering the pheromone dropped by every ants

l 1

0.45
0.45
0.45
0.45
0.55

0.55
0.45
0.45
0.45
0.45

0.45
0.55
0.45
0.45
0.45

0.45
0.45
0.55
0.45
0.45

0.45
0.45
0.45
0.55
0.45

0
0.095
0
0
0
0
0
0.095
0
0
0
0
0
0.095
0
0
0
0
0
00.95
0.095
0
0
0
0

0
0.095
0
0
0
0
0
0.095
0
0
0
0
0
0.095
0
0
0
0
0
00.95
0.095
0
0
0
0

Considering the pheromone dropped by the best ant of the

present iteration
l 1

0.45 0.55 0.45 0.45 0.45

0.095

0.45 0.45 0.55 0.45 0.45

0.095

0.45 0.45 0.45 0.55 0.45

0.095

0.45 0.45 0.45 0.45 0.55

00.95

0.55 0.45 0.45 0.45 0.45

0.095

514

ACO toolbox for Matlab

Place ant at the first node randomly
j = start_ants (number of nodes)

Choose next node using pijk

j = next_node(i, feasible neighborhood, , , , )

Pheromone update
= update_phero(solution, f(solution), , )

515

54

Fujitsu-Siemens logistics

516

Optimization of a logistic system

Poisson
Suppliers
(Delay)

Order # 1
A
B

Request the components

Component stock

A
B
B

Order # 2

Exponential

C
A
B
C
E
Order # 3
A

C
D
E

Order # 1
Orders
Order # 3
Order # 2
Order # 1

A
Scheduling
B
decision process
C

517

55

Optimization results
Best optimization result for a specific day
Method

fL

Orders at correct
date

Orders delay

max (T)

(T)

T[s]

GA

25.12

12

23

2.61

32

ACO

25.12

23

23

2.61

954

Average optimization results for a specific day

Method

Orders at correct date

(fL)

T[s]

GA

25.93

1.06

11.64

0.24

32.1

0.95

ACO

25.35

0.16

11.96

0.40

934.6

20.30
518

Comparing results
ACO takes less iterations to find the optimum
GA is much faster than ACO

519

56

Comparing results
GA and ACO present similar performances
ACO has less variation between runs

GA are faster than ACO

Is there any advantage to use ACO?
ACO records information!
How to use this information?
Dynamic environments
Distributed optimization

520

57