OPTIMISATION

Why do you need Meta Heuristics

Dr.S.P.Anbuudayasankar, M.E.., M.B.A., PhD

Associate Professor
Department of Mechanical Engineering Amrita School of Engineering Amrita Vishwa Vidyapeetham

Coimbatore - 641112
spanbu@yahoo.com
1

Optimization vs Decision Problems

Optimization Problem
A computational problem in which the object is to find the best of all possible solutions (i.e. find a solution in the feasible region which has the minimum or maximum value of the

objective function.)

Decision Problem
A problem with a yes or no answer.
2

Objective Function & Search Space

Objective Function
some function of decision variables

Max (Min) Subject to (s.t.)

equality (=) constraints inequality (

, , ,

) constraints

Search Space

Range or values of decisions variables that will be searched during optimization

3

Types of Solutions

A solution to an optimization problem

specifies the values of the decision variables, and therefore also the
value of the objective function

A feasible solution

satisfies all constraints

An optimal solution

is feasible and provides the best objective function value

A near-optimal solution

is feasible and provides a superior objective function value, but not

necessarily the best.

Continuous vs Combinatorial

Optimization problems can be

continuous (an infinite number of feasible solutions)

combinatorial (a finite number of feasible solutions).

Constraints

Constraints can be Hard (must be satisfied) or Soft (is desirable to satisfy)

Example: In our course schedule a hard constraint is that no classes overlap a soft constraint is that no class be before 10 AM

Aspects of an Optimization Problem

Objectives Single or

Multiple

Size number of decision variables,

range/count of possible values of decision variables

search space size

Degree of constraints

number of constraints
difficulty of satisfying constraints proportion of feasible solutions to search space

Aspects of an Optimization Problem (Contd.)

Deterministic
Stochastic

(all variables are deterministic) or

(the objective function and/or some decision variables and/or some constraints are random

variables)

Decomposition

decomposite a problem into series problems, and then solve them independently

Relaxation

solve problem beyond relaxation, and then can

solve it back easier
8

Models
The essence of Operations Research lies in the

construction and use of models.

A model is a simplified representation of something real.

Optimization need Model

Guess..What model is this ?

min s.t.
j i i j

cijxij
i,
j, i, j.

xij < si, xij > dj, xij > 0,

10

Transportation Model !
min s.t.
j i i j

cijxij
i,
j, i, j.

xij < si, xij > dj, xij > 0,

Costs Supplier Kumar Ganesh Sudhir Gopal Capacity

Chennai 21 35 55 43 200,000

PLANT Bearings Madurai Cochin Coimbatore Available 50 40 35 275,000 30 22 42 400,000 20 25 70 300,000 25 37 58 500,000 600,000 225,000 350,000

12

Modeling become

P NP NP- hard & NP complete

13

What is P

P = Set of problems than can be solved in polynomial time

What is Polynomial Time ?

Reasonable Time

What is NP

NP = Set of problems for which a solution can be verified in polynomial time

What is NP Complete

Minesweeper

TSP

What is NP-Hard

Allocation Problem

5 Patient 1 4 Hospital 1

Patient 2

Hospital 2

? Commonly Believed Relationship

NP

NPHard

P NP Complete

18

When modeling become

NP-complete
&

NP-hard We need.
19

Heuristics
and

Metaheuristics

20

21

Heuristics

Heuristics

are rules to search to find optimal or near-optimal solutions

Problem Specific Algorithms

Examples

FIFO

LIFO
earliest due date first largest processing time first shortest distance first, etc.

Vogels Approximation Method to solve transportation problem

22

Heuristics

Examples of problems that are tackled with heuristic algorithms :

Cutting/Packing

Graph problems Scheduling/Timetabling

23

Packing Problem
How it works (Packing stage)

Gap

How it works (Packing stage)

2

1
Gap

How it works (Packing stage)

2

How it works (Packing stage)

2

How it works (Packing stage)

2

Gap

How it works (Packing stage)

3 4

Gap

How it works (Packing stage)

4

3 1 X

Gap

How it works (Packing stage)

4

Gap 2

3 1

How it works (Packing stage)

4 2

3 3 1

X X

How it works (postprocessing)

Due to the nature of the algorithm, towers may form

Towers are boxes with large height dimensions and lower width

The tallest tower is removed from the packing, rotated

so that it is effectively knocked down and placed back into the

packing at the lowest point available

This is repeated until the solution is not improved any further

How it works

Gap

9 12
10 6 11 4 7

832

How it works

9 12
10 6 11 4 7

832

How it works

9 12
10 6 11 4 7

832

How it works

9 12
10 6 11 4 7

832

How it works

old height Improvement

new height
1

9 12
10 6 11 4 7

832

Have you been paying attention?

1
2

Q) How many different orders can these boxes be placed in?

Answer 4 x 3 x 2 x 1 = 24 different orders

Cracking a combination lock

How do we pack all of these?

(50 pieces)

Thats not so easy

50 pieces means 50 x 49 x..2 x 1 different orderings = 304140932017133780436126081660647688443776415689605

12000000000000

Thats quite a lot really. If a computer could evaluate 1000 orderings per second itd still take approximately:

964424568801159882154128873860501295166718720476931 506 years! (and thats without being allowed to rotate the pieces)

Travelling Salesman Problem

One of the most researched Problems

43

Traveling Salesman Problem (TSP)

2

Depot

44

Combinatorial Explosion

45

Combinatorial Explosion
Cities Routes 1 1 2 1 3 3 4 12 5 60 6 360 7 2520 8 20160 9 181440 10 1814400 11 19958400

46

Combinatorial Explosion
A 10 city TSP has 181,000 possible solutions

A 20 city TSP has 10,000,000,000,000,000 possible solutions

A 50 City TSP has 100,000,000,000,000,000,000,000,000,000,00 0,000,000,000,000,000,000,000,000,000,000 possible solutions

There are 1,000,000,000,000,000,000,000 litres of water on the planet

Mchalewicz, Z, Evolutionary Algorithms for Constrained Optimization Problems, CEC 2000 (Tutorial)
47

Traveling Salesman Problem

If there are n cities,

How many possible solutions exist

The number of possible routes is (n!)/2

49

50

91.9 CPU years on a Intel Xeon 2.8 GHz processor

51

Proven NP hard problem

Vehicle Routing Problems (VRPs)

9 8 7 1 5 6

2 3

Depot

Heuristic for VRP

OVERLOAD

OK !

Depot

Heuristic for VRP

OVERLOAD

Depot

Heuristic for VRP

OK !

Depot

56

Meta-Heuristics what are they?

Wikipedia says

A metaheuristic is a heuristic method for solving a very general class of computational problems by combining user given black-box procedures usually

heuristics themselves in a hopefully efficient way.

57

Meta-heuristics

Genetic algorithms
Simulated Annealing Ant Colony Optimisation Particle Swarm Optimisation Tabu Search Hill Climbing / Greedy search

many more
58

Understanding the Concept

Local Maxima f (X) Global Maxima

Local Minima

Global Minima

Rastrigins Function
Global minimum at ( 0, 0 )

61