Scatter Search:

Methodology and
Applications
Manuel Laguna
University of Colorado
Rafael Mart
University of Valencia

Based on
Scatter Search: Methodology and
Implementations in C
Laguna, M. and R. Mart
Kluwer Academic Publishers, Boston, 2003.

Scatter Search
Methodology

Metaheuristic
A metaheuristic refers to a master strategy
that guides and modifies other heuristics
to produce solutions beyond those that are
normally generated in a quest for local
optimality.
A metaheuristic is a procedure that has
the ability to escape local optimality
4

Typical Search Trajectory

100
90

Objective Function

80
70
60
Value

50

Best Value

40
30
20
10
0
1

10

13

16

19

22

25

28

31

34

37

40

Iteration

Metaheuristic Classification
x/y/z Classification

x = A (adaptive memory) or M (memoryless)

y = N (systematic neighborhood search) or S (random
sampling)
z = 1 (one current solution) or P (population of
solutions)

Some Classifications

Tabu search (A/N/1)

Genetic Algorithms (M/S/P)
Scatter Search (M/N/P)
6

Scatter Search
P
Diversification Generation
Method

Repeat until |P| = PSize

Improvement
Method

Improvement
Method

Reference Set
Update Method

Solution Combination
Method

Subset Generation
Method
RefSet
Stop if no more
new solutions

Scatter Search with Rebuilding

P
Diversification Generation
Method

Repeat until |P| = PSize

Improvement
Method

Improvement
Method

Solution Combination
Method

Reference Set
Update Method
Stop if MaxIter
reached
Improvement
Method

Subset Generation
Method
RefSet
No more new
solutions

Diversification Generation
Method

Tutorial
Unconstrained Nonlinear Optimization
Problem
Minimize

100 x 2

2 2
x1

+ (1 x1 ) + 90 x 4
2

2 2
x3

+ (1 x3 ) +
2

10.1 (x 2 1) + (x 4 1) + 19.8(x 2 1)(x 4 1)

Subject to

10 xi 10

for i = 1, K ,4

Diversification Generation
Method
Subrange 1

-10

Subrange 2

-5

Subrange 3

Subrange 4

+5

+10

Probability of selecting a subrange is proportional to a frequency count

10

Diverse Solutions
Solution
1
2
3
4
5
6
7
8
9
10

x1
1.11
-9.58
7.42
8.83
-6.23
1.64
0.2
-3.09
-6.08
-1.97

x2
0.85
-6.57
-1.71
-8.45
7.48
3.34
-3.64
6.62
0.67
8.13

x3
9.48
-8.81
9.28
4.52
6
-8.32
-5.3
-2.33
-6.48
-5.63

x2
-6.35
-2.27
5.92
3.18
7.8
-8.66
-7.03
-3.12
1.48
8.02

f(x)
835546.2
1542078.9
901878.0
775470.7
171450.5
546349.8
114023.8
7469.1
279099.9
54537.2

11

Improvement Method
Solution
1
2
3
4
5
6
7
8
9
10

x1
2.5
-0.52
-2.6
0.49
-3.04
-1.4
-0.36
-1.63
-0.8
-2.47

x2
5.4
-0.5
5.9
0.53
9.45
2.46
-0.31
2.51
0.69
5.32

x3
2.59
0.35
4.23
2.47
1.14
0.37
0.8
0.73
-1.16
-2.92

x2
5.67
-0.14
10
5.89
0.41
-3.94
1.14
0.56
1.5
8.15

f(x)
1002.7
138.5
7653.7
213.7
720.1
1646.7
57.1
21.5
11.2
1416.7

Nelder and Mead (1965)

12

Reference Set Update Method

(Initial RefSet)
b1 high-quality
solutions

Objective function
value to measure
quality

b2 diverse
solutions

Max-min criterion and

Euclidean distances to
measure diversity

RefSet of size b
13

Initial RefSet
High-Quality Solutions
Solution
number in P
35
46
34
49
38

x1

x2

x3

x4

-0.0444
1.133
-0.0075
1.1803
1.0323

0.0424
1.2739
0.0438
1.4606
0.9719

1.3577
-0.6999
1.4783
-0.344
-0.8251

1.8047
0.5087
2.2693
0.2669
0.695

f(x)
2.1
3.5
3.5
5.2
5.3

Diverse Solutions
Solution
37
30
45
83
16

x1
-3.4331
3.8599
-4.4942
-0.2414
6.1626

x2
10
10
10
-6.5307
10

x3
1.0756
-4.0468
3.0653
-0.9449
0.1003

x4
0.3657
10
10
-9.4168
0.1103

f(x)
1104.1
9332.4
13706.1
17134.8
78973.2
14

Subset Generation Method

All pairs of reference solutions that include
at least one new solution
The method generates (b2-b)/2 pairs from
the initial RefSet

15

Combination Method
y
x3 = x1 - r(x2 - x1)

x3 = (9,7)
r = 2/3

10

x4 = x1 + r(x2 - x1)
x5 = x2 + r(x2 - x1)

9
x1 = (5,7)

8
7
6
5

x4 = (6.5,5.5)
r = 1/2

x2 = (8,4)

4
3
x5 = (11,1)
r=1

2
1

10

11

12

13

16

Alternative Combination Method

y
x3 = x1 - r(x2 - x1)
10

x4 = x1 + r(x2 - x1)
x5 = x2 + r(x2 - x1)

x3
x1 = (5,7)

8
7
6
x4
5

x2 = (8,4)

4
3
x5
2
1

10

11

12

13

17

Reference Set Update Method

Quality

1
2
.
.
.

Best

1
2
.
.
.

Best

Worst

New trial
solution

Worst
RefSet of size b

Updated RefSet

18

Static Update
Pool of new trial solutions
Quality

1
2
.
.
.

Best

Updated RefSet = Best b from RefSet Pool

b

Worst
RefSet of size b
19

RefSet after Update

x1

x2

x3

x4

f(x)

1.1383

1.2965

0.8306

0.715

0.14

0.7016

0.5297

1.2078

1.4633

0.36

0.5269

0.287

1.2645

1.6077

0.59

1.1963

1.3968

0.6801

0.446

0.62

0.3326

0.1031

1.3632

1.8311

0.99

0.3368

0.1099

1.3818

1.9389

1.02

0.3127

0.0949

1.3512

1.8589

1.03

0.7592

0.523

1.3139

1.7195

1.18

0.2004

0.0344

1.4037

1.9438

1.24

1.3892

1.9305

0.1252

-0.0152

1.45

20

Tutorial
0-1 Knapsack Problem
Maximize

10x1 + 14x2 + 9x3 + 8x4 + 7x5 + 5x6 + 9x7 + 3x8

S.t.

7x1 + 12x2 + 8x3 + 9x4 + 8x5 + 6x6 + 11x7 + 5x8 < 100
xi = { 0, 1}

for i = 1, , 8

21

Additional Strategies
Reference Set

Rebuilding
Multi-tier

Subset Generation

Subsets of size > 2

Combination Method

Variable number of solutions

22

Rebuilding
RefSet

Rebuilt RefSet
b1

b2

Diversification
Generation Method

Reference Set
Update Method

23

2-Tier RefSet
Solution Combination
Method
Improvement
Method

RefSet
Try here first

b1

If it fails, then
try here

b2

24

3-Tier RefSet
Solution Combination
Method
Improvement
Method

RefSet

Try here first

b1

If it fails, then
try here

b2

Try departing
solution here

b3

25

Subset Generation
Subset Type 1: all 2-element subsets.
Subset Type 2: 3-element subsets derived
from the 2-element subsets by augmenting
each 2-element subset to include the best
solution not in this subset.
Subset Type 3: 4-element subsets derived
from the 3-element subsets by augmenting
each 3-element subset to include the best
solutions not in this subset.
Subset Type 4: the subsets consisting of the
best i elements, for i = 5 to b.
26

Subsets of Size > 2

90%
80%
70%
60%
50%
40%
30%
20%
10%
Random

0%
Type 1

Type 2

LOLIB
Type 3

Type 4

27

Variable Number of Solutions

Quality

1
2
.
.
.

Best

Generate 5 solutions
Generate 3 solutions
Generate 1 solution

Worst
RefSet of size b

28

Hybrid Approaches
Use of Memory

Tabu Search mechanisms for intensification

and diversification

GRASP Constructions
Combination Methods

GA Operators
Path Relinking

29

Multiobjective Scatter Search

This is a fruitful research area
Many multiobjective evolutionary
approaches exist (Coello, et al. 2002)
SS can use similar techniques developed
for MOEA (multiobjective evolutionary
approches)
30

Multiobjective EA Techniques
Independent Sampling
Search on f(x) = w f (x)
i i

Change weights and rerun

Criterion Selection

Divide reference set into k subsets

Admission to ith subset is according to fi(x)

31

Scatter Search
Applications and Implementations

Scatter Search Elements

Diversification Generator Method
Improvement Method
Reference Set

Initialization
Update
Rebuild

Problem Independent

Subset Generation Method

Solution Combination Method
33

C Code Objective
Illustrate the methods and strategies
Use it in different applications
Extend the previous version:

http://www.uv.es/~rmarti/sscode.html
Modularity
Versatility
Non experts (use as it is with small changes)
Experts (can easily introduce important modifications)

Ansi C (different platforms and systems)

34

Basic Design
P
Diversification Generation
Method

Repeat until |P| = PSize

Improvement
Method

Improvement
Method

Reference Set
Update Method

Solution Combination
Method

Subset Generation
Method
RefSet
Stop if no more
new solutions

35

RefSet Data Structure

typedef struct REFSET
{ int b;
/*
double **sol;
/*
double *ObjVal;
/*
int *order;
/*
int *iter;
/*
int NewSolutions; /*
} REFSET;

RefSet

Size */
Solutions */
Objective value of solutions */
Order of solutions */
Entering iteration */
=1 if new element has been added */

nvar

b
sol
ObjVal
order
iter
NewSolutions
36

Main Function
int main(void)
{
SS *pb;
int nvar = 10;
int b
= 10;
int PSize = 100;

/*
/*
/*
/*

Pointer to problem data

Number of variables
Size of reference set
Size of P

*/
*/
*/
*/

pb = SSProblem_Definition(nvar,b,PSize);
/* Insert here the Scatter Search code */
SSFree_DataStructures(pb);
return 0;
}

37

Basic Design
pb = SSProblem_Definition(nvar,b,PSize);
SSCreate_P(pb);
SSCreate_RefSet(pb);
While(pb->rs->NewSolutions)
SSUpdate_RefSet(pb);
SSBestSol(pb,sol,&value);
SSFree_DataStructures(pb);

38

Advanced Design
pb = SSProblem_Definition(nvar,b,PSize);
SSCreate_P(pb);
SSCreate_RefSet(pb);
for(Iter=1; Iter<100; Iter++)
{
if(pb->rs->NewSolutions)
SSUpdate_RefSet(pb);
else
SSRebuild_RefSet(pb);
}
SSBestSol(pb,sol,&value);
SSFree_DataStructures(pb);
39

SSCreate_P
while(currentPSize <= pb->p->PSize)
{
SSGenerate_Sol(pb,sol);
obj_val = sol_value(sol);
SSImprove_solution(pb,sol,&obj_val);
/* Check whether sol is a new solution */
j=1;equal=0;
while(j<currentPSize && !equal)
equal=SSEqualSol(sol,pb->p->sol[j++],pb->nvar);
/* Add improved solution to P */
if(!equal){
for(j=1;j<=pb->nvar;j++)
pb->p->sol[currentPSize][j]=sol[j];
pb->p->ObjVal[currentPSize++] = obj_val;
}
}

40

SSCreate_RefSet
/* Order solutions in P */
p_order = SSOrder(pb->p->ObjVal, pb->p->PSize, pb->Opt);
b1 = pb->rs->b / 2;
/* Add the first (best) b1 solutions in P to RefSet */
/* Compute minimum distances from P to RefSet */
for(i=1;i<=pb->p->PSize;i++)
min_dist[i]= SSDist_RefSet(pb,b1,pb->p->sol[i]);
/* Add b-b1 diverse solutions to RefSet */
for(i=b1+1;i<=pb->rs->b;i++) {
a=SSMax_dist_index(pb,min_dist);
/* Copy sol a from P to Refset */
SSUpdate_distances(pb,min_dist,i);
}
/* Compute the order in RefSet: order */
pb->rs->NewSolutions = 1; pb->CurrentIter = 1;

41

SSUpdate_RefSet
pb->rs->NewSolutions=0;
SSCombine_RefSet(pb);
pb->CurrentIter++;
for(a=1;a<=pb->pool_size;a++)
{
value=sol_value(pb->pool[a]);
SSImprove_solution(pb,pb->pool[a],&value);
SSTryAdd_RefSet(pb,pb->pool[a],value);
}
pb->pool_size=0;

42

Advanced Designs
Reference Set Update

Dynamic / Static
2 Tier / 3 Tier

Subset Generation
Use of Memory

Explicit Memory
Attributive Memory

Path Relinking
43

An Example
The Linear Ordering Problem
Given a matrix of weights E = {eij}mxm, the LOP consists of
finding a permutation p of the columns (and rows) in
order to maximize the sum of the weights in the upper
triangle
m 1

Maximize CE ( p ) =

i =1 j = i +1

pi p j

Applications

Triangulation for Input-Output Economic Tables.

Aggregation of individual preferences
Classifications in Sports

An Instance
1
1
2
3
4

0 12 5 3
4 0 2 6

8 3 0 9

11
4
2
0

p=(1,2,3,4)
cE(p)=12+5+3+2+6+9=37

3
4
1
2

0
2

3
0 11 4

3 0 12

3 4 0
8

p*=(3,4,1,2)
cE(p*)=9+8+3+11+4+12=47

SS for the LOP

Complete the generic code
Design specific methods for the problemdependent elements

Diversification Generator Method

Improvement Method
Combination Method

46

Diversification Generator
Use of problem structure to create methods in order to
achieve a good balance between quality and diversity.
Quality

Deterministic constructive method

Diversity

Random Generator
Systematic Generators (Glover, 1998)

GRASP constructions.

The method randomly selects from a short list of the most

attractive sectors.

Use of Memory

Modifying a measure of attractiveness proposed by Becker with

a frequency-based memory measure that discourages sectors
from occupying positions that they have frequently occupied
47

Diversity vs. Quality

Compare several diversification generators
Create a set of 100 solutions with each one
1.6
1.4
1.2

d = Standardized Diversity
C = Standardized Quality

C+ d

1.0
0.8

0.6

0.4

0.2
0.0
DG05

DG04

DG02

DG01

DG09

DG08

DG06

DG03

DG07

DG10

Procedure

48

Improvement Method
INSERT_MOVE (pj, i) consist of deleting pj from
its current position j to be inserted in position i
Apply a first strategy

scans the list of sectors in search for the first sector

whose movement results in an improvement
0 12 5 3 1

6 0 3 6 4
8 5 0 5 7

E( p) = 2 7 2 0 3
8 0 3 1 0

9 1 6 2 13

2 9 4 5 8

8 3

4 2
0 3

6 0
4 1

0 4

1 0

MoveValue = CE(p) - CE(p)

CE (p) = 78 + (1 - 4) + (6 - 0) + (2 - 6) +
(13 - 4) = 78 + 8 = 86

49

Solution Combination Method

The method scans (from left to right) each reference
permutation.

Each reference permutation votes for its first element that is still
not included in the combined permutation (incipient element).
The voting determines the next element to enter the first still
unassigned position of the combined permutation.
The vote of a given reference solution is weighted according to
the incipient elements position.

Incipient element
(3,1,4,2,5) votes for 4
(1,4,3,5,2) votes for 4
(2,1,3,5,4) votes for 5

Solution under construction:

(3,1,2,4,_ )

50

Experiments with LOLIB

49 Input-Output Economic Tables
GD

CK

CK10

TS

SS

Optima
deviation

0.15% 0.15%

0.02%

0.04%

0.01%

Number of
optima

11

11

27

33

42

Run time
(seconds)

0.01

0.10

1.06

0.49

2.35

51

Another Example
A commercial SS implementation

OptQuest Callable Library (by OptTek)

As other context-independent methods
separates the method and the evaluation.
Output

Optimization
Procedure

Input

System
Evaluator

52

OptQuest based Applications

Solution Generator

OptQuest
Callable Library
User-written
Application

Solution Evaluator

System
Evaluator

53

Feasibility and Evaluation

User Implementation
F(x*)
x

Constraint
Mapping

x*

Complex System
Evaluator

The OptQuest engine

generates a new solution

G(x )

Penalty
Function

P(x*)

Returns to
OptQuest
54

Comparison with Genocop

Test on 28 hard nonlinear instances
1.0E+13

Average objective function value

(Logarithmic scale)

1.0E+12

Genocop

OCL

1.0E+11
1.0E+10
1.0E+09
1.0E+08
1.0E+07
1.0E+06
1.0E+05
1.0E+04
1.0E+03

1000

2000

3000

4000

5000

6000

7000

8000

9000

10000

Evaluations
55

Conclusions
The development of metaheuristics usually
entails a fair amount of experimentation
(skill comes from practice).
Code objectives:

Quick Start
Benchmark
Advanced Designs

Scatter Search provides a flexible

framework to develop solving methods.
56

Call for Papers

European Journal of Operational Research
Feature Issue on
SCATTER SEARCH METHODS
FOR OPTIMIZATION
Deadline for submissions: June 30, 2003
http://www.uv.es/~rmarti/ejor.html
57

Questions

58