Advanced

Examples
and Ideas

Three Layer Evolutionary

Approach
Local perceptions, such as
Encoded behaviors
Time intervals
bald head or long beard

Evolve
Perceptio
ns

or internal states

Evolve
Behavior
s

Evolve
Motions

Global perceptions, possibly

encoded such as narrow
Corridor or beautiful
Behaviors such as go forward
Princess
until you find a wall, else turn
randomly right or left

Motions as timed
sequences of encoded
actions, for instance
RFRFLL

Evolve in hierarchy
1. Together or separately
2. Feedback from model or from real
world
3. First evolve motions and encode them.
4. Then evolve behaviors.
5. Finally develop perceptions.
If you see
a beautiful
If you
Go to the end
princess
see a
of the corridor
go to her
dragon
and then look
and bow
escape
for food

Evolve in hierarchy
avoid
obstacles
Look for
energy
sources in
advance

Execute
optimal
motions

Save energy

Execute
actions that
you enjoy

What if robot likes to play

soccer and sees the ball
but is low on energy?

Optimizing a motion

Parking
a Truck

Find the control

Solving this
analytically would be
very difficult

Question; How to represent the

chromosomes?
Here you see several snapshots of a
movie about parking a truck, stages of
the solution process.

 t is time u

Another example

Learning
Obstacle
Avoiding

Similar to Braitenberg Vehicle but has 8 sensors

how

Input and output

data are some
form of MV logic

How would
you represent
chromosomes
?
Design
Crossovers?
1. Robot can move freely but
has to avoid obstacles
2. This can be like the lowest
level of behaviors in
subsumption or other
behavioral architecture for
all your robots

Remember the goal when

you create the fitness
function

The key
to
success
is often in
fitness
function

Number of collisions

When you
train longer
you decrease
the number of
collisions

Time of
learning

Applicatio
ns and
Problems

General GA
Schema

Evolutionary Methods
Optimization problems:
Single objective optimization problems
Multi-Objective optimization Problems

More examples of problems in which we use

evolutionary algorithms and similar methods.

Search Problems (Path search)

Optimal multi-robot coordination
Multi-task optimization
Optimal motion planning of robot arms (Trajectory planning
of manipulators )
Motion optimization (optimization of controller parameters morphology in different control schemas)

PID (PI)
Fuzzy
Neural
Hybrid (neuro-fuzzy)

Path planning and tracking (mobile robots)

Optimal motion planning of robot arms
Trajectory planning of manipulators

Vision computational optimization

What are these other

algorithm?
Evolutionary Algorithms - Related techniques:

Ant colony optimization (ACO)

Particle swarm optimization
Differential evolution
1. You can try them
Memetic algorithm (MA)
in your homework
Simulated annealing
1 if GA or GP is
too easy for you.
Stochastic optimization
2. Using them gives
Tabu search
you higher
Reactive search optimization (RSO)
possibility of
Harmony search (HS)
creating a
Non-Tree Genetic programming (NT GP)successful
superior method
Artificial Immune Systems (AIS)
for a new problem
Bacteriological Algorithms (BA)

GA-operators
Selection

Roulette
Tournament
Stochastic sampling
Rank based selection
Boltzmann selection
Nonlinnear ranking selection

Crossover
One point
Multiple points

Mutation

Read in Auxiliary Slides

about these methods.
Or invent your own
operators for your
problem.

Your design parameters to

be decided
Genotype length
Fixed length genotype
Variable-length genotype

Population
Fixed population
Variable population
Species inside population
Geometrical separation

Drawbacks of GA
time-consuming when dealing with a
large population
premature convergence
Dealing with multiple objective problems

Solutions

Niches
Islands
Pareto approach
Others

More examples of using GA

in robotics

Trajectory
Planning
Problems

GA and Trajectory Planning

1. GA techniques for robot arm to identify the
optimal trajectory based on minimum joint torque
requirements (P. Garg and M. Kumar, 2002)
2. path planning method based on a GA while
adopting the direct kinematics and the inverse
dynamics (Pires and Machado, 2000)
3. point-to-point trajectory planning of flexible
redundant robot manipulator (FRM) in joint space
(S. G. Yue et al., 2002)
4. point-to-point trajectory planning for a 3-link
(redundant) robot arm, objective function is to
minimizing traveling time and space (Kazem,
Mahdi, 2008)
Projects last years

Optimal path generation of

robot manipulators
1.
2.
3.
4.
5.
6.

Control Schema
Robotic arm kinematic model
Controller type
Objective function - optimal path
Optimization algorithm (method)
GA use smooth operators and avoids
sharp jumps in the parameter values.

 Adaptive Control Schema Track Control error

function between outputs of a real system and
mathematical model

What we optimize?
Which parameters must be optimized?
How many objectives (single objective or
multiobjective)?
Collision free? (How to model collision in GA?)

 Three join Manipulator

A three-joint robotic manipulator system has

three inputs and three outputs.
The inputs are the torques applied to the joints
and the outputs are the velocities of the joints
No ripples

Design of robotic
controllers

For n-DOF we will have n inputs u i, i=1n, (ui

i)
Controller
PID (PI)
Neural network (multilayer perceptron, recurrent
NN, RBF based NN)
Fuzzy
Neuro-Fuzzy (hybrid)

Use of Neural Networks

NN: We must to adapt the weights and eventually the bias
The chromosome:

Adapt the weights wk

ij

FUZZY LOGIC

Fuzzy Logic

i
Ri : if x1i is A1i and K xiN is AN

then

i is B i
y1i is B1i and K y M
M

Aggregation of rules
defuzzification
free-of-obstacles workspace (Mucientes, et. al, 2007)
wall-following behavior in a mobile robot

Learning FUZZY LOGIC

Controllers
Learning of fuzzy rule-based controllers
Find a rule for the system
Step 1: evaluate population;
Step 2: eliminate bad rules and fill up population;
Step 3: scale the fitness values;
Step 4: repeat NI iterations for Step 4 to Step 9
Step 5: select the individuals of the population;
Step 6: crossover and mutate the individuals;
Step 7: evaluate population;
Step 8: eliminate bad rules and fill up population;
Step 9: scale the fitness values.
Step 10: Add the best rule to the final rule set.
Step 11: Penalize the selected rule.
Step 12: If the stop conditions are not fulfilled go to
Step 1

Encoding fuzzy controls

The chromosome encode the rules:

C i (b1i , c1i , b2i , c2i , K

i , ci )
bNV
NV

Sn is constant in this application but it can be also variable to

be optimized
wall-following behavior of the robot
the robot is exploring an unknown area
moving between two points in a map

Requirements
maintain a suitable distance from the wall that is being followed
to move at a high velocity whenever the layout of the
environment is permitting
avoid sharp movements (progressive turns and changes in
velocity)

Path-based robot behaviors

The requirements are encoded in
Universes of discourse and precisions of the
variables
right-hand distance (RD)
the distances quotient (DQ), based on left-hand
distance
Orientation
linear velocity of the robot (LV)
Linear acceleration
Angular velocity

Path of the robot (simulated environments)

Fast, reliable, no harm to

robot or to environment

This is useful for out

PSU Guide Robot
1. Do not harm
humans
2. Do not harm robot

 Fixed points: the desired Cartesian path Pt is given the

problem is to find the set of joint paths P in order to
minimize the cumulative error between desire and real path
Find the set
during trajectory

Pr Fk ( P )
E

N j 1 N

N j 1

Pt (i, j ) Pr (i, j )

i 2 j 1

Pk is the kinematic model

Free end points case

of joint
paths, next
smooth it

i2

E (i )

Minimize the
cumulative
error

Weighted Global Fitness

fitness function (minimization)

1
F
1 E

Global fitness: Linear function of individual objectives

Ftotal 1 fot 2 f q 3 fc 4tT
Fot excessive driving (sum of all maximum torques), fq the
total joint traveling distance of the manipulator, f c - total
Cartesian trajectory length, tT - total consumed time for robot
motion

Penalty function
Population initialization (probability distribution)
Random uniform
Gaussian

example

Drug
Delivery
Problem

Drug delivery using microrobots (Tao, et. al,

2005)
1. (GA)based area coverage approach for robot path
planning.
2. Drawbacks of most currently available drug delivery
methods are that the drug target area, delivery amount,
and
release speed are hard to be precisely controlled.

3. It is very difficult or impossible to eliminate side effects.

4. Open issues
1.actively control the delivery process
2.Access to appropriate areas that cannot be reached using
traditional devices

5. Current Issues

1.On-line path planning (solve unexpected obstacles problem)

2.Optimal path planning (efficiency, path planning)

 microcontroller is used to guide the robot movement

GA-based approach uses fine grid cell
decomposition for area coverage
Because the robot will move cell by cell, the start
point of chromosomes has to be changed
dynamically whenever the robot reaches the center
of a cell
The end point of a chromosome is not fixed and
needs to be determined by applying GA operators.
The robots may move from the center of a cell to its
8 adjacent cells along 8 directions.
some obstacles are unknown before drug delivery
(the robot discover these obstacles during the
motion)

 Expandable chromosomes
Deleting the path
Crossover operator

 New mutation operators

Travel further
Delete
Reverse delete
Stretch
Shortcut

The algorithm keep mind the visited

nodes
Extension to operational research?

Other applications using evolutionary

algorithms
Autonomous mobile robot navigation Path planning using ant colony optimization
and fuzzy cost function evaluation (Garcia, et.
al, 2009).
Legged Robots and Evolutionary Design
Optimal path and gait generations (Pratihar,
Debb, and Gosh, 2002) 0/1 absence or
presence of rule

six-legged robot
collision-free coordination of multiple
robots (Peng and Akela, 2005)

What if you want to optimize two

parameters at the same time?

Pareto
Optimizati
on

Pareto
Evolutionary
Methods

Definitions

ophelimity

noun economic,
Economic
satisfaction. The
ability to please

What is better
this or this?

We want to optimize both functions f1 and

f2

Biobjective
means two
objectives
to reach
Pareto solutions for
different algorithms

Pareto
Front

We have x and y, two objectives here

Pareto front
The single objective optimisation problem (SOP)
conduct to a minimization (or maximization) of
one cost function, less or more complex, that is
a single objective is taken into account.
Conversely, the multi-objective optimization
problem takes into account two or more
objective that has to be minimized (or
maximized) simultaneously.
Some objectives can be in competition, so a
simultaneous minimization is not possible, but
only a trade-off among them.
Some time, the number of objectives can be high,
like 16 objectives or more that make the multiobjective optimization problem (MOP) and
interesting and challenging area of research

Example of Pareto Optimization

of two parameters

Optimization
of Airplane
Wings

 Two objectives: Maximize lift, and

* In most of the
design space
the red method
is better than
the blue method
* It is good to
use many Pareto
methods and
modify
parameters

Two objectives: Maximize lift, and minimize drag

1.
2.

3.

We optimize many
parameters,
We may switch
between subsets of
them.
Subsets can have
two elements each.

Multi-Pareto

Threedimensional

Minimization
Problem

Pareto
Front

General multiobjective
optimization problem
The multiobjective optimization
problem could be generally
formulated as minimization of vector
objectives Jt(x) subject to a number
of constraints and bounds:

Pareto-optimal set
1. In the case of competing objectives a trade-off is
involved such a problem usually has no unique
solution.

Instead, we can admit a set of solutions, equally valid nondominated as a set of alternative solutions known as
Pareto-optimal set

2. In what follows we assume without loss of generality

that all the function objectives must be minimized.

If we have a maximization case fi we simply minimize the

function -fi.

3. For any two points that are usually named candidate

solutions V1,V2, V1 dominates V2 in the Pareto
sense (P-dominance) if and only if the following
condition hold

f i (V1 ) f i (V2 ), i {1,..., m1 }

f j (V1 ) f j (V2 ), for at least one j {1,..., m1 }

The Pareto set

1. The Pareto set is the set of PO (ParetoOptimal) solution in design domain and the
Pareto Front (PF) is the set of PO solutions in
the objective domain.
2. The most popular way to solving the MOP
(Multi Objective Optimization Problem) is to
reduce the minimization problem to a scalar
form by aggregating the objectives in
weighted sum, with the sum of weights
constant:
n
n
min J t wi J i , wi 1
i 1

i 1

3. The weighted sum method has a serious

drawback, the method usually fail in the
case of nonconvex PF.

Example of a clear picture

of Pareto points

Nice properties
1. GA can provide an elegant solution for tradeoff
among different minimization of cost function
for each variable versus total cost or other
variable.
2. Non-convex solutions
3. Immigrants, possible solution for jump from
local minima.
4. Dealing with many variables (e.g. 16 variables)

Multi-Robots
Pareto optimal multi-robot coordination with
acceleration constraints (Jung and Ghrist,
2008)
1. collection of robots sharing a common
environment
2. each robot constrained to move on a roadmap
in its configuration space
3. each robot wishes to travel to a goal while
optimizing elapsed time considering vectorvalued (Pareto) optima
4. all illegal or collision sets are removed.

Conclusions
1. GA is not a universal panacea to optimization
problems.
2. Coding the problem into a genotype is the most
important challenge!
3. The best selection schema of individuals for crossover
operator is difficult to be chosen apriori (tournament
selection seems to be more promising)
4. A number of parameters are determined empirically:
1. Size of population
2. pc and pm even often values inspired from biology are
given
3. Other parameters in hybrid or more sophisticated GA

Good properties
1.

One of the most important element in the design of a

decoder-based evolutionary algorithm is its genotypic
representation.

2.

The genotype-decoder pair must exhibit efficiency, locality,

and heritability to enable effective evolutionary search

3.

locality, and heritability:

1. small changes in genotypes should correspond to small
changes in the solutions they represent,
and
1. solutions generated by crossover should combine
features of their parents

Sources
Dragos Arotaritei

exampl
e