Department of Computer

science University of
Peshawar
Introduction to Genetic Algorithms

Advanced Algorithms

AdvAlgo-Spr08

Genetics: Introduction
The study of inheritance, fundamental to all of
biology
The study of (i) structure (ii) function and the
(iii) transmission of genes from parents to
offspring.
Gene: A specific sequence of nucleotide
bases, that carry information required for
constructing proteins.
Proteins: Provide structural components of
cells, tissues and enzymes for essential
biochemical reactions. AdvAlgo-Spr08

Genetics: Introduction Cont

Genes of living organisms contain an internal
description encoded in the chemical text of
DeoxyriboNucleic Acid (DNA).
It is this description and NOT the organism
itself which is handed down from generation
to generation (inheritance).
The complete set of instructions for making
an organism is called its genome. The human
genome consists of DNA and associated
protein molecules.
The human genome is AdvAlgo-Spr08
estimated to comprise

Genetics: Introduction Cont

The Human Genome at Four Levels of Detail

AdvAlgo-Spr08

Genetics: Writing on the Wall

The Gene is by far the most sophisticated program around.
- Bill Gates, Business Week, June 27, 1994

10 Hottest jobs of 2025

Time Magazine May 2000

Gene programmers Digital genome maps will

allow lab technicians to create customized
prescriptions, altering individual genes by rewriting
lines of computer code. After scanning your DNA
for defects, doctors will use gene therapy and
"smart" molecules to prevent a variety of diseases,
including certain cancers.
AdvAlgo-Spr08

Genetic Algorithms: Overview

A class of probabilistic optimized search
algorithms based on the mechanics of natural
selection and natural genetics.
Inspired by the biological evolution process.
Dinosaurs are dead, cockroaches still
surviving.
Based on the survival of the fittest among
string structures with a structured yet
randomized exchange to form search
algorithm.
AdvAlgo-Spr08

Uses concepts of Natural Selection and

Genetic Algorithms: Overview

Formally introduced in the 1970s by John
Holland at University of Michigan.
Now became attractive for some types of
optimization problems because of
price/performance improvements of
computational systems.
Work very well on mixed (continuous and
discrete), combinatorial problems where little
is known about the underlying search space.
AdvAlgo-Spr08

Genetic Algorithms: Overview

Less susceptible to getting 'stuck' at local
optima than gradient search methods. But
tend to be computationally expensive
In every search a new set of artificial
creatures (strings) are created using bits and
pieces of the fittest of the old; occasionally a
new part tried.
Randomized genetic algorithms are NOT a
random walk, employ historical information.
Widely-used in business, science and
AdvAlgo-Spr08
engineering.

Genetic Algorithms: Theme

Robustness i.e. the balance between efficiency
and efficacy necessary for survival in many
different environments.
If artificial systems can be made more robust,
costly redesigns can be reduced or eliminated.
Self-repair, self-guidance and reproduction are
the rule in biological systems, but barely exist
in most sophisticated AI systems.

AdvAlgo-Spr08

Robustness: Traditional optimization and

search
Three main types of search methods:
1. Calculus based
a. Direct
b. Indirect

2. Enumerative
3. Random

AdvAlgo-Spr08

10

Calculus: Traditional optimization and

search
Indirect techniques: Seek local maxima by
solving non-linear set of equations, by setting
gradient of objective function to zero.
Given a smooth unconstrained function,
finding a possible (single) peak starts by
restricting search to those points with slopes
of zero in all directions.
Direct techniques: Hopping the function,
and moving in a direction related to the local
gradient.
Notion of hill climbing: AdvAlgo-Spr08
Find the local best by

11

Calculus: Problem

AdvAlgo-Spr08

12

Calculus: Problems
Both methods are local in scope. The optima
sought is best in the neighborhood of current
point.
Clearly zero finding in the neighborhood of
lower peak will cause us to miss the higher
peak.
Once the lower peak is reached, further
improvement may be sought by random
restart or other tricks.
Based on the existence of derivatives (well
AdvAlgo-Spr08
defined slope) and implied
smoothness.

13

Real-Life:

Unsuitability of traditional optimization for

search

AdvAlgo-Spr08

14

Enumerative: Traditional optimization and

search
Within a finite search space or discretized
infinite search space, look at objective
function value at every point in space one at
a time. Problem?
Analysis:
Attractive as mimicking human search.
Works when number of possibilities are small.
NOT efficient as practical spaces are too
large.
DP breaks down on problems of moderate size
and complexity, what Bellman
1961 called
AdvAlgo-Spr08
15

Random: Traditional optimization and

search

Became popular because of shortcomings of

calculus based and enumerative schemes.
But random techniques that search and save
the best must also be discontinued because
of inefficiency.
In the long run do no better than
enumerative.
Simulate Annealing uses random process to
held guide search to minimal energy states.
Note that GA use random choice as a tool in
directed search process.
AdvAlgo-Spr08

16

Conclusion: Traditional optimization and

search

Random

Directionless

Conventional search methods NOT

robust

They are NOT useful

AdvAlgo-Spr08

17

Efficiency

Comparison: Traditional optimization and

search

Robust scheme

Specialized scheme

Enumeration or
Random walk
Combinatorial

unimodal

Problem Type
AdvAlgo-Spr08

multimodal
18

Genetic Algorithms: Classes of Search

Techniques
Search Techniques
Calculus Base
Techniques
Fibonacci

Sort

Tabu Search

Enumerative
Techniques

Guided random search

techniques
DFS
Hill
Climbing

Simulated
Anealing
Genetic
Programming
AdvAlgo-Spr08

Dynamic
BFS
Programming

Evolutionary
Algorithms
Genetic
Algorithms
19

Genetic Algorithms:

A genetic algorithm maintains a

population of candidate solutions
for the problem at hand, and
makes it evolve by iteratively
applying a set of stochastic
operators.

AdvAlgo-Spr08

20

Genetic Algorithms: Difference

GAs differ from more normal optimization
and search procedures in 4 ways:
GAs work with a coding of the parameter set,
not the parameters themselves.
GAs search from a population of points, not a
single point.
GAs use payoff (objective function) information,
not derivatives or other auxiliary knowledge.
GAs use probabilistic transition rules, not
deterministic rules.
AdvAlgo-Spr08

21

Genetic Algorithms: Stochastic Operators

Selection replicates the most successful
solutions found in a population at a rate
proportional to their relative quality. Also
known as Reproduction.
Examples of selections schemes:
Roulette wheel selection (probabilistic
selection based on fitness).
Rank selection (pick the best individual
each time).
Tournament selection (select K individuals,
AdvAlgo-Spr08
22
and keep best for reproduction).

Genetic Algorithms: Stochastic Operators

Cont
Recombination decomposes two distinct
solutions and then randomly mixes their parts
to form novel solutions. Also known as
Crossover.
Mutation randomly perturbs a candidate
solution.

AdvAlgo-Spr08

23

Genetic Algorithms: Metaphors to Nature

Genetic
Algorithms
Optimization
problem
Feasible solutions
Solutions quality
(fitness function)

Nature
Environment
Individuals living in
that environment
Individuals degree
of adaptation to its
surrounding
environment
AdvAlgo-Spr08

24

Genetic Algorithms: Metaphors Cont

Genetic
Nature
Algorithms
A set of feasible
A population of
solutions
organisms (species)
Stochastic operators Selection,
recombination and
mutation in natures
evolutionary process
Iteratively applying Evolution of
a set of stochastic
populations to suit
operators on a set of their environment
feasible solutions
AdvAlgo-Spr08

25