The Genetic Algorithms (GA)

Based on the mechanics of biological

evolution.
Initially developed by John Holland, of
University Michigan
(1970’s)
 To
To understand
design processes
artificialin natural
systemssystems.
retaining the robustness
and adaptation properties of natural systems.
Provide efficient techniques for optimization and machine learning
applications.
It is a stochastic algorithm.

It does not use gradient information.

Widely used in business, science and engineering.

2
Genetic Algorithms: Basic idea

Start with a population of candidate solutions.

Variation: Introduce variation by applying

two operators: crossover and mutation.

Survival of the fittest: Use a fitness criterion

to bias the evolution towards desired features.

3
Genetic Algorithms Techniques

GAs are a particular class of evolutionary algorithms.

The techniques common to all GAs are:
•Inheritance - offspring acquire characteristics
•Mutation - change, to avoid similarity
•Natural Selection -variations improve survival
•Crossover - recombination

GAs are best used when the objective function is:

•Discontinuous
•Highly nonlinear
•High dimension
•Stochastic
•Has unreliable or undefined derivatives

4
 Biological Terminology
Gene – a single encoding of part of the solution space,
i.e. either single bits or short blocks of adjacent bits that
encode an element of the candidate 1
solution
Chromosome – a string of genes that represents
a 1 0 1 1 0
solution
Population – the number of chromosomes available to
0 1 1 0 1
test
1 1 0 0 1
1 0 1 0 1
1 0 1 0 1
1 0 0 0 1
0 0 1 1 1
5
Mapping: Nature vs. computer
NATURE COMPUTER

POPULATION SET OF SOLUTIONS

INDIVIDUAL SOLUTION TO A PROBLEM

FITNESS QUALITY OF SOLUTION

CHROMOSOME ENCODING FOR A SOLUTION

GENE PART OF ENCODING OF A SOLUTION

REPRODUCTION CROSSOVER

6
 Features of Genetic Algorithms
Not too fast but cover large search space
• Capable of quickly finding promising regions of the search
space but may take a relatively long time to reach the optimal
solution.
Good heuristics for combinatorial problems
Usually emphasize combining information from
good parents (crossover)
Different GAs use different
• Representations
• Mutations
• Crossovers
• Selection mechanisms
7
 Representatio
n
 GAs on primarily two types of representations:
• Binary Coded
[0110, 0011, 1101, ….]
• Real Coded
[13.2, -18.11, 5.72, …]
 Binary-Coded (genotype) GAs must decode a chromosome into
a real value (phenotype), for evaluating the fitness value.
 Real-Coded GAs can be regarded as GAs that operate on the actual
real value (phenotype).
• For Real-Coded GAs, no genotype-to-phenotype
mapping is
needed.

9
 Selection
A proportion of the existing population is selected
to bread a new bread of generation.
Parents with better fitness have better chances to produce
offspring.

Rank selection
 Rank selection first ranks the population and then every
chromosome receives fitness from this ranking.
• Selection is based on this ranking rather than absolute differences
in fitness.
• The best will have fitness 1, second best 2 etc. and the worst will
have fitness N (number of chromosomes in population).

10
Tournament selection
Runs a "tournament" among a few individuals chosen at random
from the population and selects the winner (the one with the best
fitness) for crossover
• Two entities are picked out of the pool, their fitness is compared, and
the better is permitted to reproduce.
• Selection pressure can be easily adjusted by changing the tournament
size.
• Deterministic tournament selection selects the best individual in each
tournament.

Chromosomes Fitness
0 0 1 0 1 1 1 38
Winner chromosome
0 0 1 0 1 1 1
1 1 0 1 0 0 0 45 Select minimum
fitness value

11
Roulette-wheel selection
•Assign each individual part of the wheel according to proportion
of fitness value.
The probability that a individual will be selected is given by the ratio
of
its fitness to the sum of fitness of all individuals of the current population.
f (hi )
P(hi ) N


j f (h ) j
1
where hi is the i t h individual of current population and size of
population is N .
•Spin wheel (N) times to select (N)
individuals
Ex. Fitness values of chromosomes are: P(a) = (4/10)*100 = 40%
f ( a )  4; P(b) = (3/10)*100 = 30%
f ( b )  3;
P(c) = (2/10)*100 = 20%
f (c)  2;
P(d) = (1/10)*100 = 10%
f ( d )  1;
12
 Crossover
Crossover is a genetic operator that combines (mates) two individuals
(parents) to produce two new individuals (Childs).
• The idea behind crossover is that the new chromosome may be
better than both of the parents if it takes the best characteristics from
each of the parents.
Single-Point Crossover
•Choose a random point
•Split parents at this crossover point
•Create children by exchanging tails
•Probability of crossover is typically in range (0.6, 0.9)

Parents Children
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1

13
Two-Point Crossover
•Choose two random points
•Split parents at these crossover points
•Create children by exchanging middle
portion
Parents Children
1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 1 1 1

0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0

Uniform Crossover
•Randomly generated mask
1 1 0 1 0 0 0 1 0 1

•Create children by exchanging genes according to

mask
1 1 1 1 Parents
1 1 1 1 1 1 0 0 1 0 1 1 1 0 1 0

0 0 0 0 Children
0 0 0 0 0 0 1 1 0 1 0 0 0 1 0 1
14
Real value Crossover
•Choose two parents randomly from the selected candidates.
•Generate a random number (r) for each dimension of parents.
•Create children by combining the weighted portion of both
parents.
  1 

 2 * r 
 
1

 if r 
b    1 
  0.5
 
  1   1
 1 r 
   if r  0.5

2* 1
child 1 ( j) 
2
1  b * parent
1 ( j)  1  b * parent
2

( j) 
1
child 2 ( j)  1  b * parent
1 ( j)  1  b * parent
2
2

( j)where
• r is a random number {0,1}
•  is a crossover operator
• child ( j) represent dimension of child.
jth 25
 Mutation
 Mutation is a genetic operator used to maintain genetic diversity
from one generation of a population of chromosomes to the next.
It is analogous to biological mutation.
 Mutation Probability determines how often the parts of a
chromosome will be mutated.

Bit-reversal Mutation
•Alter each gene independently
•Choose a random point
•Create children by altering the gene of choosen point
•Mutation probability is typically in range
{(1/population_size), (1/chromosome_length)}

(0 – 1)
Parents Children
1 1 0 1 1 0 0 1 0 1 0 0 1 0 0 1 1 0 1 0

(1 – 0)
26
Polynomial Mutation
•Select one parent randomly from the selected parents.
•Generate a random number (r) for each dimension of
parent.

  1 
 
d  2* r
  1 if r
1
 1  0.5

1 2* 1 r  
  1 

 if r  0.5
child ( j)  parent ( j) 
d
• r is a random number {0,1}
where
•  is a mutation operator
• child ( j) represent dimension of child.
jth

33
 Fitness Function
 A fitness function quantifies the optimality of a solution
(chromosome) so that that particular solution may be
ranked against all the other solutions.
 It depicts the closeness of a given ‘solution’ to the
desired result.
 Watch out for its speed.
 Most functions are stochastic and designed so that a
small proportion of less fit solutions are selected. This
helps keep the diversity of the population large,
preventing premature convergence on poor solutions.

35
 Terminatio
n
 A solution is found that satisfies minimum criteria

 Fixed number of generations found

 Allocated budget (computation, time/money) reached

 The highest ranking solution’s fitness is reaching

or has reached

36
Issues with Genetic Algorithms

Choosing parameters:
•Population size
•Crossover and mutation probabilities
•Selection, deletion policies
•Crossover, mutation operators, etc.
•Termination criteria

Performance:
•Can be too slow but covers a large search
space
•Is only as good as the fitness function

38
 The Basic Genetic Algorithm
{ % Generate random population of chromosomes
Initialize population;
% Evaluate the fitness of each chromosome in the population
Evaluate population; [Fitness]
% Create, accept and test a new population:
while Termination_Criteria_Not_Satisfied
{ % Select according to fitness
Select parents for [Selection]
% With a crossover probability perform crossover or copy parents
reproduction;
Perform crossover; [Crossover]
% With a mutation probability mutate offspring at each position in chromosome
Perform mutation; [Mutation]
Accept new generation;
Evaluate population; [Fitness]
}
}

39
Minimize  f (x) 
sin(x),
where 0x
(2* ) x
2.620
Initialize population : 4.526
0.001
• Search space is of single dimension (given).
1.900
• Let population size (N) = 10 0.922
0.580
• Initialize population with 10 1.170
chromosomes
2.171
having single dimensioned real value.
2.493
• ] .
These [0,2 * values
between are randomly distributed 3.385
in chromosomes

40
Evaluate function:
• Find out objective function values (fitness values)
for all chromosomes.
x f
2.620 (x)
0.498
4.526 -0.983
0.001 0.001
1.900 0.946
0.922 0.797
0.580 0.548
1.170 0.921
2.171 0.825
2.493 0.604
3.385 -0.241
chromosomes fitness

41
 Tournament Selection:

x f
Run this loop N ( 10) times
p
f
2.620 (x)
0.498 2 random
x p
1.170
4.526 -0.983
positions
(x) 4.526
4 1.900
0.001 0.001 0.946 1.170 2.620
7 1.170
1.900 0.946 0.921 0.580
0.922 0.797 4.526
0.580 0.548 if  f 1x  f2 3.385
1.170 0.921 select
x   x , 1
0.001
2.171 0.825 else 2.620
2.493 0.604 4.526
select  x2 ,
3.385 -0.241 2.620
chromosomes fitness
parents

42
 Crossover:
Select Probability of Crossover Pc  0.8
Distribution index for Crossover (µ)= 20

Run this loop Pc * N (0.8*10  8)

p cc
2 random
positions p times
cc
1.170
2.611
10 2.620 2.611
4.526
3.394
6 3.385 3.394
2.620
0.330
0.580 Generate random number – (r) 4.196
4.526   1 
2 * r 
 

if r  2.625
 
3.385 1  1 
0.5
b   
 
 4.521

 1   1
0.001 1 r  1.146
   if r  0.5
2.620 
2*
1 1 b * p  1  b *
cc1 2  
 
4.526 1

2.620
 1 p2 
parents cc 2  1  b * 1p  1  b 2* 4.469
2
p 
children
7/29/2013 43
INDIAN INSTITUTE OF TECHNOLOGY BHUBANESWAR
Mutation:
Select Probability of Mutation\ Pm  0.2
Distribution index for Mutation (η) = 20

cc Solutions generated after Cross over:16= N c

Run this loop Pm * N c *V (0.2*16*1  4) times
2.611 N c:

position cc cm cm
3.394
1 random 2.594
5 2.625 2.594
0.330
2.751
4.196
2.200
2.625 Generate random number – (r) 0.010
4.521
  1  Mutated

d  2*r1 1
1.146  children
  if r 
 1 
0.5
1 2*1  
  1
r if
4.469 cm  cc 
children
d r  0.5
44
 Selection of N chromosomes for next generation:
Combine chromosomes and children: Select first N (=10)
Intermediate population is chromosomes for next
f (R) R generation
 x   -0.985 4.539 x
R  cc  -0.983 4.526 4.539
-0.982 4.521 4.526
 m
c -0.971 4.470 4.521
-0.912 4.291 4.470
Evaluate function -0.870 4.196 4.291
 -0.418 3.572 4.196
f (x) 
  -0.250 3.394 3.572
f (R)   f Sort in -0.241 3.385 3.394
 f m  ascending
3.385
(cc )  order according
to fitness value   3.262
(c )
chromosomes
0.946 1.899
45
 Initial State After 2 generation

After 5 generation After 10 generation

46
Applications of Genetic Algorithms
•Optimization–numerical and combinatorial optimization problems,
e.g. travelling salesman, routing, graph colouring and partitioning
•Robotics–trajectory planning
•Machine learning–designing neural networks, classification and
prediction,
e.g. prediction of weather or protein structure
•Signal processing–filter design
•Design–semiconductor layout, aircraft design, communication networks
•Automatic programming–evolve computer programs for specific
tasks, design cellular automata and sorting networks
•Economics–development of bidding strategies, emergence of
economics
markets
•Immune systems–model somatic mutations
•Ecology–model symbiosis, resource flow 47