An Exordium of Genetic

Algorithms (GAs)

Mithun.k

Mtech – Technology Management

1
 Overview

History of Genetic Algorithm

Introduction to Genetic Algorithms
GA operators and parameters
Examples for the optimization using GA
Classification of GA
Conclusion

2
Unveiling the History of

Genetic Algorithm
 4

History Of Genetic Algorithms

 Four historical paradigms that have served as the basis for
Evolutionary Computing are

1) evolutionary programming (Forgel 1966)

2) evolutionary strategies (Recheuberg 1973)
3) genetic algorithms (Holland 1975)
4) genetic programming (John Koza 1992)

 differs in the nature of the

o representation schemes,
o the reproduction operators
o and selection methods.
 5

History continue…

1)Genetic Algorithm
 Representation - fixed-length bit string

0111011011011
 the string is “evaluated as a collection of structural features
of a solution that have little or no interactions”.
||
“Gene represents an entity that is structurally independent
of other genes”.
 6

History continue…

 Position in string - feature of an individual

Value stored in position - how feature is expressed?

 Reproduction :

* Cross over
* Mutation
* Inversion
 7

History continue…

2)Genetic Programming
 Representation - variable-sized tree of functions and values
 entire tree corresponds to a single function that may be
evaluated.
 Reproduction :
* Cross over
* Mutation
* Inversion
 8

History continue…

3)Evolutionary Strategies
Representation - fixed-length real-valued vector.

Reproduction
o Gaussian Mutation
o Intermediate Recombination
 9

History continue…

4)Evolutionary Programming
 Representing individuals’ phenotypically.
 Applied in finite state machines capable of responding to
environmental stimuli and developing operators for
effecting structural and behavioral change over time.
 any type of computing tool could be evolved in some way
using the GA principles
 Reproduction – Mutation
 No exchange of materials between individuals in population
 Why Genetic Algorithm…?
 Conventional methods selecting only one individual at a
time which make them to be in a danger of trapped in a
local minima or maxima.

10
 What Are Genetic Algorithms (GAs)?

Genetic Algorithms are

search and optimization techniques
based on Darwin’s Principle of
Natural Selection.

11
 Darwin’s Principle Of Natural Selection
 IF there are organisms that reproduce, and
 IF offspring's inherit traits from their progenitors, and
 IF there is variability of traits, and
 IF the environment cannot support all members of a growing
population,
 THEN those members of the population with less-adaptive
traits will die out & those members
with more-adaptive traits will survive.
 The result is the evolution of species.

12
 An Example of Natural Selection
 Giraffes have long necks.

Giraffes with slightly longer necks could feed on leaves of higher branches
when all lower ones had been eaten off.

 They had a better chance of survival.

 Favorable characteristic propagated
through generations of giraffes.
 Now, evolved species has long necks.

NOTE: Longer necks may have been a deviant characteristic (mutation)

initially but since it was favorable, was propagated over generations. Now
an established trait.

13
 An Example of Natural Selection continue…

14
 Evolution Through Natural Selection

Initial Population Of Animals

Struggle For Existence-Survival Of the Fittest

Surviving Individuals Reproduce, Propagate Favorable

Characteristics

Millions Of Years

Evolved Species
(Favorable Characteristic Now A Trait Of Species)

15
 Nature to Computer Mapping

Nature Computer

Population Set of solutions.

Individual Solution to a problem.
Fitness Quality of a solution.
Chromosome Encoding for a Solution.
Gene Part of the encoding of a solution.
Reproduction Crossover

16
 Flowchart of Genetic Algorithm

17
 Flowchart Explanation
Step 1: Create a Random Initial Population
An initial population is created from a random selection of solutions
These solutions have been seen as represented by chromosomes as in
living organisms
A chromosome is a packet of genetic information organized in a
standard way that defines completely and individual (solution).
The genetic structure (way in which that information is packed and
defined) enables the solutions to be manipulated.
The genetic operands (way in which that information can be
manipulated) enables the solutions to reproduces and evolve

18
 Flowchart Explanation continue…

Step 2:Evaluate Fitness

 A value for fitness is assigned to each solution (chromosome) depending

on how close it actually is to solving the problem.

 Therefore we need to define the problem, model it, simulate it or have a

data set as sample answers

 Each possible solution has to be tested in the problem & the answer

evaluated (or marked) on how good it is

 The overall mark of each solution relative to all the marks of all solutions

produces a fitness ranking.

19
 Flowchart Explanation continue…

Step 3:Produce Next Generation

Those chromosomes with a higher fitness value are more likely to

reproduce offspring
The population for the next Generation will be produced using the
genetic operators
Reproduction by Copy or Crossing Over and Mutation will be
applied to the chromosomes according to the selection rule
This rule states that the fitter and individual is, the higher the
probability it has to reproduce
Note that this works with probabilities!

20
 Flowchart Explanation continue…

Step 4:Next Generation or Termination

If the population in the last generation contains a solution that produces an
output that is close enough or equal to the desired answer then the problem
has been solved.
This is the ideal termination criterion of the evolution
If this is not the case, then the new generation will go through the same
process as their parents did, and the evolution will continue
A termination criterion that always must be included is Time-Out
 Since one drawback of Evolutionary Programming is that is very difficult
(impossible most of the time) to know if the ideal termination criterion is
going to be satisfied, or when.

21
 GA Operators and Terminologies

22
 23

GA Operators and Terminologies

1) Encoding
2) Fitness function
3) Selection
4) Recombination
 Crossover
 Mutation
 1) Encoding
The process of representing the solution in the form of
a string that conveys the necessary information.

 Just as in a chromosome, each gene controls a particular

characteristic of the individual, similarly, each bit in the string
represents a characteristic of the solution.

 Binary Encoding
 Octal Encoding
 Hexadecimal Encoding
 Permutation Encoding
 Value Encoding

24
24
 Encoding Methods

Binary Encoding – Most common method of encoding.

Chromosomes are strings of 1s and 0s and each position in the
chromosome represents a particular characteristic of the
problem.

Chromosome A 10110010110011100101
Chromosome B 11111110000000011111

25
 Encoding Methods continue…

Permutation Encoding – Useful in ordering problems

such as the Traveling Salesman Problem (TSP). Example. In
TSP, every chromosome is a string of numbers, each of which
represents a city to be visited.

Chromosome A 1  5  3  2  6  4  7  9  8
Chromosome B 8  5  6  7  2  3  1  4  9

26
 Encoding Methods continue…

Value Encoding – Used in problems where complicated

values, such as real numbers, are used and where binary
encoding would not suffice.

Good for some problems, but often necessary to develop some

specific crossover and mutation techniques for these
chromosomes.

Chromosome A 1.235  5.323  0.454  2.321  2.454

Chromosome B (left), (back), (left), (right), (forward)

27
 2) Fitness Function

A fitness function quantifies the optimality of a

solution (chromosome) so that that particular solution
may be ranked against all the other solutions.
 A fitness value is assigned to each solution depending on how
close it actually is to solving the problem.

 Ideal fitness function correlates closely to goal + quickly

computable.

 Example. In TSP, f(x) is sum of distances between the cities in

solution. The lesser the value, the fitter the solution is.

28
 3)Selection

Selection is the process of choosing one or two

parents from the population for reproduction.

 The higher the fitness value, the more chance an individual has
to be selected ,which will increase the convergence rate of GA.

 Basic Types
 a)Proportionate based :- based on fitness criteria
 b)Ordinal based :- based on ranking within the population

29
 Selection methods

a) Roulette Wheel Selection

b) Rank Selection
c) Tournament Selection
d) Elitism

30
 a)Roulette Wheel Selection 31

 Each current string in the population has a slot assigned to it

which is in proportion to it’s fitness.

 We spin the weighted roulette wheel thus defined n times

(where n is the total number of solutions).

 Each time Roulette Wheel stops, the string corresponding to

that slot is created.

Strings that are fitter are assigned a larger slot and hence have
a better chance of appearing in the new population.

31
 Example Of Roulette Wheel Selection

No. String Fitness % Of Total

1 01101 169 14.4

2 11000 576 49.2

3 01000 64 5.5

10011 361 30.9

4

Total 1170 100.0

32
 Roulette Wheel For Example

33
 b)Rank Selection
If the best chromosome fitness is 90%, its
circumference occupies 90% of Roulette wheel, and
then other chromosomes have too few chances to be
selected.

Ranks the population and every chromosome

receives fitness from the ranking.
The worst has fitness 1 and the best has fitness N.
It preserves diversity and results in slow
convergence.
34
 c)Tournament Selection
The Tournament selection strategy provides
selective pressure by holding a tournament
competition among N individuals.

The best individual from the tournament is the one

with the highest fitness, which is the winner of
tournament.

Tournament competitions and the winner are then

inserted into the mating pool.
35
 Tournament Selection (example)
Mating Pool
7993
6872
4 1 3 2 5 6
6872
4 3 2 1 5 6
8971
4 3 2 1 5 6 8673 3 6 4 1 2 5
5 2 6 4 3 1 8673
5 2 6 4 3 1
8971
3 6 4 1 2 5 7993

7993 4 1 3 2 5 6
4 1 3 2 5 6 8142
8479
6 3 4 5 2 1
2 6 3 4 5 1
8142
6872 2 6 3 4 5 1
4 3 2 1 5 6 6872
8479 4 3 2 1 5 6 8673
6 3 4 5 2 1
8142 5 2 6 4 3 1
2 6 3 4 5 1 8142
2 6 3 4 5 1

36
 d) Elitism
Elitism is a method which copies the best
chromosome to the new offspring population
before crossover and mutation.

 When creating a new population by crossover or mutation

the best chromosome might be lost.

 Forces GAs to retain some number of the best individuals

at each generation.

 Has been found that elitism significantly improves

performance.

37
 4)Recombination

It is the process for getting new offspring for the

mating pool.

Types
a) Crossover
b) Mutation

38
 a) Crossover - Recombination

It is the process in which two chromosomes (strings)

combine their genetic material (bits) to produce a
new offspring which possesses both their
characteristics.

 Two strings are picked from the mating pool at random to cross
over.

 The method chosen depends on the Encoding Method.

39
 Crossover Methods - Recombination

i) Single Point Crossover

ii) Two Point Crossover
iii) Uniform Crossover

40
 Crossover methods- Recombination

1)Single Point Crossover

Chromosome1 11011 | 00100110110

Chromosome 2 11011 | 11000011110

Offspring 1 11011 | 11000011110

Offspring 2 11011 | 00100110110

41
 Crossover methods- Recombination

2)Two-Point Crossover- Two random points are chosen on

the individual chromosomes (strings) and the genetic
material is exchanged at these points.

Chromosome1 11011 | 00100 | 110110

Chromosome 2 10101 | 11000 | 011110

Offspring 1 10101 | 00100 | 011110

Offspring 2 11011 | 11000 | 110110

NOTE: These chromosomes are different from the last example.

42
 Crossover methods- Recombination

3)Uniform Crossover- Each gene (bit) is selected randomly

from one of the corresponding genes of the parent
chromosomes.

Chromosome1 11011 | 00100 | 110110

Chromosome 2 10101 | 11000 | 011110

Offspring 10111 | 00000 | 110110

NOTE: Uniform Crossover yields ONLY 1 offspring.

43
 b) Mutation - Recombination

It is the process by which a string is deliberately

changed so as to maintain diversity in the population
set.

Mutation Probability - determines how often the parts of

a chromosome will be mutated.
Types :-
 flipping
 interchanging
 reversing

44
 b)Mutation - Recombination continue…
Flipping :-

Interchanging : -

Reversing : -

45
 Example 1

Genetic Algorithms To Solve The

Maximum of Function

46
 GA to solve max< > continue…

47
 GA to solve max< > continue…

Here I’m using Roulette Wheel Selection.

48
 GA to solve max< > continue…

The actual count of string no 1 is 1, hence it occurs once in the mating pool. The
actual count of string no 2 is 2, hence it occurs twice in the mating pool. Since the
actual count of string no 3 is 0, it does not occur in the mating pool. Similarly, the
actual count of string no 4 being 1, it occurs once in the mating pool.

crossover operation is performed.

49
 GA to solve max< >
continue…

Mutation is performed

population average fitness has improved from 288.75 to 636.5 in 1

generation

50
 Example 2

Genetic Algorithms To Solve The Traveling

Salesman Problem (TSP)

51
 The Problem

The Traveling Salesman Problem is defined as:

‘We are given a set of cities and a symmetric distance matrix
that indicates the cost of travel from each city to every other
city.

The goal is to find the shortest circular tour, visiting every city
exactly once, so as to minimize the total travel cost, which
includes the cost of traveling from the last city back to the first
city’.

52 Traveling Salesman Problem

 TSP Encoding

 Each city is represented with an integer .

 Consider 6 Indian cities –

Mumbai, Cochin, Calcutta, Delhi , Bangalore and Chennai
and assign a number to each.

Mumbai 1
Cochin 2
Calcutta 3
Delhi 4
Bangalore 5
Chennai 6

53
 TSP Encoding continue…

 Thus a path would be represented as a sequence of integers

from 1 to 6.

 The path [1 2 3 4 5 6 ] represents a path from Mumbai to

Cochin, Cochin to Calcutta, Calcutta to Delhi, Delhi to
Bangalore, Bangalore to Chennai, and finally from Chennai to
Mumbai.

 This is an example of Permutation Encoding as the position

of the elements determines the fitness of the solution.

54
 Fitness Function
 The fitness function will be the total cost of the tour
represented by each chromosome.

 This can be calculated as the sum of the distances traversed in

each travel segment.

The Lesser The Sum, The Fitter The Solution

Represented By That Chromosome.

55
 Distance/Cost Matrix For TSP
1 2 3 4 5 6

1 0 863 1987 1407 998 1369

2 863 0 1124 1012 1049 1083

3 1987 1124 0 1461 1881 1676

4 1407 1012 1461 0 2061 2095

5 998 1049 1881 2061 0 331

6 1369 1083 1676 2095 331 0

Cost matrix for six city example. Distances in

Kilometers

56
 Fitness Function continue…

 So, for a chromosome [4 1 3 2 5 6], the total cost of travel or

fitness will be calculated as shown below

 Fitness = 1407 + 1987 + 1124 + 1049 + 331+ 2095

= 7993 kms.

 Since our objective is to Minimize the distance, the lesser the

total distance, the fitter the solution.

57
 Selection Operator

 Used Tournament Selection.

 As the name suggests tournaments are played between two

solutions and the better solution is chosen and placed in the
mating pool.

 Two other solutions are picked again and another slot in the
mating pool is filled up with the better solution.

58
 Tournament Selection continue…

Mating Pool
7993
6872
4 1 3 2 5 6
6872
4 3 2 1 5 6
8971
4 3 2 1 5 6 8673 3 6 4 1 2 5
5 2 6 4 3 1 8673
5 2 6 4 3 1
8971
3 6 4 1 2 5 7993

7993 4 1 3 2 5 6
4 1 3 2 5 6 8142
8479
6 3 4 5 2 1
2 6 3 4 5 1
8142
6872 2 6 3 4 5 1
4 3 2 1 5 6 6872
8479 4 3 2 1 5 6 8673
6 3 4 5 2 1
8142 5 2 6 4 3 1
2 6 3 4 5 1 8142
2 6 3 4 5 1

59
 Why we cannot use single-point crossover ?

 Single point crossover method randomly selects a crossover point

in the string and swaps the substrings.
 This may produce some invalid offsprings as shown below.

4 1 3 2 5 6 4 1 3 1 5 6

4 3 2 1 5 6 4 3 2 2 5 6

60
 Crossover Operator

 Used the Enhanced Edge Recombination operator

 This operator is different from other genetic sequencing

operators in that it emphasizes adjacency information instead
of the order or position of items in the sequence.

 The algorithm for the Edge-Recombination Operator involves

constructing an Edge Table first.

61
 Edge Table

The Edge Table is an adjacency table that lists links into and out
of a city found in the two parent sequences.

If an item is already in the edge table and we are trying to insert

it again, that element of a sequence must be a common edge and
is represented by inverting it's sign.

62
 Finding The Edge Table
Parent 1 4 1 3 2 5 6
Parent 2 4 3 2 1 5 6

1 4 3 2 5

2 -3 5 1  

3 1 -2 4  

4 -6 1 3

5 1 2 -6

6 -5 -4

63
 Enhanced Edge Recombination Algorithm

1. Choose the initial city from one of the two parent tours.

2. Remove all occurrences of the current city from the left hand side of the
edge table.( These can be found by referring to the edge-list for the current
city).

3. If the current city has entries in it's edge-list, go to step 4 otherwise go to

step 5.

4. Determine which of the cities in the edge-list of the current city has the
fewest entries in it's own edge-list. The city with fewest entries becomes the
current city. In case a negative integer is present, it is given preference. Ties
are broken randomly. Go to step 2.

5. If there are no remaining unvisited cities, then stop. Otherwise, randomly

choose an unvisited city and go to step 2.
64
 Enhanced Edge Recombination Algorithm continue…

Step 1 Step 2

1 4 3 2 5
1 3 2 5

2 -3 5 1
2 -3 5 1

3 1 -2 4  
3 1 -2

4 -6 1 3
4 -6 1 3

5 3 2 -6  
5 3 2 -6

6 -5 -4
6 -5

4 4 6

65
 Enhanced Edge Recombination Algorithm
continue…

Step 3 Step 4

1 3 2 5 1 3 2

2 -3 5 1  
2 -3 1

3 1 -2  
3 1 -2

4 1 3  
4 1 3

5 3 2  
5 3 2

   
6 -5 6

4 6 5 4 6 5 1

66
 Enhanced Edge Recombination Algorithm
continue…

Step 5 Step 6

1 3 2 1 2

 
2 -3 2

  3 -2
3 -2

4 3 4

5 3 2 5 2

 
6 6

4 6 5 1 3 2
4 6 5 1 3

67
 Mutation Operator
 The mutation operator induces a change in the solution, so as
to maintain diversity in the population and prevent Premature
Convergence.
 In our project, we mutate the string by randomly selecting any
two cities and interchanging their positions in the solution, thus
giving rise to a new tour.

4 1 3 2 5 6

4 5 3 2 1 6

68
 Classification of Genetic
Algorithm

69
 Classification of Genetic Algorithm

1) Simple Genetic Algorithm (SGA)

2) Parallel and Distributed Genetic Algorithm (PGA and
DGA)
a) Master-Slave Parallelization
b) Fine Grained Parallel GAs (Cellular GAs)
c) Multiple-Deme Parallel GAs (Distributed GAs or Coarse Grained
GAs)
d) Hierarchical Parallel Algorithms
3) Adaptive Genetic Algorithm (AGA)
4) Fast Messy Genetic Algorithm (FmGA)
5) Independent Sampling Genetic Algorithm (ISGA)

70
 1) Simple Genetic Algorithm (SGA)

 A simple genetic algorithm composed of three operations

 Reproduction
 Crossover
 Mutation
 SGAs are useful and efficient when,
 The search space is large, complex or poorly understood.
 Domain knowledge is scarce or expert knowledge is difficult to
encode to narrow the search space.
 No mathematical analysis is available.
 Traditional search methods fail.

71
 2)Parallel and Distributed Genetic Algorithm (PGA and DGA)

 Parallel execution of various SGAs is called PGA.

 Each of the Processing Elements starts with a different
initial subpopulation, evolves and stops independently.
 There are no inter-PE communications.
 Methods of PGA are
 Independent PGA
 Migration PGA
 Partition PGA

72
 2)Parallel and Distributed Genetic Algorithm (PGA and DGA) continue…

Master-Slave Parallelization:-

 The master stores the population and does the selection

 The slaves evaluate the fitness and apply the genetic operators like crossover and
mutation.
 Communication occurs only when slaves return values to master.
 Modes
o Synchronous - master waits until it receives the fitness value for entire population.
o Asynchronous - master doesn’t stop for any slow processors.

73
 2)Parallel and Distributed Genetic Algorithm (PGA and DGA) continue…

Coarse & Fine grained PGA’s :-

 population is divided into a number of subpopulations called
demes.
 Demes are separated from one another and individuals compete
only within a deme.
 Coarse grained - small number of demes with many individuals
 Fine Grained - large number of demes with small number of individuals

 migration is used to move the individuals from one deme to

another (island model).

74
 2)Parallel and Distributed Genetic Algorithm (PGA and DGA) continue…

Hierarchical Parallel Algorithms : -

 Combination of Master-Slave with fine and coarse grained PGA

75
 3)Adaptive Genetic Algorithm (AGA)

GAs whose parameters, such as the population size, the

crossing over probability, or the mutation probability are
varied while the GA is running .

 The mutation rate is changed according to changes in the

population; the longer the population does not improve, the
higher the mutation rate is chosen.

 It is decreased again as soon as an improvement of the

population occurs.

76
 Advantages of GA
 Parallelism
 Solution space is wider.
 Easy to discover global optimum
 Only uses function evaluations.
 Handles noisy functions well.
 Handles large, poorly understood search spaces easily
 They require no knowledge or gradient information about the
response surface
 Discontinuities present on the response surface have little
effect on overall optimization performance
 They are resistant to becoming trapped in local optima

77
 Limitations
 The problem of identifying fitness function
 The problem of choosing the various parameters like the size
of the population, mutation rate, cross over rate, the selection
method and its strength.
 Cannot easily incorporate problem specific information
 Not good at identifying local optima
 No effective terminator.
 Not effective for smooth unimodal functions
 Needs to be coupled with a local search technique.
 Require large number of response (fitness) function evaluations

78
 Applications
Time series forecasting.
Robot trajectory planning
Image Processing
Design–semiconductor layout, aircraft design, keyboard
configuration, communication networks
Scheduling–manufacturing, facility scheduling, resource allocation
Machine Learning–Designing neural networks, both architecture
and weights, improving classification algorithms, classifier systems
Signal Processing–filter design
Combinatorial Optimization–set covering, traveling salesman
(TSP), Sequence scheduling, routing, bin packing, graph coloring
and partitioning

79
 Conclusion

80
 References

 Genetic Algorithm In Search, Optimization & Machine Learning

- D. E. Goldberg
 Handbook of Genetic Algorithm
- Lawrence Davis
 Introduction To Genetic Algorithms
- S.N.Sivanandam,
S.N.Deepa
 www.geneticalgorithms.ai-depot.com/Tutorial/Overview.html
 www.obitko.com/tutorials/genetic-algorithms/
 www.geneticprogramming.com
 en.wikipedia.org/wiki/Genetic_algorithm

81
Thank You