Introduction to Genetic Algorithm

Debasis Samanta

Indian Institute of Technology Kharagpur

[email protected]

26.02.2016

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 1 / 26

Limitations of the traditional optimization
approaches

Limitations:

Computationally expensive.
For a discontinuous objective function, methods may fail.
Method may not be suitable for parallel computing.
Discrete (integer) variables are difficult to handle.
Methods may not necessarily adaptive.

Evolutionary algorithms have been evolved to address the above

mentioned limitations of solving optimization problems with traditional
approaches.

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 2 / 26

Evolutionary Algorithms

The algorithms, which follow some biological and physical behaviors:

Biologic behaviors:
Genetics and Evolution –> Genetic Algorithms (GA) Behavior
of ant colony –> Ant Colony Optimization (ACO) Human
nervous system –> Artificial Neural Network (ANN)

In addition to that there are some algorithms inspired by some physical

behaviors:
Physical behaviors:
Annealing process –> Simulated Annealing (SA)
Swarming of particle –> Particle Swarming Optimization (PSO)
Learning –> Fuzzy Logic (FL)

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 3 / 26

Genetic Algorithm

It is a subset of evolutionary algorithm:

Ant Colony optimization

Swarm Particle Optimization

Models biological processes:

Genetics
Evolution
To optimize
highly
complex
objective
functions:
Very
difficult to
model
mathema
Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 4 / 26
Background of Genetic Algorithm

Firs time itriduced by Ptrof. John Holland (of Michigan University, USA,
1965).
But, the first article on GA was published in 1975.

Principles of GA based on two fundamental biological processes:

Genetics: Gregor Johan Mendel (1865)
Evolution: Charles Darwin (1875)

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 5 / 26

A brief account on genetics
The basic building blocks in living bodies are cells. Each cell carries the
basic unit of heredity, called gene

Nucleus

Chromosome

Other cell bodies

For a particular specie, number of chromosomes is fixed.

Examples
Mosquito: 6
Frogs: 26
Human: 46
Goldfish: 94
etc.
Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 6 / 26
A brief account on genetics
Genetic code

Spiral helix of protein substance is called DNA.

For a specie, DNA code is unique, that is, vary uniquely from one to
other.
DNA code (inherits some characteristics from one generation to
next generation) is used as biometric trait.

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 7 / 26

A brief account on genetics

Reproduction

+ = Organism’s cell :
Cell division
x y
gamete diploid

(Reproductive cell has Each chromosome from

haploid half the number of diploid both haploids are combined
chromosomes) to have full numbers

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 8 / 26

A brief account on genetics
Crossing over

Kinetochore

Information from
two different Combined into so that diversity
organism’s body in information is possible
cells
Random crossover points
makes infinite diversities

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 9 / 26

A brief account on evolution
Evolution : Natural Selection

Four primary premises:

Information propagation: An offspring has many of its

characteristics of its parents (i.e. information passes from parent to
its offspring). [Heredity]

Population diversity: Variation in characteristics in the next

generation. [Diversity]

Survival for exitence: Only a small percentage of the offspring

produced survive to adulthood. [Selection]

Survival of the best: Offspring survived depends on their

inherited characteristics. [Ranking]

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 10 / 26

A brief account on evolution

Mutation:

To make the process forcefully dynamic when variations in population

going to stable.

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 11 / 26

Biological process : A quick overview

Genetics

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 12 / 26

Working of Genetic Algorithm

Definition of GA:
Genetic algorithm is a population-based probabilistic search and
optimization techniques, which works based on the mechanisms of
natural genetics and natural evaluation.

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 13 / 26

Framework of GA

Start

Note:
An individual in the
population is
corresponding to a
Initial Population possible solution

No
Converge ? Selection

Yes

Reproduction
Stop

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 14 / 26

Working of Genetic Algorithm

Note:

GA is an iterative process. It
is a searching technique.
Working cycle with / without
convergence.
Solution is not necessarily guranteed. Usually, terminated with a
local optima.

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 15 / 26

Framework of GA: A detail view

Start

Define parameters

Parameter representation

Create population
Initialize population
Apply cost
function to each of
the population

No
Converge ? Evaluate the fitness

Selectio
Yes

n
Select Mate
Stop

Crossover

Reproducti
Mutation

on
Inversion

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 16 / 26

Optimization problem solving with GA

For the optimization problem, identify the following:

Objective function(s)

Constraint(s)

Input parameters

Fitness evaluation (it may be algorithm or mathematical formula)

Encoding

Decoding

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 17 / 26

GA Operators

In fact, a GA implementation involved with the realization of the

following operations.

Encoding: How to represent a solution to fit with GA

framework.

Convergence: How to decide the termination criterion.

Mating pool: How to generate next solutions.

Fitness Evaluation: How to evaluate a solution. Crossover:

How to make the diverse set of next solutions. Mutation: To

explore other solution(s).

Inversion: To move from one optima to other.

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 18 / 26

Different GA Strategies

Simple Genetic Algorithm (SGA) Steady

State Genetic Algorithm (SSGA) Messy

Genetic Algorithm (MGA)

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 19 / 26

Simple GA

Start

Create Initial population

of size N

Evaluate each individuals

Convergence
Yes Return the individual(s) with
Criteria meet ?
best fitness value

No

Select Np individuals
(with Stop
repetition)

Create mating pool (randomly) (Pair of

parent for generating new offspring)

Perform crossover and

create new offsprings Reproduction

Mutate the offspring

Perform inversion on
the offspring

Replace all individuals in the last generation

with new offsprings created

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 20 / 26

Important parameters involved in Simple GA

SGA Parameters

Initial population size : N

Size of mating pool, Np : Np = p%ofN

Convergence threshold δ

Mutation µ

Inversion η

Crossover ρ

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 21 / 26

Salient features in SGA

Simple GA features:

Have overlapping generation (Only fraction of individuals are

replaced).

Computationally expensive.

Good when initial population size is large.

In general, gives better results.

Selection is biased toward more highly fit individuals; Hence, the

average fitness (of overall population) is expected to increase in
succession.
The best individual may appear in any iteration.

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 22 / 26

Steady State Genetic Algorithm (SSGA)
Start

Generate Initial population of size N

Evaluate each individuals

Select two individual without

repetition

Crossover Mutation

Inversion

Yes Reject the

offspring
if
duplicated

No

Evaluate the offspring

If the offspring are better than the

worst individuals then replace the
worst individuals with the offspring

Convergence
meet ?

Return the solutions

Stop

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 23 / 26

Salient features in Steady-state GA

SGA Features:

Generation gap is small.

Only two offspring are produced in one generation.

It is applicable when

Population size is small

Chromosomes are of longer length
Evaluation operation is less computationally expensive (compare to
duplicate checking)

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 24 / 26

Salient features in Steady-state GA

Limitations in SSGA:

There is a chance of stuck at local optima, if

crossover/mutation/inversion is not strong enough to diversify the
population).

Premature convergence may result.

It is susceptible to stagnation. Inferiors are neglected or removed

and keeps making more trials for very long period of time without
any gain (i.e. long period of localized search).

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 25 / 26

 Issues for GA Practitioners

Choosing basic implementation issues:
¨ representation
¨ population size, mutation rate, ...
¨ selection, deletion policies
¨ crossover, mutation operators

Termination Criteria

Performance, scalability

Solution is only as good as the evaluation
function (often hardest part)

Wendy Williams 26 Genetic Algorithms: A Tutorial

Metaheuristic Algorithms
Benefits of Genetic Algorithms

Concept is easy to understand

Modular, separate from application

Supports multi-objective optimization

Good for “noisy” environments

Always an answer; answer gets better
with time

Inherently parallel; easily distributed

Wendy Williams 27 Genetic Algorithms: A Tutorial

Metaheuristic Algorithms
Benefits of Genetic Algorithms (cont.)

Many ways to speed up and improve a
GA-based application as knowledge
about problem domain is gained

Easy to exploit previous or alternate
solutions

Flexible building blocks for hybrid
applications

Substantial history and range of use
Wendy Williams 28 Genetic Algorithms: A Tutorial
Metaheuristic Algorithms
 When to Use a GA

Alternate solutions are too slow or overly
complicated

Need an exploratory tool to examine new
approaches

Problem is similar to one that has already been
successfully solved by using a GA

Want to hybridize with an existing solution

Benefits of the GA technology meet key problem
requirements

Wendy Williams 29 Genetic Algorithms: A Tutorial

Metaheuristic Algorithms
 Some GA Application Types
Domain Application Types
Control gas pipeline, pole balancing, missile evasion, pursuit

Design semiconductor layout, aircraft design, keyboard

configuration, communication networks
Scheduling manufacturing, facility scheduling, resource allocation

Robotics trajectory planning

Machine Learning designing neural networks, improving classification

algorithms, classifier systems
Signal Processing filter design

Game Playing poker, checkers, prisoner’s dilemma

Combinatorial set covering, travelling salesman, routing, bin packing,

graph colouring and partitioning
Optimization

Wendy Williams 30 Genetic Algorithms: A Tutorial

Metaheuristic Algorithms
Wendy Williams 31 Genetic Algorithms: A Tutorial
Metaheuristic Algorithms