Module VI: Genetic Algorithm (GA)

Genetic Algorithm (GA) is a search-based optimization technique based on the principles of Genetics
and Natural Selection. It is frequently used to find optimal or near-optimal solutions to difficult
problems which otherwise would take a lifetime to solve.

Genetic Algorithms:

Nature has always been a great source of inspiration to all mankind. Genetic Algorithms (GAs) are
search based algorithms based on the concepts of natural selection and genetics. GAs are a subset of
a much larger branch of computation known as Evolutionary Computation.

GAs were developed by John Holland and his students and colleagues at the University of Michigan,
most notably David E. Goldberg and has since been tried on various optimization problems with a high
degree of success.

In GAs, we have a pool or a population of possible solutions to the given problem. These solutions
then undergo recombination and mutation (like in natural genetics), producing new children, and the
process is repeated over various generations. Each individual (or candidate solution) is assigned a
fitness value (based on its objective function value) and the fitter individuals are given a higher chance
to mate and yield more “fitter” individuals. This is in line with the Darwinian Theory of “Survival of the
Fittest”.

In this way we keep “evolving” better individuals or solutions over generations, till we reach a stopping
criterion.

Genetic Algorithms are sufficiently randomized in nature, but they perform much better than random
local search (in which we just try various random solutions, keeping track of the best so far), as they
exploit historical information as well.

Advantages of GAs

GAs have various advantages which have made them immensely popular. These include −

 Does not require any derivative information (which may not be available for many real-
world problems).

 Is faster and more efficient as compared to the traditional methods.

 Has very good parallel capabilities.

 Optimizes both continuous and discrete functions and also multi-objective problems.

 Provides a list of “good” solutions and not just a single solution.

 Always gets an answer to the problem, which gets better over the time.

 Useful when the search space is very large and there are a large number of parameters
involved.

Limitations of GAs

Like any technique, GAs also suffer from a few limitations. These include −

 GAs are not suited for all problems, especially problems which are simple and for which
derivative information is available.
  Fitness value is calculated repeatedly which might be computationally expensive for some
problems.

 Being stochastic, there are no guarantees on the optimality or the quality of the solution.

 If not implemented properly, the GA may not converge to the optimal solution.

GA – Motivation

Genetic Algorithms have the ability to deliver a “good-enough” solution “fast-enough”. This makes
genetic algorithms attractive for use in solving optimization problems. The reasons why GAs are
needed are as follows −

Solving Difficult Problems

In computer science, there is a large set of problems, which are NP-Hard. What this essentially
means is that, even the most powerful computing systems take a very long time (even years!) to
solve that problem. In such a scenario, GAs prove to be an efficient tool to provide usable near-
optimal solutions in a short amount of time.

Failure of Gradient Based Methods

Traditional calculus based methods work by starting at a random point and by moving in the
direction of the gradient, till we reach the top of the hill. This technique is efficient and works very
well for single-peaked objective functions like the cost function in linear regression. But, in most
real-world situations, we have a very complex problem called as landscapes, which are made of
many peaks and many valleys, which causes such methods to fail, as they suffer from an inherent
tendency of getting stuck at the local optima as shown in the following figure.
Getting a Good Solution Fast

Some difficult problems like the Travelling Salesperson Problem (TSP), have real-world applications
like path finding and VLSI Design. Now imagine that you are using your GPS Navigation system, and it
takes a few minutes (or even a few hours) to compute the “optimal” path from the source to
destination. Delay in such real world applications is not acceptable and therefore a “good-enough”
solution, which is delivered “fast” is what is required.

Evolution Strategies (ESs) are a sub-class of nature-inspired direct search

(and optimization) methods belonging to the class of Evolutionary Algorithms (EAs) which use
mutation, recombination, and selection applied to a population of individuals containing candidate
solutions in order to evolve iteratively better and better solutions. Evolution Strategies can be applied
in all fields of optimization including continuous, discrete, combinatorial search spaces without and
with constraints as well as mixed search spaces. It can also be applied to a set of objective functions
in context of multi-objective optimization.

Evolutionary Programming (EP)

It is a stochastic OPTIMIZATION strategy similar to GENETIC ALGORITHMs, but instead places
emphasis on the behavioural linkage between PARENTS and their OFFSPRING, rather than seeking to
emulate specific GENETIC OPERATORS as observed in nature. EVOLUTIONARY PROGRAMMING is
similar to EVOLUTION STRATEGIES. The basic EP method involves 3 steps (Repeat until a threshold for
iteration is exceeded or an adequate solution is obtained):

(1) Choose an initial POPULATION of trial solutions at random. The number of solutions in a
population is highly relevant to the speed of OPTIMIZATION, but no definite answers are available as
to how many solutions are appropriate (other than >1) and how many solutions are just wasteful.

(2) Each solution is replicated into a new POPULATION. Each of these OFFSPRING solutions are
mutated according to a distribution of MUTATION types, ranging from minor too extreme with a
continuum of mutation types between. The severity of MUTATION is judged on the basis of the
functional change imposed on the PARENTS.

(3) Each OFFSPRING solution is assessed by computing its FITNESS. Typically, a stochastic
tournament is held to determine N solutions to be retained for the POPULATION of solution ,
although this is occasionally performed deterministically. There is no requirement that the
POPULATION SIZE be held constant, however, nor that only a single OFFSPRING be generated from
each PARENT.
It should be pointed out that EP typically does not use any CROSSOVER as a GENETIC OPERATOR.

The main differences between ES and EP are:

1. SELECTION: EP typically uses STOCHASTIC SELECTION via a tournament. Each trial SOLUTION
in the POPULATION faces competition against a preselected number of opponents and receives
a "win" if it is at least as good as its opponent in each encounter. SELECTION then eliminates those
SOLUTIONS with the least wins. In contrast, ES typically uses deterministic SELECTION in which the
worst SOLUTIONS are purged from the POPULATION based directly on their function evaluation.
 2. RECOMBINATION: EP is an abstraction of EVOLUTION at the level of reproductive POPULATIONs
(i.e., SPECIES) and thus no RECOMBINATION mechanisms are typically used because
RECOMBINATION does not occur between SPECIES (by definition: see Mary’s biological species
concept). In contrast, ES is an abstraction of EVOLUTION at the level of INDIVIDUAL behaviour. When
self-adaptive information is incorporated this is purely GENETIC information (as opposed to
PHENOTYPIC) and thus some forms of RECOMBINATION are reasonable and many forms of
RECOMBINATION have been implemented within ES. Again, the effectiveness of such operators
depends on the problem at hand.

Genetic programming typically starts with a population of randomly generated computer programs
composed of the available programmatic ingredients. Genetic programming iteratively transforms a
population of computer programs into a new generation of the population by applying analogy of
naturally occurring genetic operations. These operations are applied to individual(s) selected from the
population. The individuals are probabilistically selected to participate in the genetic operations based
on their fitness (as measured by the fitness measure provided by the human user in the third
preparatory step). The iterative transformation of the population is executed inside the main
generational loop of the run of genetic programming. The executional steps of genetic programming
(that is, the flowchart of genetic programming) are as follows:

(1) Randomly create an initial population (generation 0) of individual computer programs composed
of the available functions and terminals.
(2) Iteratively perform the following sub-steps (called a generation) on the population until the
termination criterion is satisfied:
(a) Execute each program in the population and ascertain its fitness (explicitly or
Implicitly) using the problem’s fitness measure.

(b) Select one or two individual program(s) from the population with a probability
Based on fitness (with reselection allowed) to participate in the genetic operations in (c).

(c) Create new individual program(s) for the population by applying the following genetic
operations with specified probabilities:

i) Reproduction: Copy the selected individual program to the new population.

(ii) Crossover: Create new offspring program(s) for the new population by recombining randomly
chosen parts from two selected programs.
(iii) Mutation: Create one new offspring program for the new population by randomly mutating a
randomly chosen part of one selected program.
(iv) Architecture-altering operations: Choose an architecture-altering operation from the available
repertoire of such operations and create one new offspring program for the new population by
applying the chosen architecture-altering operation to one selected program.

(3) After the termination criterion is satisfied, the single best program in the population produced
during the run (the best-so-far individual) is harvested and designated as the result of the run. If the
run is successful, the result may be a solution (or approximate solution) to the problem.
Basic Terminology

Before beginning a discussion on Genetic Algorithms, it is essential to be familiar with some

basic terminology which will be used throughout this tutorial.
 Population − It is a subset of all the possible (encoded) solutions to the given problem.
The population for a GA is analogous to the population for human beings except that
instead of human beings, we have Candidate Solutions representing human beings.
 Chromosomes − A chromosome is one such solution to the given problem.
 Gene − A gene is one element position of a chromosome.
 Allele − It is the value a gene takes for a particular chromosome.

 Genotype − Genotype is the population in the computation space. In the computation

space, the solutions are represented in a way which can be easily understood and
manipulated using a computing system.
 Phenotype − Phenotype is the population in the actual real world solution space in
which solutions are represented in a way they are represented in real world situations.
 Decoding and Encoding − For simple problems, the phenotype and genotype spaces
are the same. However, in most of the cases, the phenotype and genotype spaces are
different. Decoding is a process of transforming a solution from the genotype to the
phenotype space, while encoding is a process of transforming from the phenotype to
genotype space. Decoding should be fast as it is carried out repeatedly in a GA during
the fitness value calculation.
For example, consider the 0/1 Knapsack Problem. The Phenotype space consists of
solutions which just contain the item numbers of the items to be picked.
However, in the genotype space it can be represented as a binary string of length n
(where n is the number of items). A 0 at position x represents that xth item is picked
while a 1 represents the reverse. This is a case where genotype and phenotype spaces
are different.
  Fitness Function − A fitness function simply defined is a function which takes the
solution as input and produces the suitability of the solution as the output. In some
cases, the fitness function and the objective function may be the same, while in others
it might be different based on the problem.
 Genetic Operators − These alter the genetic composition of the offspring. These
include crossover, mutation, selection, etc.
The basic structure of a GA is as follows −
Start with an initial population (which may be generated at random or seeded by other
heuristics), select parents from this population for mating. Apply crossover and mutation
operators on the parents to generate new off-springs. And finally these off-springs replace
the existing individuals in the population and the process repeats. In this way genetic
algorithms actually try to mimic the human evolution to some extent.

Fig: Basic Structure

 Genotype Representation
One of the most important decisions to make while implementing a genetic algorithm is deciding the
representation that we will use to represent our solutions. It has been observed that improper
representation can lead to poor performance of the GA.

Therefore, choosing a proper representation, having a proper definition of the mappings between the
phenotype and genotype spaces is essential for the success of a GA.

In this section, we present some of the most commonly used representations for genetic algorithms.
However, representation is highly problem specific and the reader might find that another
representation or a mix of the representations mentioned here might suit his/her problem better.

Binary Representation
This is one of the simplest and most widely used representation in GAs. In this type of representation
the genotype consists of bit strings. For some problems when the solution space consists of Boolean
decision variables – yes or no, the binary representation is natural. Take for example the 0/1 Knapsack
Problem. If there are n items, we can represent a solution by a binary string of n elements, where the
xth element tells whether the item x is picked (1) or not (0).

For other problems, specifically those dealing with numbers, we can represent the numbers with their
binary representation. The problem with this kind of encoding is that different bits have different
significance and therefore mutation and crossover operators can have undesired consequences. This
can be resolved to some extent by using Gray Coding, as a change in one bit does not have a massive
effect on the solution.

Real Valued Representation

For problems where we want to define the genes using continuous rather than discrete variables,
the real valued representation is the most natural. The precision of these real valued or floating
point numbers is however limited to the computer.

Integer Representation
For discrete valued genes, we cannot always limit the solution space to binary ‘yes’ or ‘no’. For
example, if we want to encode the four distances – North, South, East and West, we can encode
them as {0,1,2,3}. In such cases, integer representation is desirable.
Permutation Representation
In many problems, the solution is represented by an order of elements. In such cases permutation
representation is the most suited.

A classic example of this representation is the travelling salesman problem (TSP). In this the
salesman has to take a tour of all the cities, visiting each city exactly once and come back to the
starting city. The total distance of the tour has to be minimized. The solution to this TSP is naturally
an ordering or permutation of all the cities and therefore using a permutation representation makes
sense for this problem.

Genetic Algorithms - Population

Population is a subset of solutions in the current generation. It can also be defined as a set of
chromosomes. There are several things to be kept in mind when dealing with GA population −

 The diversity of the population should be maintained otherwise it might lead to premature
convergence.

 The population size should not be kept very large as it can cause a GA to slow down, while a
smaller population might not be enough for a good mating pool. Therefore, an optimal
population size needs to be decided by trial and error.

The population is usually defined as a two dimensional array of – size population, size x,
chromosome size.

Population Initialization
There are two primary methods to initialize a population in a GA. They are −

 Random Initialization − Populate the initial population with completely random solutions.

 Heuristic initialization − Populate the initial population using a known heuristic for the
problem.

It has been observed that the entire population should not be initialized using a heuristic, as it can
result in the population having similar solutions and very little diversity. It has been experimentally
observed that the random solutions are the ones to drive the population to optimality. Therefore,
with heuristic initialization, we just seed the population with a couple of good solutions, filling up the
rest with random solutions rather than filling the entire population with heuristic based solutions.
It has also been observed that heuristic initialization in some cases, only effects the initial fitness of
the population, but in the end, it is the diversity of the solutions which lead to optimality.

Population Models
There are two population models widely in use −

Steady State

In steady state GA, we generate one or two off-springs in each iteration and they replace one or two
individuals from the population. A steady state GA is also known as Incremental GA.

Generational

In a generational model, we generate ‘n’ off-springs, where n is the population size, and the entire
population is replaced by the new one at the end of the iteration.

Genetic Algorithms - Fitness Function

The fitness function simply defined is a function which takes a candidate solution to the problem as
input and produces as output how “fit” our how “good” the solution is with respect to the problem
in consideration.

Calculation of fitness value is done repeatedly in a GA and therefore it should be sufficiently fast. A
slow computation of the fitness value can adversely affect a GA and make it exceptionally slow.

In most cases the fitness function and the objective function are the same as the objective is to
either maximize or minimize the given objective function. However, for more complex problems
with multiple objectives and constraints, an Algorithm Designer might choose to have a different
fitness function.

A fitness function should possess the following characteristics −

 The fitness function should be sufficiently fast to compute.

 It must quantitatively measure how fit a given solution is or how fit individuals can be
produced from the given solution.

In some cases, calculating the fitness function directly might not be possible due to the inherent
complexities of the problem at hand. In such cases, we do fitness approximation to suit our needs.

The following image shows the fitness calculation for a solution of the 0/1 Knapsack. It is a simple
fitness function which just sums the profit values of the items being picked (which have a 1),
scanning the elements from left to right till the knapsack is full.
Genetic Algorithms - Parent Selection
Parent Selection is the process of selecting parents which mate and recombine to create off-springs
for the next generation. Parent selection is very crucial to the convergence rate of the GA as good
parents drive individuals to a better and fitter solutions.

However, care should be taken to prevent one extremely fit solution from taking over the entire
population in a few generations, as this leads to the solutions being close to one another in the
solution space thereby leading to a loss of diversity. Maintaining good diversity in the population is
extremely crucial for the success of a GA. This taking up of the entire population by one extremely fit
solution is known as premature convergence and is an undesirable condition in a GA.

Fitness Proportionate Selection

Fitness Proportionate Selection is one of the most popular ways of parent selection. In this every
individual can become a parent with a probability which is proportional to its fitness. Therefore,
fitter individuals have a higher chance of mating and propagating their features to the next
generation. Therefore, such a selection strategy applies a selection pressure to the more fit
individuals in the population, evolving better individuals over time.

Consider a circular wheel. The wheel is divided into n pies, where n is the number of individuals in
the population. Each individual gets a portion of the circle which is proportional to its fitness value.

Two implementations of fitness proportionate selection are possible −

Roulette Wheel Selection

In a roulette wheel selection, the circular wheel is divided as described before. A fixed point is
chosen on the wheel circumference as shown and the wheel is rotated. The region of the wheel
which comes in front of the fixed point is chosen as the parent. For the second parent, the same
process is repeated.

It is clear that a fitter individual has a greater pie on the wheel and therefore a greater chance of
landing in front of the fixed point when the wheel is rotated. Therefore, the probability of choosing
an individual depends directly on its fitness.

Implementation wise, we use the following steps −

 Calculate S = the sum of a finesses.

 Generate a random number between 0 and S.

 Starting from the top of the population, keep adding the finesses to the partial sum P, till
P<S.

 The individual for which P exceeds S is the chosen individual.

Tournament Selection
In K-Way tournament selection, we select K individuals from the population at random and select
the best out of these to become a parent. The same process is repeated for selecting the next
parent. Tournament Selection is also extremely popular in literature as it can even work with
negative fitness values.
Rank Selection
Rank Selection also works with negative fitness values and is mostly used when the individuals in the
population have very close fitness values (this happens usually at the end of the run). This leads to
each individual having an almost equal share of the pie (like in case of fitness proportionate
selection) as shown in the following image and hence each individual no matter how fit relative to
each other has an approximately same probability of getting selected as a parent. This in turn leads
to a loss in the selection pressure towards fitter individuals, making the GA to make poor parent
selections in such situations.

In this, we remove the concept of a fitness value while selecting a parent. However, every individual
in the population is ranked according to their fitness. The selection of the parents depends on the
rank of each individual and not the fitness. The higher ranked individuals are preferred more than
the lower ranked ones.
 Chromosome Fitness Value Rank

A 8.1 1

B 8.0 4

C 8.05 2

D 7.95 6

E 8.02 3

F 7.99 5

Random Selection

In this strategy we randomly select parents from the existing population. There is no selection
pressure towards fitter individuals and therefore this strategy is usually avoided.

Genetic Algorithms - Crossover

Crossover Operators
The crossover operator is analogous to reproduction and biological crossover. In this more than one
parent is selected and one or more off-springs are produced using the genetic material of the
parents. Crossover is usually applied in a GA with a high probability – pc .

In this section we will discuss some of the most popularly used crossover operators. It is to be noted
that these crossover operators are very generic and the GA Designer might choose to implement a
problem-specific crossover operator as well.

One Point Crossover

In this one-point crossover, a random crossover point is selected and the tails of its two parents are
swapped to get new off-springs.
Multi Point Crossover

Multi point crossover is a generalization of the one-point crossover wherein alternating segments
are swapped to get new off-springs.

Uniform Crossover

In a uniform crossover, we don’t divide the chromosome into segments, rather we treat each gene
separately. In this, we essentially flip a coin for each chromosome to decide whether or not it’ll be
included in the off-spring. We can also bias the coin to one parent, to have more genetic material in
the child from that parent.

Whole Arithmetic Recombination

This is commonly used for integer representations and works by taking the weighted average of the
two parents by using the following formulae −

 Child1 = α.x + (1-α).y

 Child2 = α.x + (1-α).y

Obviously, if α = 0.5, then both the children will be identical as shown in the following image.

Davis’ Order Crossover (OX1)

OX1 is used for permutation based crossovers with the intention of transmitting information about
relative ordering to the off-springs. It works as follows −

 Create two random crossover points in the parent and copy the segment between them
from the first parent to the first offspring.

 Now, starting from the second crossover point in the second parent, copy the remaining
unused numbers from the second parent to the first child, wrapping around the list.

 Repeat for the second child with the parent’s role reversed.
There exist a lot of other crossovers like Partially Mapped Crossover (PMX), Order based crossover
(OX2), Shuffle Crossover, Ring Crossover, etc.

Genetic Algorithms - Mutation

In simple terms, mutation may be defined as a small random tweak in the chromosome, to get a new
solution. It is used to maintain and introduce diversity in the genetic population and is usually applied
with a low probability – pm. If the probability is very high, the GA gets reduced to a random search.
Mutation is the part of the GA which is related to the “exploration” of the search space. It has been
observed that mutation is essential to the convergence of the GA while crossover is not.

Mutation Operators
In this section, we describe some of the most commonly used mutation operators. Like the crossover
operators, this is not an exhaustive list and the GA designer might find a combination of these
approaches or a problem-specific mutation operator more useful.

Bit Flip Mutation

In this bit flip mutation, we select one or more random bits and flip them. This is used for binary
encoded GAs.

Random Resetting
Random Resetting is an extension of the bit flip for the integer representation. In this, a random
value from the set of permissible values is assigned to a randomly chosen gene.
Swap Mutation
In swap mutation, we select two positions on the chromosome at random, and interchange the
values. This is common in permutation based encodings.

Scramble Mutation
Scramble mutation is also popular with permutation representations. In this, from the entire
chromosome, a subset of genes is chosen and their values are scrambled or shuffled randomly.

Inversion Mutation
In inversion mutation, we select a subset of genes like in scramble mutation, but instead of shuffling
the subset, we merely invert the entire string in the subset.

Genetic Algorithms - Survivor Selection

The Survivor Selection Policy determines which individuals are to be kicked out and which are to be
kept in the next generation. It is crucial as it should ensure that the fitter individuals are not kicked
out of the population, while at the same time diversity should be maintained in the population.

Some GAs employ Elitism. In simple terms, it means the current fittest member of the population is
always propagated to the next generation. Therefore, under no circumstance can the fittest member
of the current population be replaced.

The easiest policy is to kick random members out of the population, but such an approach frequently
has convergence issues, therefore the following strategies are widely used.

Age Based Selection

In Age-Based Selection, we don’t have a notion of a fitness. It is based on the premise that each
individual is allowed in the population for a finite generation where it is allowed to reproduce, after
that, it is kicked out of the population no matter how good its fitness is.
For instance, in the following example, the age is the number of generations for which the individual
has been in the population. The oldest members of the population i.e. P4 and P7 are kicked out of the
population and the ages of the rest of the members are incremented by one.

Fitness Based Selection

In this fitness based selection, the children tend to replace the least fit individuals in the population.
The selection of the least fit individuals may be done using a variation of any of the selection policies
described before – tournament selection, fitness proportionate selection, etc.

For example, in the following image, the children replace the least fit individuals P1 and P10 of the
population. It is to be noted that since P1 and P9 have the same fitness value, the decision to remove
which individual from the population is arbitrary.
 Genetic Algorithms - Termination Condition
The termination condition of a Genetic Algorithm is important in determining when a GA run will end.
It has been observed that initially, the GA progresses very fast with better solutions coming in every
few iterations, but this tends to saturate in the later stages where the improvements are very small.
We usually want a termination condition such that our solution is close to the optimal, at the end of
the run.

Usually, we keep one of the following termination conditions −

 When there has been no improvement in the population for X iterations.

 When we reach an absolute number of generations.

 When the objective function value has reached a certain pre-defined value.

For example, in a genetic algorithm we keep a counter which keeps track of the generations for which
there has been no improvement in the population. Initially, we set this counter to zero. Each time we
don’t generate off-springs which are better than the individuals in the population, we increment the
counter.

However, if the fitness any of the off-springs is better, then we reset the counter to zero. The algorithm
terminates when the counter reaches a predetermined value.

Like other parameters of a GA, the termination condition is also highly problem specific and the GA
designer should try out various options to see what suits his particular problem the best.

Schema Theorem
Researchers have been trying to figure out the mathematics behind the working of genetic algorithms,
and Holland’s Schema Theorem is a step in that direction. Over the year’s various improvements and
suggestions have been done to the Schema Theorem to make it more general.

In this section, we don’t delve into the mathematics of the Schema Theorem, rather we try to develop
a basic understanding of what the Schema Theorem is. The basic terminology to know are as follows
−

 A Schema is a “template”. Formally, it is a string over the alphabet = {0,1,*}, where * is don’t
care and can take any value. Therefore, *10*1 could mean 01001, 01011, 11001, or 11011

Geometrically, a schema is a hyper-plane in the solution search space.

 Order of a schema is the number of specified fixed positions in a gene.

  Defining length is the distance between the two furthest fixed symbols in the gene.

The schema theorem states that this schema with above average fitness, short defining length and
lower order is more likely to survive crossover and mutation.

Genetic Algorithms - Application Areas

Genetic Algorithms are primarily used in optimization problems of various kinds, but they are
frequently used in other application areas as well.

In this section, we list some of the areas in which Genetic Algorithms are frequently used. These are −

 Optimization − Genetic Algorithms are most commonly used in optimization problems

wherein we have to maximize or minimize a given objective function value under a given set
of constraints. The approach to solve Optimization problems has been highlighted throughout
the tutorial.
 Economics − GAs are also used to characterize various economic models like the cobweb
model, game theory equilibrium resolution, asset pricing, etc.

 Neural Networks − GAs are also used to train neural networks, particularly recurrent neural
networks.

 Parallelization − GAs also have very good parallel capabilities, and prove to be very effective
means in solving certain problems, and also provide a good area for research.

 Image Processing − GAs are used for various digital image processing (DIP) tasks as well like
dense pixel matching.

 Vehicle routing problems − With multiple soft time windows, multiple depots and a
heterogeneous fleet.

 Scheduling applications − GAs are used to solve various scheduling problems as well,
particularly the time tabling problem.

 Machine Learning − as already discussed, genetics based machine learning (GBML) is a niche
area in machine learning.

 Robot Trajectory Generation − GAs have been used to plan the path which a robot arm takes
by moving from one point to another.

 Parametric Design of Aircraft − GAs have been used to design aircrafts by varying the
parameters and evolving better solutions.

 DNA Analysis − GAs have been used to determine the structure of DNA using spectrometric
data about the sample.

 Multimodal Optimization − GAs are obviously very good approaches for multimodal
optimization in which we have to find multiple optimum solutions.

 Traveling salesman problem and its applications − GAs have been used to solve the TSP,
which is a well-known combinatorial problem using novel crossover and packing strategies.
 Traveling salesman problem using Genetic Algorithm approach
Travelling salesman problem is a combinatorial optimization problem. Which in terms of problem
classification falls into NP-hard problem. A general problem of TSP is “Given a list of cities and the
distances between each pair of cities, what is the shortest possible route that visits each city and
returns to the origin city?”. Here we will be solving this problem using a genetic algorithm in python.
It’s kind of basic implementation of genetic algorithm. Genetic Algorithm is used to find a solution to
the traveling salesman problem (TSP). The TSP is described as follows:

“Given a list of cities and the distances between each pair of cities, what is the shortest possible route
that visits each city and returns to the origin city?”

Given this, there are two important rules to keep in mind:

1. Each city needs to be visited exactly one time

2. We must return to the starting city, so our total distance needs to be calculated accordingly

The approach

Let’s start with a few definitions, rephrased in the context of the TSP:

 Gene: a city (represented as (x, y) coordinates)

 Individual (aka “chromosome”): a single route satisfying the conditions above

 Population: a collection of possible routes (i.e., collection of individuals)

 Parents: two routes that are combined to create a new route

 Mating pool: a collection of parents that are used to create our next population (thus creating
the next generation of routes)

 Fitness: a function that tells us how good each route is (in our case, how short the distance is)
  Mutation: a way to introduce variation in our population by randomly swapping two cities in
a route

 Elitism: a way to carry the best individuals into the next generation

A Markov process
A Markov process is a stochastic process that satisfies the Markov property (sometimes characterized
as "memorylessness"). In simpler terms, it is a process for which predictions can be made regarding
future outcomes based solely on its present state and—most importantly—such predictions are just
as good as the ones that could be made knowing the process's full history. In other
words, conditional on the present state of the system, its future and past states are independent.

A Markov chain is a type of Markov process that has either a discrete state space or a discrete index
set (often representing time), but the precise definition of a Markov chain varies. For example, it is
common to define a Markov chain as a Markov process in either discrete or continuous time with a
countable state space (thus regardless of the nature of time), but it is also common to define a Markov
chain as having discrete time in either countable or continuous state space (thus regardless of the
state space).

Types of Markov chains:

The system's state space and time parameter index need to be specified. The following table gives an
overview of the different instances of Markov processes for different levels of state space generality
and for discrete time v. continuous time:

Countable state space Continuous or general state space

(discrete-time) Markov chain on a Markov chain on a measurable state

Discrete-time
countable or finite state space space (for example, Harris chain)

Any continuous stochastic

Continuous-time Markov process or
Continuous-time process with the Markov property (for
Markov jump process
example, the Wiener process)

Note that there is no definitive agreement in the literature on the use of some of the terms that signify
special cases of Markov processes. Usually the term "Markov chain" is reserved for a process with a
discrete set of times, that is, a discrete-time Markov chain (DTMC), but a few authors use the term
"Markov process" to refer to a continuous-time Markov chain (CTMC) without explicit mention. In
addition, there are other extensions of Markov processes that are referred to as such but do not
necessarily fall within any of these four categories Moreover, the time index need not necessarily be
real-valued; like with the state space, there are conceivable processes that move through index sets
with other mathematical constructs. Notice that the general state space continuous-time Markov
chain is general to such a degree that it has no designated term.

While the time parameter is usually discrete, the state space of a Markov chain does not have any
generally agreed-on restrictions: the term may refer to a process on an arbitrary state space. However,
many applications of Markov chains employ finite or countably infinite state spaces, which have a
more straightforward statistical analysis. Besides time-index and state-space parameters, there are
many other variations, extensions and generalizations. For simplicity, most of this article concentrates
on the discrete-time, discrete state-space case, unless mentioned otherwise.

Tabu Search (TS)

Tabu Search is a commonly used meta-heuristic used for optimizing model parameters to solve
combinatorial optimization problems. A meta-heuristic is a general strategy that is used to guide and
control actual heuristics. Tabu Search is often regarded as integrating memory structures into local
search strategies. As local search has a lot of limitations, Tabu Search is designed to combat a lot of
those issues.

The basic idea of Tabu Search is to penalize moves that take the solution into previously visited search
spaces (also known as tabu).

Algorithm:

Step 1: We first start with an initial solution s = S₀. This can be any solution that fits the criteria for an
acceptable solution.

Step 2: Generate a set of neighbouring solutions to the current solution s labeled N(s). From this set
of solutions, the solutions that are in the Tabu List T(S) are removed with the exception of the solutions
that fit the Aspiration Criteria. This new set of results is the new N(s).

Step 3: Choose the best solution out of N(s) and label this new solution s’. If the solution s’ is better
than the current best solution, update the current best solution. After, regardless if s’ is better than s,
we update s to be s’.

Step 4: Update the Tabu List T(s) by removing all moves that are expired past the Tabu Tenure and
add the new move s’ to the Tabu List.

Step 5: If the Termination Criteria are met, then the search stops or else it will move onto the next
iteration.
Tabu Search, however, does deterministically accept non-improving solutions in order to prevent
getting stuck in local minimums.

Examples of Problems to Solve with TS

 N-Queens Problem

 Traveling Salesman Problem (TSP)

 Minimum Spanning Tree (MST)

 Assignment Problems

 Vehicle Routing

 DNA Sequencing

Simulated Annealing
The Simulated Annealing algorithm is based upon Physical Annealing in real life. Physical Annealing is
the process of heating up a material until it reaches an annealing temperature and then it will
be cooled down slowly in order to change the material to a desired structure. When the material is
hot, the molecular structure is weaker and is more susceptible to change. When the material cools
down, the molecular structure is harder and is less susceptible to change.

Another important part of this analogy is the following equation from Thermal Dynamics:
This equation calculates the probability that the Energy Magnitude will increase. We can calculate this
value given some Energy Magnitude and some temperature t along with the Boltzmann constant k.

Simulated Annealing

Simulated Annealing (SA) mimics the Physical Annealing process but is used for optimizing parameters
in a model. This process is very useful for situations where there are a lot of local minima such that
algorithms like Gradient Descent would be stuck at.

Example of a problem with a local minima

In problems like the one above, if Gradient Descent started at the starting point indicated, it would be
stuck at the local minima and not be able to reach the global minima.

Algorithm

Step 1: We first start with an initial solution s = S₀. This can be any solution that fits the criteria for an
acceptable solution. We also start with an initial temperature t = t₀.

Step 2: Setup a temperature reduction function alpha. There are usually 3 main types of temperature
reduction rules:

Each reduction rule reduces the temperature at a different rate and each method is better at
optimizing a different type of model. For the 3rd rule, beta is an arbitrary constant.

Step 3: Starting at the initial temperature, loop through n iterations of Step 4 and then decrease the
temperature according to alpha. Stop this loop until the termination conditions are reached. The
termination conditions could be reaching some end temperature, reaching some acceptable threshold
of performance for a given set of parameters, etc. The mapping of time to temperature and how fast
the temperature decreases is called the Annealing Schedule.

Step 4: Given the neighbourhood of solutions N(s), pick one of the solutions and calculate the
difference in cost between the old solution and the new neighbour solution. The neighbourhood of a
solution are all solutions that are close to the solution. For example, the neighbourhood of a set of 5
parameters might be if we were to change one of the five parameters but kept the remaining four the
same.

Step 5: If the difference in cost between the old and new solution is greater than 0 (the new solution
is better), then accept the new solution. If the difference in cost is less than 0 (the old solution is
better), then generate a random number between 0 and 1 and accept it if it’s under the value
calculated from the Energy Magnitude equation from before.

In the Simulated Annealing case, the equation has been altered to the following:

Where the delta c is the change in cost and the t is the current temperature.

The P calculated in this case is the probability that we should accept the new solution.
Some of Problems to Optimize with Simulated Annealing

 Travelling Salesman Problem (TSP)

 Scheduling Problems

 Task Allocations

 Graph colouring and partitioning

 Non-linear function optimizations

Ant Colony Optimization technique

The algorithmic world is beautiful with multifarious strategies and tools being developed round the
clock to render to the need for high-performance computing. In fact, when algorithms are inspired by
natural laws, interesting results are observed. Evolutionary algorithms belong to such a class of
algorithms. These algorithms are designed so as to mimic certain behaviours as well as evolutionary
traits of the human genome. Moreover, such algorithmic design is not only constrained to humans but
can be inspired by the natural behaviour of certain animals as well. The basic aim of fabricating such
methodologies is to provide realistic, relevant and yet some low-cost solutions to problems that are
hitherto unsolvable by conventional means.

Different optimization techniques have thus evolved based on such evolutionary algorithms and
thereby opened up the domain of metaheuristics. Metaheuristic has been derived from two Greek
words, namely, Meta meaning one level above and heuriskein meaning to find. Algorithms such as
the Particle Swarm Optimization (PSO) and Ant Colony Optimization (ACO) are examples of swarm
intelligence and metaheuristics. The goal of swarm intelligence is to design intelligent multi-agent
systems by taking inspiration from the collective behaviour of social insects such as ants, termites,
bees, wasps, and other animal societies such as flocks of birds or schools of fish.

Concept:

Ant Colony Optimization technique is purely inspired from the foraging behaviour of ant colonies, first
introduced by Marco Dorigo in the 1990s. Ants are eusocial insects that prefer community survival
and sustaining rather than as individual species. They communicate with each other using sound,
touch and pheromone. Pheromones are organic chemical compounds secreted by the ants that
trigger a social response in members of same species. These are chemicals capable of acting like
hormones outside the body of the secreting individual, to impact the behaviour of the receiving
individuals. Since most ants live on the ground, they use the soil surface to leave pheromone trails
that may be followed (smelled) by other ants.

Ants live in community nests and the underlying principle of ACO is to observe the movement of the
ants from their nests in order to search for food in the shortest possible path. Initially, ants start to
move randomly in search of food around their nests. This randomized search opens up multiple routes
from the nest to the food source. Now, based on the quality and quantity of the food, ants carry a
portion of the food back with necessary pheromone concentration on its return path. Depending on
these pheromone trials, the probability of selection of a specific path by the following ants would be
a guiding factor to the food source. Evidently, this probability is based on the concentration as well as
the rate of evaporation of pheromone. It can also be observed that since the evaporation rate of
pheromone is also a deciding factor, the length of each path can easily be accounted for.

In the above figure, for simplicity, only two possible paths have been considered between the food
source and the ant nest. The stages can be analysed as follows:

1. Stage 1: All ants are in their nest. There is no pheromone content in the environment. (For
algorithmic design, residual pheromone amount can be considered without interfering with
the probability)

2. Stage 2: Ants begin their search with equal (0.5 each) probability along each path. Clearly, the
curved path is the longer and hence the time taken by ants to reach food source is greater
than the other.

3. Stage 3: The ants through the shorter path reaches food source earlier. Now, evidently they
face with a similar selection dilemma, but this time due to pheromone trail along the shorter
path already available, probability of selection is higher.

4. Stage 4: More ants return via the shorter path and subsequently the pheromone
concentrations also increase. Moreover, due to evaporation, the pheromone concentration in
the longer path reduces, decreasing the probability of selection of this path in further stages.
Therefore, the whole colony gradually uses the shorter path in higher probabilities. So, path
optimization is attained.