Optimization

using Genetic Algorithm

Jonecis A. Dayap
 One of the pillars of Data Science is optimization.

What is optimization?

• Optimization problems are one of the key types of data analytics problems.
• An optimization problem consists of maximizing or minimizing a real function by systematically
choosing input values from an allowed set and computing the value of the function.
• It is useful in finding the best solution to a problem (which could be minimizing or maximizing the
functional form f(x)).

https://medium.com/analytics-vidhya/optimization-acb996a4623c#:~:text=%E2%80%9CAn%20optimization%20problem%20consists%20of%20maximizing%20or%20minimizing,set%20and%20computing%20the%20value%20of%20the%20function.
In order to find solution to optimization problems (relating to minimizing the functions) including machine
learning based problems, one must get a good understanding of the concepts of Local minima / global minima
and local maxima / global maxima.

The point at which a function takes the minimum value is called as global minima.

Fig 1. Local Minima and Global Minima in 2 and 3-dimensional space

https://vitalflux.com/local-global-maxima-minima-explained-examples/
Genetic algorithms (GAs)

• search-based optimization technique based on the principles of Genetics and Natural Selection.
• This algorithm reflects the process of natural selection where the fittest individuals are selected for
reproduction in order to produce offspring of the next generation.
• solve a variety of optimization problems that are not well suited for standard optimization algorithms,
including problems in which the objective function is discontinuous, nondifferentiable, stochastic, or
highly nonlinear.
• GAs were developed at the University of Michigan by John Holland and his students and colleagues,
most notably David E. Goldberg.

https://www.mathworks.com/help/gads/what-is-the-genetic-algorithm.html
 The process of natural selection starts with the selection of fittest individuals from a population.
They produce offspring which inherit the characteristics of the parents and will be added to the
next generation. If parents have better fitness, their offspring will be better than parents and have a
better chance at surviving. This process keeps on iterating and at the end, a generation with the
fittest individuals will be found.

Replace

Selection Produce

Population Parents Offspring

https://towardsdatascience.com/introduction-to-genetic-algorithms-including-example-code-e396e98d8bf3
This notion can be applied for a search problem. We consider a set of solutions for a problem
and select the set of best ones out of them.

Population
Five phases are considered in a genetic
algorithm.
Offspring Fitness
Calculation
1.Initial population
2.Fitness function
3.Selection
4.Crossover
5.Mutation Mating:
Mating
Crossover
Pool
Mutation

Parents
Selection

Genetic Algorithm Flowchart

https://towardsdatascience.com/introduction-to-genetic-algorithms-including-example-code-e396e98d8bf3
Phase 1. Initial Population

The process begins with a set of individuals which is called a Population.

Each individual is a solution to the problem you want to solve.
An individual is characterized by a set of parameters (variables) known as
Genes. Genes are joined into a string to form a Chromosome (solution).

In a genetic algorithm, the set of genes of an individual is represented using a

string, in terms of an alphabet. Usually, binary values are used (string of 1s
and 0s). We say that we encode the genes in a chromosome.

there are different types of

representations for genes such as
binary, value, permutation, and
others

https://towardsdatascience.com/introduction-to-genetic-algorithms-including-example-code-e396e98d8bf3
Encoding
Encoding of chromosomes is one of the problems, when you are starting to solve problem with GA.
Encoding depends on the problem.
1. Binary encoding
 In binary encoding every chromosome is a string of bits, 0 or 1.
 Knapsack problem
• There are things with given value and size, The knapsack has given capacity. Select things to maximize the value of
things in knapsack, but do not extend knapsack capacity.

2. Permutation encoding
 In permutation encoding, every chromosome is a string of numbers, which represents number in a sequence.
 It can be used in ordering problems, such as travelling salesman problem or task ordering problem.

3. Value Encoding
 In value encoding, every chromosome is a string of some values. Values can be anything connected to problem, form
numbers, real numbers or chars to some complicated objects.
 Direct value encoding can be used in problems, where some complicated value, such as real numbers, are used.
 Finding weights for neural network
• There is some neural network with given architecture. Find weights for inputs of neurons to train the network for
wanted output.

https://www.obitko.com/tutorials/genetic-algorithms/encoding.php
Phase 2. Fitness Function

The fitness function determines how fit an individual is

(the ability of an individual to compete with other
individuals). It gives a fitness score to each individual.
The probability that an individual will be selected for
reproduction is based on its fitness score.

https://towardsdatascience.com/introduction-to-genetic-algorithms-including-example-code-e396e98d8bf3
Phase 3. Selection

The idea of selection phase is to select the fittest individuals

and let them pass their genes to the next generation.
Two pairs of individuals (parents) are selected based on their
fitness scores. Individuals with high fitness have more chance
to be selected for reproduction.

https://towardsdatascience.com/introduction-to-genetic-algorithms-including-example-code-e396e98d8bf3
Phase 4. Crossover
Crossover is the most significant phase in a genetic algorithm.
For each pair of parents to be mated, a crossover point is
chosen at random from within the genes.
For example, consider the 1 point crossover as shown below.

Offspring are created by exchanging the genes of parents among

themselves until the crossover point is reached.

The new offspring are added to the population.

crossover has different types such as blend, one point, two points, uniform, and
others.

https://towardsdatascience.com/introduction-to-genetic-algorithms-including-example-code-e396e98d8bf3
Crossover Operators

One point crossover

one crossover point is selected, binary string from beginning

of chromosome to the crossover point is copied from one
parent, the rest is copied from the second parent

Two points crossover

Two crossover point are selected, binary string from

beginning of chromosome to the first crossover point is
copied from one parent, the part from the first to the second
crossover point is copied from the second parent and the
rest is copied from the first parent

https://www.tutorialspoint.com/genetic_algorithms/genetic_algorithms_mutation.htm
Phase 5. Mutation

In certain new offspring formed, some of their genes can be

subjected to a mutation with a low random probability. This
implies that some of the bits in the bit string can be flipped.

Mutation occurs to maintain diversity within the population and

prevent premature convergence.

There are different types of mutation such as bit flip, swap, inverse, uniform,
non-uniform, Gaussian, shrink, and others.

https://towardsdatascience.com/introduction-to-genetic-algorithms-including-example-code-e396e98d8bf3
Mutation Operators
0 0 1 1 0 1 0 0 1 0
Bit Flip Mutation

In this bit flip mutation, we select one or more random bits and
0 0 0 1 0 1 0 0 1 0
flip them. This is used for binary encoded GAs.

Swap Mutation

In swap mutation, we select two positions on the chromosome at 1 2 3 4 5 6 7 8 9 0

random, and interchange the values. This is common in
permutation based encodings.
1 2 7 4 5 6 3 8 9 0
Uniform Mutation
This operator replaces the value of the chosen gene with a
uniform random value selected between the user-specified upper
and lower bounds for that gene. This mutation operator can only
be used for integer and float genes.

https://www.tutorialspoint.com/genetic_algorithms/genetic_algorithms_mutation.htm
Termination

The algorithm terminates if the population has converged (does

not produce offspring which are significantly different from the
previous generation). Then it is said that the genetic algorithm has
provided a set of solutions to our problem.

Note:
The population has a fixed size. As new generations are formed,
individuals with least fitness die, providing space for new
offspring.

The sequence of phases is repeated to produce individuals in each

new generation which are better than the previous generation.

https://towardsdatascience.com/introduction-to-genetic-algorithms-including-example-code-e396e98d8bf3
Genetic Algorithm Implementation
in Python
Pseudocode

https://towardsdatascience.com/genetic-algorithm-implementation-in-python-5ab67bb124a6
This tutorial will implement the genetic
algorithm optimization technique in Python
based on a simple example in which we are Population

trying to maximize the output of an equation.

Offspring Fitness
The tutorial uses the value representation for Calculation
genes, one point crossover, and uniform
mutation.

Mating:
Mating
Crossover
Pool
Mutation

Parents
Selection

Genetic Algorithm Flowchart

https://towardsdatascience.com/genetic-algorithm-implementation-in-python-5ab67bb124a6