Genetic Algorithms

Mohammed Miskeen Ali

 Introduction
• A genetic algorithm is an adaptive heuristic search algorithm inspired
by "Darwin's theory of evolution in Nature."
• It is used to solve optimization problems in machine learning.
• It is one of the important algorithms as it helps solve complex
problems that would take a long time to solve(NP hard Problems).
• Genetic Algorithms are being widely used in different real-world
applications, for example, Designing electronic circuits, image
processing, and Machine Learning.
 Motivation
The motivation behind using GAs includes:
• Solving Difficult Problems:NP-Hard problems can be solved
• Complexity: They can handle complex, multi-dimensional, and poorly
understood search spaces.
• Adaptability: GAs adapt to the search space as they proceed,
potentially finding good solutions even when traditional methods fail.
• Robustness: They are robust against changes in the problem domain
and can escape local optima.
• Parallelism: GAs can inherently take advantage of parallel process
Basic terminologies used in Genetic Algorithms
• Population: Population is the subset of all possible or probable solutions, which can
solve the given problem.
• Chromosomes: A chromosome is one of the solutions in the population for the given
problem, and the collection of gene generate a chromosome.
• Gene: A chromosome is divided into a different gene, or it is an element of the
chromosome.

• Fitness Function: The fitness function is used to determine the individual's fitness
level in the population. It means the ability of an individual to compete with other
individuals. In every iteration, individuals are evaluated based on their fitness
function.
• Genetic Operators: In a genetic algorithm, the best individual mate to regenerate
offspring better than parents. Here genetic operators play a role in changing the
genetic composition of the next generation.
Basic terminologies used in Genetic Algorithms
• Selection: Process of choosing better solutions for reproduction.
There are three types of Selection methods available, which are:
Roulette wheel selection, Tournament selection, Rank-based selection
• Crossover: Combining parts of two solutions to create a new solution.
There are four styles of Crosssover methods are available
One point crossover,Two-point crossover, Livery crossover, Inheritable
Algorithms crossover
• Mutation: Randomly altering parts of a solution to maintain genetic
diversity.
There are three styles of mutation methods are available
Flip bit mutation, Gaussian mutation, Exchange/Swap mutation
 Working of Genetic algorithm
• The genetic algorithm works on the evolutionary generational cycle to
generate high-quality solutions. These algorithms use different
operations that either enhance or replace the population to give an
improved fit solution.
• It basically involves five phases to solve the complex optimization
problems, which are given as below:
1. Initialization
2. Fitness Assignment
3. Selection
4. Reproduction
5. Termination
 1. Initialization
The process of a genetic algorithm starts by generating the set of
individuals, which is called population. Here each individual is the
solution for the given problem.One of the most popular techniques for
initialization is the use of random binary strings.
 2. Fitness Assignment
• The following image shows the fitness calculation for a solution of the
0/1 Knapsack. It is a simple fitness function which just sums the profit
values of the items being picked (which have a 1), scanning the
elements from left to right till the knapsack is full.
 3. Selection
• The selection phase involves the selection of individuals for the
reproduction of offspring. All the selected individuals are then
arranged in a pair of two to increase reproduction. Then these
individuals transfer their genes to the next generation.
• There are three types of Selection methods available, which are:
a) Roulette wheel selection
b) Tournament selection
c) Rank-based selection
Roulette wheel selection Tournament
Selection

Rank Selection
 4. Reproduction
• After the selection process, the creation of a child occurs in the
reproduction step. In this step, the genetic algorithm uses two variation
operators that are applied to the parent population. The two operators
involved in the reproduction phase are Crossover and Mutation
 Crossover Operators
• The crossover plays a most significant role in the reproduction phase
of the genetic algorithm. In this process, a crossover point is selected
at random within the genes. Then the crossover operator swaps genetic
information of two parents from the current generation to produce a
new individual representing the offspring.
1.One Point Crossover 2.Two Point Crossover
 Mutation
• The mutation operator inserts random genes in the offspring (new
child) to maintain the diversity in the population. It can be done by
flipping some bits in the chromosomes.
• Flip bit mutation

• Swap mutation
 5. Termination
• After the reproduction phase, a stopping criterion is applied as a base
for termination. The algorithm terminates after the threshold fitness
solution is reached. It will identify the final solution as the best
solution in the population.
Illustrative example
 Hypothesis space search in Genetic
Algorithms
Hypothesis space search is a concept in machine learning and optimization
where we search through a space of possible hypotheses to find the best
one. the hypotheses are potential solutions to an optimization problem.
In this context:
• Hypothesis: A potential solution to the problem.
• Hypothesis Space: All possible solutions.
• Search Operators: Crossover and mutation that explore new areas of the
hypothesis space.
• Fitness Landscape: The mapping of hypotheses to their fitness values,
which the GA aims to navigate to find the global optimum.
 Genetic Programming (GP)
Genetic Programming extends the concept of Genetic Algorithms (GAs) to the domain of
programs to solve specific tasks.By representing programs as trees and applying evolutionary
operations, GP can automatically discover effective solutions across various domains.
• Representation: Solutions are represented as tree structures, where nodes are functions or
operations, and leaves are inputs or constants. eg: sin x +√ +y

• Fitness Evaluation: Each program is executed and its output is evaluated against the desired
outcome.
• Genetic Operators: Similar to GAs on each iteration it produces new generation but adapted to
tree structures (e.g., subtree crossover and mutation).
• GP applies evolutionary operations such as selection, crossover, and mutation to evolve
programs that perform specific tasks. GP's flexibility and adaptability make it a powerful tool for
a wide range of applications, from symbolic regression to automated design and artificial
 Models of Evolution and Learning
The study of evolution and learning models explores how organisms adapt
and improve over time through natural selection and experience. In
computational contexts, these concepts are applied to algorithms that
simulate these processes to solve complex problems.
• Darwinian Evolution: Survival of the fittest, where better solutions
reproduce more successfully.
• Lamarckian Evolution: The idea (rarely used in GAs) that individuals
can pass on characteristics acquired during their lifetime. Eg: Giraffe
• Baldwin Effect: The notion that learned behaviors can influence
evolutionary fitness. Eg:Fish
 Parallelizing Genetic Algorithms
Genetic Algorithms (GAs) are powerful optimization tools inspired by natural
selection and genetics. However, they can be computationally intensive,
especially for large and complex problems. Parallelizing Genetic Algorithms
(PGAs) aims to reduce computational time and improve efficiency by
distributing the workload across multiple processors or machines.It uses
multiple algorithm to solve single task.

• Fine-Grained Parallelism: Each individual or a small group of individuals

is processed in parallel, suitable for massively parallel systems.
• Coarse-Grained Parallelism: Each processor handles a separate sub-
population, occasionally exchanging individuals (migration).
Parallelization can significantly speed up GAs
 Advantages and Limitations of Genetic Algorithm
Advantages
• The parallel capabilities of genetic algorithms are best.
• It helps in optimizing various problems such as discrete functions, multi-
objective problems, and continuous functions.
• It provides a solution for a problem that improves over time.
• A genetic algorithm does not need derivative information.
Limitations
• Genetic algorithms are not efficient algorithms for solving simple problems.
• It does not guarantee the quality of the final solution to a problem.
• Repetitive calculation of fitness values may generate some computational
challenges.