History of Evolutionary

Computing, Genetic Algorithim

History of Evolutionary Computing
• Evolutionary Computing is a subfield of artificial intelligence (AI) and
computational intelligence that draws inspiration from the process of
natural evolution. Its roots can be traced back to the 1950s and
1960s, where early pioneers began to model problem-solving
techniques on biological evolution.
1950s-1960s: Early Foundations

• Evolutionary Programming (EP): One of the earliest forms of evolutionary

algorithms, developed by Lawrence J. Fogel in 1960. He aimed to use
simulated evolution to create artificial intelligence by evolving finite-state
machines.
• Genetic Algorithms (GA): Introduced by John Holland in the 1960s, genetic
algorithms were inspired by the process of natural selection. Holland’s key
contribution was formalizing the use of operators like crossover, mutation,
and selection for problem-solving.
• Evolution Strategies (ES): Developed by Ingo Rechenberg and Hans-Paul
Schwefel in the 1960s, this approach focused on optimization problems,
primarily in engineering, by simulating the process of natural evolution using
mutations and selection.
1970s: Formalization and Growth
• Canonical Genetic Algorithm: John Holland's work culminated in his
book "Adaptation in Natural and Artificial Systems" (1975), where he
outlined the theoretical foundation of genetic algorithms. His research
formalized the principles of selection, crossover, and mutation to solve
complex optimization problems.
• Evolutionary Programming and Evolution Strategies Expand: The
ideas of evolutionary programming and evolution strategies began to
gain traction in research, with new variations of the algorithms being
developed for practical applications in optimization and simulation.
1980s: Broader Adoption and Hybrid
Approaches
• Genetic Programming (GP): Developed by John Koza in the late
1980s, genetic programming extended genetic algorithms to evolve
computer programs instead of fixed-length bit strings. This opened the
door for automatic programming and solving a broader range of
problems.
• Integration with Other Techniques: During this time, evolutionary
computing began to be combined with other methods like neural
networks and simulated annealing. These hybrid approaches helped
expand its applications to real-world problems, especially in
engineering and optimization.
1990s: Emergence as a Field
• Standardization and Growth: The 1990s saw the formalization of
evolutionary algorithms (EAs) as a recognized field within AI and
optimization. Conferences, journals, and communities dedicated to
evolutionary computing were established, such as the Genetic and
Evolutionary Computation Conference (GECCO) and IEEE Transactions
on Evolutionary Computation.
• Practical Applications: Evolutionary computing was increasingly
applied to diverse fields such as engineering design, finance,
scheduling, robotics, and bioinformatics. Its ability to solve complex,
multi-objective optimization problems was highly valued.
2000s-Present: Modern
Developments
• Multi-objective Evolutionary Algorithms (MOEAs): With increasing
complexity in real-world problems, researchers developed algorithms
like NSGA-II (Non-dominated Sorting Genetic Algorithm II) and
SPEA2 (Strength Pareto Evolutionary Algorithm 2) to tackle
optimization problems with multiple objectives.
• Computational Power and Parallelism: The growth of computing
power, including GPUs and distributed computing, allowed
evolutionary algorithms to handle larger datasets and more complex
simulations. Parallel evolutionary algorithms emerged to exploit these
advancements.
• Evolutionary Deep Learning: In recent years, evolutionary computing
has been applied to deep learning and neural networks. Algorithms
like neuroevolution use evolutionary principles to optimize neural
architectures, providing an alternative to traditional gradient-based
methods in certain cases.
• Applications in Real-World Domains: Today, evolutionary computing is
widely used in domains such as optimization of machine learning
models, robotics, industrial design, game playing, and complex system
modeling.
Genetic Algorithm
• Genetic Algorithm is one of the heuristic algorithms.
• They are used to solve optimization problems.
• They are inspired by Darwin’s Theory of Evolution.
• They are an intelligent exploitation of a random search.
• Although randomized, Genetic Algorithms are by no means random.
• Genetic Algorithms are being widely used in different real-world
applications, for example, Designing electronic circuits, code-
breaking, image processing, and artificial creativity.
What is a Genetic Algorithm?
• Genetic Algorithms(GAs) are adaptive heuristic search algorithms that
belong to the larger part of evolutionary algorithms.
• Genetic algorithms are based on the ideas of natural selection and
genetics.
• These are intelligent exploitation of random searches provided with
historical data to direct the search into the region of better
performance in solution space.
• They are commonly used to generate high-quality solutions for
optimization problems and search problems.
Terminologies
• Population: Population is the subset of all possible or probable solutions,
which can solve the given problem.
• Chromosomes: A chromosome is one of the solutions in the population for the
given problem, and the collection of gene generate a chromosome.
• Gene: A chromosome is divided into a different gene, or it is an element of the
chromosome.
• Allele: Allele is the value provided to the gene within a particular chromosome.
• Fitness Function: The fitness function is used to determine the individual's
fitness level in the population. It means the ability of an individual to compete
with other individuals. In every iteration, individuals are evaluated based on
their fitness function.
• Genetic Operators: In a genetic algorithm, the best individual mate to
regenerate offspring better than parents. Here genetic operators play a role in
changing the genetic composition of the next generation.
Operators of Genetic Algorithms
• Once the initial generation is created, the algorithm evolves the
generation using following operators –
• 1) Selection Operator: The idea is to give preference to the
individuals with good fitness scores and allow them to pass their genes
to successive generations.
• 2) Crossover Operator: This represents mating between individuals.
Two individuals are selected using selection operator and crossover
sites are chosen randomly. Then the genes at these crossover sites are
exchanged thus creating a completely new individual (offspring).
• 3) Mutation Operator: The key idea is to insert random genes in offspring
to maintain the diversity in the population to avoid premature
convergence.
How Genetic Algorithm Work?
• The genetic algorithm works on the evolutionary generational cycle to
generate high-quality solutions. These algorithms use different
operations that either enhance or replace the population to give an
improved fit solution.
• It basically involves five phases to solve the complex optimization
problems, which are given as below:
• Initialization
• Fitness Assignment
• Selection
• Reproduction
• Termination
1. Initialization

• The process of a genetic algorithm starts by generating the set of

individuals, which is called population.
• Here each individual is the solution for the given problem. An
individual contains or is characterized by a set of parameters called
Genes.
• Genes are combined into a string and generate chromosomes, which
is the solution to the problem. One of the most popular techniques
for initialization is the use of random binary strings.
2. Fitness Assignment

• Fitness function is used to determine how fit an individual is? It

means the ability of an individual to compete with other individuals.
In every iteration, individuals are evaluated based on their fitness
function.
• The fitness function provides a fitness score to each individual. This
score further determines the probability of being selected for
reproduction.
• The high the fitness score, the more chances of getting selected for
reproduction.
3. Selection
• The selection phase involves the selection of individuals for the
reproduction of offspring. All the selected individuals are then
arranged in a pair of two to increase reproduction. Then these
individuals transfer their genes to the next generation.

• There are three types of Selection methods available, which are:

• Roulette wheel selection

• Tournament selection
• Rank-based selection
4. Reproduction
• After the selection process, the creation of a child occurs in the
reproduction step. In this step, the genetic algorithm uses two variation
operators that are applied to the parent population. The two operators
involved in the reproduction phase are given below:

• Crossover: The crossover plays a most significant role in the

reproduction phase of the genetic algorithm. In this process, a
crossover point is selected at random within the genes. Then the
crossover operator swaps genetic information of two parents from the
current generation to produce a new individual representing the
offspring.
Crossover:

The genes of parents are exchanged among themselves until the crossover point is
met. These newly generated offspring are added to the population. This process is
also called or crossover. Types of crossover styles available:
One point crossover
Two-point crossover
Livery crossover
Inheritable Algorithms crossover
Mutation

• The mutation operator inserts random genes in the offspring (new

child) to maintain the diversity in the population. It can be done by
flipping some bits in the chromosomes.
• Mutation helps in solving the issue of premature convergence and
enhances diversification. The below image shows the mutation
process:
• Types of mutation styles available,
• Flip bit mutation
• Gaussian mutation
• Exchange/Swap mutation
Mutation
5. Termination

• After the reproduction phase, a stopping criterion is applied as a base

for termination. The algorithm terminates after the threshold fitness
solution is reached. It will identify the final solution as the best
solution in the population.
Flowchart of
Genetic
Algorithm
Advantages of Genetic Algorithm

• The parallel capabilities of genetic algorithms are best.

• It helps in optimizing various problems such as discrete functions,
multi-objective problems, and continuous functions.
• It provides a solution for a problem that improves over time.
• A genetic algorithm does not need derivative information.
Limitations of Genetic Algorithms

• Genetic algorithms are not efficient algorithms for

solving simple problems.
• It does not guarantee the quality of the final solution to
a problem.
• Repetitive calculation of fitness values may generate
some computational challenges.
Basic Steps of Genetic Algorithms
1. Initialization: Create an initial population of random individuals.
2. Evaluation: Compute the fitness of each individual in the population using
the fitness function.
3. Selection: Select individuals based on their fitness to participate in
reproduction.
4. Crossover: Combine pairs of selected individuals to produce offspring
(new solutions).
5. Mutation: Apply random mutations to some of the offspring.
6. Replacement: Replace the old population with the new one, typically
keeping the fittest individuals.
7. Termination: Repeat the process until a stopping criterion is met, such as a
maximum number of generations or achieving a satisfactory fitness level.
Genetic Algorithm Flow

1.Problem: Maximize a mathematical function f(x).

2.Step 1: Initialize a population of random solutions (e.g., random
values for x).
3.Step 2: Evaluate the fitness of each individual by computing f(x).
4.Step 3: Select the best individuals to act as parents.
5.Step 4: Apply crossover and mutation to generate offspring.
6.Step 5: Replace the old population with the new one and repeat the
process until the solution converges or the stopping condition is met.
Algorithm
• Step-01:
• Randomly generate a set of possible solutions to a problem.
• Represent each solution as a fixed length character string.
• Step-02:
• Using a fitness function, test each possible solution against the problem to
evaluate them.
• Step-03:
• Keep the best solutions.
• Use best solutions to generate new possible solutions.
• Step-04:
• Repeat the previous two steps until-
• Either an acceptable solution is found
• Or until the algorithm has completed its iterations through a given
number of cycles / generations.
Genetic Algorithm Cycle

• The Genetic Algorithm (GA) cycle represents the iterative process

through which a population of potential solutions evolves toward
better solutions over time. This cycle consists of several steps that
simulate the mechanisms of natural selection, crossover, and
mutation found in biological evolution.
Steps in the Genetic Algorithm Cycle
• Initialization:The cycle begins by generating an initial population of
random individuals (potential solutions). Each individual is
represented by a chromosome, typically encoded as a binary string, a
set of real numbers, or other appropriate formats for the problem at
hand.
• Fitness Evaluation: Each individual in the population is evaluated
using a fitness function. The fitness function measures how well an
individual solves the given problem, assigning a fitness score based on
the quality of the solution. Higher fitness values represent better
solutions.
• Selection:Based on the fitness scores, individuals are selected for
reproduction. Selection methods (e.g., roulette wheel selection,
tournament selection, or rank selection) prioritize individuals with
higher fitness, giving them a greater chance of being chosen to pass
their genes to the next generation.
• Crossover (Recombination):Selected individuals, often called
parents, undergo crossover to produce offspring. Crossover involves
exchanging segments of the parents’ chromosomes to create new
individuals (children). This operator encourages the exploration of
new regions of the solution space by combining genetic material from
two parents.Common crossover techniques include single-point
crossover, two-point crossover, and uniform crossover.
• Mutation:After crossover, a mutation operator is applied to some
offspring. Mutation introduces random changes to individual genes in
the chromosomes, ensuring genetic diversity within the population.
This prevents the algorithm from getting stuck in local optima by
exploring new, potentially better, solutions. For example, in binary
encoding, mutation could flip a bit from 0 to 1 or vice versa.

• Replacement:The newly created offspring replace some or all of the

old population, forming the next generation. Replacement strategies
vary, but the most common approach is to replace the entire
population. Sometimes elitism is used, where the best-performing
individuals from the current generation are carried over to the next
without alteration.
• Termination: The cycle repeats the steps of evaluation, selection,
crossover, and mutation for multiple generations until a termination
condition is met. Common stopping criteria include:

• A pre-defined number of generations has been reached.

• A solution has been found that meets the desired fitness level.
• The population has converged, meaning no significant improvement is
seen over several generations.
Applications of Genetic
Algorithms
• Optimization: Used in industrial design, engineering, scheduling, and
network optimization to find the best solutions within constraints.
• Machine Learning: GAs are used to optimize model parameters, neural
network structures, and hyperparameters in machine learning models.
• Artificial Intelligence: Applied in game playing, decision-making
systems, and evolutionary robotics.
• Finance: Used in portfolio optimization, algorithmic trading, and risk
management.
• Bioinformatics: Applied in gene expression analysis, drug discovery,
and protein folding optimization.
Conclusion

• Genetic Algorithms are a powerful and flexible optimization tool that

mimic evolutionary processes to solve complex problems. Their
adaptability and robustness make them suitable for a wide range of
real-world applications, especially when the search space is large and
poorly understood.
Thank you