Chapter 5 Soft Computing
Chapter 5 Soft Computing
• Basic concepts
• Genetic algorithm are good at taking larger, potential huge, search space and navigating them looking for
optimal combinations of things and solutions which we might not find in a life time.
• Genetic algorithms (GAs) are optimization and search algorithms inspired by the process of natural
selection and evolution. They are used to find approximate solutions to complex optimization and search
problems.
• A genetic algorithm is an adaptive heuristic search algorithm inspired by "Darwin's theory of evolution in
Nature.
• Are different from traditional optimization method • GA processes a number of designs at the same
• Design space converted to genetic space time.
• It work with coding of variables • GA possess number of designs at the same time.
• GA uses a population of points at one time • GA uses randomized operators for improvement
• Coding discretizes the search space
Chapter 2 Soft Computing 10/29/2023 2
Genetic Algorithms Fundamentals of GA: Basic concepts, creation of offspring, working principle, encoding,
fitness function, reproduction, crossover and mutation, numerical example of Genetic algorithm, parallel genetic
algorithm.
• Basic concepts
Genetic Algorithm Neural Network
1 A genetic algorithm (GA) is a search technique used in Neural networks are non-linear statistical data modeling
computing to find exact or approximate solutions to tools. They can be used to model complex relationships
optimization and search problems between inputs and outputs or to find patterns in data.
2 If you have a problem where you can quantify the worth When you have a number of items in different classes, a
of a solution, a genetic algorithm can perform a directed neural network can "learn" to classify items it has not
search of the solution space. (E.g. find the shortest route "seen" before. (E.g. face recognition, voice recognition)
between two points)
3 A genetic algorithm takes a long time to find an acceptable A neural network takes a long time to "learn", but then it
solution. can almost instantly classify new inputs.
4 Genetic algorithms are an optimization technique Neural Networks are used for regression/classification
5 Genetic algorithms (usually) work on discrete data Neural networks, on the other hand, (usually) work on
Chapter 2 Soft Computing continuous data 10/29/2023 3
Genetic Algorithms Fundamentals of GA: Basic concepts, creation of offspring, working principle, encoding,
fitness function, reproduction, crossover and mutation, numerical example of Genetic algorithm, parallel genetic
algorithm.
• Basic concepts
Parameter Genetic Algorithm Traditional Algorithm
Search are inspired by the principles of biological follow a deterministic and rule-based
Strategy evolution and natural selection. use a approach to solving problems. They
population-based search strategy that often rely on a predefined set of rules,
involves creating a population of potential equations, or heuristics to iteratively
solutions (chromosomes), evolving these manipulate data or perform
solutions over generations through computations to find an optimal or
selection, crossover, and mutation satisfactory solution
operations, and favoring individuals with
higher fitness.
Chapter 2 Soft Computing 10/29/2023 4
Genetic Algorithms Fundamentals of GA: Basic concepts, creation of offspring, working principle, encoding,
fitness function, reproduction, crossover and mutation, numerical example of Genetic algorithm, parallel genetic
algorithm.
• Basic concepts
Parameter Genetic Algorithm Traditional Algorithm
Exploration vs. emphasize exploration, searching the tend to focus more on exploitation,
Exploitation solution space broadly to discover refining the current solution based on
diverse candidate solutions. well-defined rules and heuristics.
Solution can handle a wide range of problem typically assume a fixed, explicit
Representation representations, including binary representation of the problem's
strings, real-valued vectors, variables and constraints
permutations, and more.
• Basic concepts
Parameter Genetic Algorithm Traditional Algorithm
• Basic concepts
Advantages Disadvantages
• The parallel capabilities of genetic algorithms • Genetic algorithms are not
are best. efficient algorithms for solving
• It helps in optimizing various problems such as simple problems.
discrete functions, multi-objective problems, • It does not guarantee the quality
and continuous functions. of the final solution to a
• It provides a solution for a problem that problem.
improves over time. • Repetitive calculation of fitness
• A genetic algorithm does not need derivative values may generate some
information. computational challenges.
Chapter 2 Soft Computing 10/29/2023 7
Genetic Algorithms Fundamentals of GA: Basic concepts, creation of offspring, working principle, encoding,
fitness function, reproduction, crossover and mutation, numerical example of Genetic algorithm, parallel genetic
algorithm.
• Basic concepts
• Basic concepts
Encoding
Encoding of chromosomes is the first question to ask when starting to solve a problem with GA.
Encoding depends on the problem heavily.
In this chapter some encodings will be introduced that have been already used with some success
Binary Encoding
Binary encoding is the most common one, mainly because the first research of GA used this type of
encoding and because of its relative simplicity.
In binary encoding, every chromosome is a string of bits - 0 or 1.
Binary encoding gives many possible chromosomes even with a small number of alleles. On the
other hand, this encoding is often not natural for many problems and sometimes corrections must be
made after crossover and/or mutation
Example of Problem: Finding a function that would approximate given pairs of values
The problem: Input and output values are given. The task is to find a function that will give
the best outputs (i.e. the closest to the wanted ones) for all inputs.
( do_until step wall )
Encoding: Chromosome are functions represented in a tree.
Chapter 2 Soft Computing 10/29/2023 12
Genetic Algorithms Fundamentals of GA: Basic concepts, creation of offspring, working principle, encoding,
fitness function, reproduction, crossover and mutation, numerical example of Genetic algorithm, parallel genetic
algorithm.
Encoding
Value Encoding
Direct value encoding can be used in problems where some more complicated values such as real numbers are
used. Use of binary encoding for this type of problems would be difficult.
In the value encoding, every chromosome is a sequence of some values. Values can be anything connected to
the problem, such as (real) numbers, chars or any objects
Value encoding is a good choice for some special problems. However, for this encoding it is often necessary to
develop some new crossover and mutation specific for the problem.
Example of Problem: Finding weights for a neural network
The problem: A neural network is given with defined architecture. Find weights between neurons in the neural
network to get the desired output from the network.
Encoding: Real values in chromosomes represent weights in the neural network
Encoding
Permutation Encoding
Permutation encoding can be used in ordering problems, such as travelling salesman problem or
task ordering problem.
In permutation encoding, every chromosome is a string of numbers that represent a position in a
sequence.
Permutation encoding is useful for ordering problems. For some types of crossover and mutation
corrections must be made to leave the chromosome consistent (i.e. have real sequence in it) for
some problems.
Selection
Introduction
As you already know from the GA outline, chromosomes are selected from the population to be parents
for crossover. The problem is how to select these chromosomes. According to Darwin's theory of
evolution the best ones survive to create new offspring. There are many methods in selecting the best
chromosomes. Examples are roulette wheel selection, Boltzman selection, tournament selection, rank
selection, steady state selection and some others.
• Creation of offspring
• The creation of offspring involves combining genetic information from
two parent individuals to produce one or more new individuals, known as
offspring or children.
• Mimics the principles of natural selection and genetic evolution to search
for optimal solutions to a problem.
• The new created offspring can then be muted (DNA modification)
• Motivated by a hope that new offspring will be better than the old one.
• The more suitable they are, the more chances they have to reproduce.
• This continues until iterations or best possible solution is met.
• Crossover and mutation are two basic operators of GA. Performance of GA depends on
them very much. The type and implementation of operators depends on the encoding and
also on the problem.
• There are many ways how to perform crossover and mutation. In this chapter we briefly
describe some examples and suggestions how to perform them several enco
• Binary Encoding
• Crossover
• Single point crossover - one crossover point is selected, binary string from the beginning of
the chromosome to the crossover point is copied from the first parent, the rest is copied
from the other parent
Parameters of GA
Basic recommendations if you have decided to implement your genetic algorithm. These recommendations are
very general. You will probably want to experiment with your own GA for a specific problem, because there is
no general theory available that would help you to tune GA parameters for any problem.
• Recommendations are often results of empiric studies of GAs that were often performed on binary encoding
only.
• Crossover rate
• Crossover rate should be high generally, about 80%-95%. (However some results show that for some problems crossover
rate about 60% is the best.)
• Mutation rate On the other side, mutation rate should be very low. Best rates seems to be about 0.5%-1%.
• Population size
• It may be surprising, that very big population size usually does not improve performance of GA (in the sense of speed of
finding solution). Good population size is about 20-30, however sometimes sizes 50-100 are reported as the best. Some
research also shows, that the best population size depends on the size of encoded string (chromosomes). It means that if you
have chromosomes with 32 bits, the population should be higher than for chromosomes with 16 bits.
Parameters of GA
• Selection
• Basic roulette wheel selection can be used, but sometimes rank selection can be better. Check the
chapter about selection for advantages and disadvantages. There are also some more
sophisticated methods that change parameters of selection during the run of GA. Basically, these
behave similarly like simulated annealing. Elitism should be used for sure if you do not use other
method for saving the best found solution. You can also try steady state selection.
• Encoding
• Encoding depends on the problem and also on the size of instance of the problem. Check the
chapter about encoding for some suggestions or look to other resources.
• Crossover and mutation type
• Operators depend on the chosen encoding and on the problem. Check the chapter about
operators for some suggestions. You can also check other sites.
Applications of GA
Genetic algorithms have been used for difficult problems (such as NP-hard problems),
for machine learning and also for evolving simple programs. They have been also used
for some art, for evolving pictures and music.
The advantage of GAs is in their parallelism. GA is travelling in a search space using
more individuals (and with genotype rather than phenotype) so that they are less likely to
get stuck in a local extreme like the other methods.
They are also easy to implement. Once you have the basic GA algorithm implemented,
you have just to write a new chromosome (just one object) to solve another problem.
With the same encoding you just change the fitness function - and you are done.
However, for some problems, choosing and implementation of encoding and fitness
function can be difficult.
Applications of GA
The disadvantage of GAs is in the computational time. GAs can be slower than other methods. But sice we can
terminate the computation in any time, the longer run is acceptable (especially with faster and faster computers).
To get an idea about some problems solved by GAs, here is a short list of some applications:
Nonlinear dynamical systems - predicting, data analysis
Designing neural networks, both architecture and weights
Robot trajectory
Evolving LISP programs (genetic programming)
Strategy planning
Finding shape of protein molecules
TSP and sequence scheduling
Functions for creating images
σ𝑛
𝑖=1 𝑓 𝑥
• (𝐴𝑉𝐺. 𝑓 𝑥) 𝑖 = [ 𝑛𝑖
𝑖
] = 1155/4=288.75
• For string 1 Expected count = 144/ 288.75 = 0.4987
• Similarly for other strings 2.1645, 0.0866, 1.2502
Chapter 2 Soft Computing 10/29/2023 29
Genetic Algorithms Fundamentals of GA: Basic concepts, creation of offspring, working principle, encoding,
fitness function, reproduction, crossover and mutation, numerical example of Genetic algorithm, parallel genetic
algorithm.
54%