Genetic

Algorithms
Classic Example of
Jonathon
Step 1 Create Sample
Population
Step 2: Design a fitness
Function
Step 3: Select the
Fittest
Step 4: Crossover &
Mutation
Crossover
Problem ???
Mutation
Jonathon at Work!!!
Introduction
After scientists became disillusioned with classical and neo-classical
attempts at modeling intelligence, they looked in other directions.
Two prominent fields arose, connectionism (neural networking, parallel
processing) and evolutionary computing.
1. Objective to survive

2. These algorithms are considered as a subfield of Computational Intelligence (CI) and Artificial Intelligence (AI)

3. Optimization problems, mostly having a black-box objective function (Maximization and Minimization Problems)
 Genetic
Algorithm
 What is GA
A genetic algorithm (or GA) is a search technique used
in computing to find true or approximate solutions to
optimization and search problems.
Genetic algorithms are categorized as global search
heuristics.
Genetic algorithms are a particular class of evolutionary
algorithms that use techniques inspired by evolutionary
biology such as inheritance, mutation, selection, and
crossover (also called recombination).
What is GA
Genetic algorithms are implemented as a computer
simulation in which a population of abstract
representations (called chromosomes or the genotype
or the genome) of candidate solutions (called
individuals, creatures, or phenotypes) to an
optimization problem evolves toward better solutions.

Traditionally, solutions are represented in binary as

strings of 0s and 1s, but other encodings are also
possible.
What is GA
The evolution usually starts from a population of
randomly generated individuals and happens in
generations.

In each generation, the fitness of every individual in

the population is evaluated, multiple individuals
are selected from the current population (based on
their fitness), and modified (recombined and
possibly mutated) to form a new population.
 What is GA
The new population is then used in the next iteration of the algorithm.
Commonly, the algorithm terminates when either a maximum number of
generations has been produced, or a satisfactory fitness level has been
reached for the population.
If the algorithm has terminated due to a maximum number of generations, a
satisfactory solution may or may not have been reached.
Key terms
Individual - Any possible solution
Population - Group of all individuals
Search Space - All possible solutions to the problem
Chromosome - Blueprint for an individual
Trait - Possible aspect (features) of an individual
Allele - Possible settings of trait (black, blond, etc.)
Locus - The position of a gene on the chromosome
Genome - Collection of all chromosomes for an
individual
Chromosome, Genes and
Genomes
Genotype and
Phenotype
Genotype:
– Particular set of genes in a genome

Phenotype:
– Physical characteristic of the genotype (smart, beautiful, healthy, etc.)
Genotype and Phenotype
Genetic Algorithms
An Example Genetic Algorithm
Procedure GA{

t = 0;

Initialize P(t);

Evaluate P(t);

While (Not Done)

Parents(t) = Select_Parents(P(t));

Offspring(t) = Procreate(Parents(t));

Evaluate(Offspring(t));

P(t+1)= Select_Survivors(P(t),Offspring(t));

t = t + 1;

}
Genetic Algorithms
Representation of Candidate Solutions
GAs on primarily two types of representations:
◦ Binary-Coded
◦ Real-Coded
Binary-Coded GAs must decode a chromosome into a CS,
evaluate the CS and return the resulting fitness back to the
binary-coded chromosome representing the evaluated CS.
Genetic Algorithms:
Binary-Coded Representations
For Example, let’s say that we are trying to optimize the
following function,
◦ f(x) = x2
◦ for 2  x  1

If we were to use binary-coded representations we would

first need to develop a mapping function form our genotype
representation (binary string) to our phenotype
representation (our CS). This can be done using the
following mapping function:
◦ d(ub,lb,l,chrom) = (ub-lb) decode(chrom)/2l-1 + lb
Genetic Algorithms:
Binary-Coded Representations
d(ub,lb,l,c) = (ub-lb) decode(c)/2l-1 + lb , where
◦ ub = 2,
◦ lb = 1,
◦ l = the length of the chromosome in bits
◦ c = the chromosome

The parameter, l, determines the accuracy (and resolution

of our search).
What happens when l is increased (or decreased)?
Genetic Algorithms:
Binary Coded Representations

Individual Individual

Chromosome: 00101 Chromosome: 00101

Fitness = ????? Fitness = 1.35

d(2,1,5,00101) = 1.16 f(1.16) = 1.35

The Fitness Assignment Process for Binary Coded

Ch ro mosomes (ub=2, lb=1, l=5)
Genetic Algorithms:
Real-Coded Representations
Real-Coded GAs can be regarded as GAs that operate on the
actual CS (phenotype).
For Real-Coded GAs, no genotype-to-phenotype mapping is
needed.
Genetic Algorithms:
Real-Coded Representations

Individual Individual

Chromosome: 1.16 Chromosome: 1.16

Fitness = ????? Fitness = 1.35

f(1.16) = 1.35

The Fitness Assignment Process for Real Coded Ch ro mosomes

Genetic Algorithms:
Parent Selection Methods
An Example Genetic Algorithm
Procedure GA{

t = 0;

Initialize P(t);

Evaluate P(t);

While (Not Done)

Parents(t) = Select_Parents(P(t));

Offspring(t) = Procreate(Parents(t));

Evaluate(Offspring(t));

P(t+1)= Select_Survivors(P(t),Offspring(t));

t = t + 1;

}
Genetic Algorithms:
Parent Selection Methods
GA researchers have used a number of parent selection
methods. Some of the more popular methods are:
◦ Proportionate Selection
◦ Linear Rank Selection
◦ Tournament Selection
Genetic Algorithms:
Proportionate Selection
In Proportionate Selection, individuals are assigned a
probability of being selected based on their fitness:
◦ pi = fi / fj
◦ Where pi is the probability that individual i will be selected,
◦ fi is the fitness of individual i, and
◦ fj represents the sum of all the fitnesses of the individuals with the
population.
This type of selection is similar to using a roulette wheel
where the fitness of an individual is represented as
proportionate slice of wheel. The wheel is then spun and
the slice underneath the wheel when it stops determine
which individual becomes a parent.
Genetic Algorithms:
Proportionate Selection
There are a number of disadvantages associated with using
proportionate selection:
◦ Cannot be used on minimization problems,
◦ Loss of selection pressure (search direction) as population converges,
◦ Susceptible to Super Individuals
Genetic Algorithms:
Linear Rank Selection
In Linear Rank selection, individuals are assigned subjective
fitness based on the rank within the population:
◦ sfi = (P-ri)(max-min)/(P-1) + min
◦ Where ri is the rank of indvidual i,
◦ P is the population size,
◦ Max represents the fitness to assign to the best individual,
◦ Min represents the fitness to assign to the worst individual.

pi = sfi / sfj Roulette Wheel Selection can be performed

using the subjective fitnesses.
One disadvantage associated with linear rank selection is
that the population must be sorted on each cycle.
Genetic Algorithms:
Tournament Selection
In Tournament Selection, q individuals are randomly
selected from the population and the best of the q
individuals is returned as a parent.
Selection Pressure increases as q is increased and decreases
a q is decreased.
Genetic Algorithms:
Genetic Procreation Operators
An Example Genetic Algorithm
Procedure GA{

t = 0;

Initialize P(t);

Evaluate P(t);

While (Not Done)

Parents(t) = Select_Parents(P(t));

Offspring(t) = Procreate(Parents(t));

Evaluate(Offspring(t));

P(t+1)= Select_Survivors(P(t),Offspring(t));

t = t + 1;

}
Genetic Algorithms:
Genetic Procreation Operators
Genetic Algorithms typically use two types of operators:
◦ Crossover (Sexual Recombination), and
◦ Mutation (Asexual)

Crossover is usually the primary operator with mutation

serving only as a mechanism to introduce diversity in the
population.
However, when designing a GA to solve a problem it is not
uncommon that one will have to develop unique crossover
and mutation operators that take advantage of the
structure of the CSs comprising the search space.
Genetic Algorithms:
Genetic Procreation Operators
However, there are a number of crossover operators that
have been used on binary and real-coded GAs:
◦ Single-point Crossover,
◦ Two-point Crossover,
◦ Uniform Crossover
Genetic Algorithms:
Single-Point Crossover
Given two parents, single-point crossover will generate a
cut-point and recombines the first part of first parent with
the second part of the second parent to create one
offspring.
Single-point crossover then recombines the second part of
the first parent with the first part of the second parent to
create a second offspring.
Genetic Algorithms:
Single-Point Crossover
Example:
◦ Parent 1: XX|XXXXX
◦ Parent 2: YY|YYYYY
◦ Offspring 1: X X Y Y Y Y Y
◦ Offspring 2: Y Y X X X X X
Genetic Algorithms:
Two-Point Crossover
Two-Point crossover is very similar to single-point crossover
except that two cut-points are generated instead of one.
Genetic Algorithms:
Two-Point Crossover
Example:
◦ Parent 1: XX|XXX|XX
◦ Parent 2: YY|YYY|YY
◦ Offspring 1: X X Y Y Y X X
◦ Offspring 2: Y Y X X X Y Y
Genetic Algorithms:
Uniform Crossover
In Uniform Crossover, a value of the first parent’s gene is
assigned to the first offspring and the value of the second
parent’s gene is to the second offspring with probability 0.5.
With probability 0.5 the value of the first parent’s gene is
assigned to the second offspring and the value of the
second parent’s gene is assigned to the first offspring.
Genetic Algorithms:
Uniform Crossover
Example:
◦ Parent 1: XXXXXXX
◦ Parent 2: YYYYYYY
◦ Offspring 1: X Y X Y Y X Y
◦ Offspring 2: Y X Y X X Y X
Genetic Algorithms:
Real-Coded Crossover Operators
For Real-Coded representations there exist a number of
other crossover operators:
◦ Mid-Point Crossover,
◦ Flat Crossover (BLX-0.0),
◦ BLX-0.5
Genetic Algorithms:
Mid-Point Crossover
Given two parents where X and Y represent a floating point
number:
◦ Parent 1: X
◦ Parent 2: Y
◦ Offspring: (X+Y)/2

If a chromosome contains more than one gene, then this

operator can be applied to each gene with a probability of
Pmp.
Genetic Algorithms:
Flat Crossover (BLX-0.0)
Flat crossover was developed by Radcliffe (1991)
Given two parents where X and Y represent a floating point
number:
◦ Parent 1: X
◦ Parent 2: Y
◦ Offspring: rnd(X,Y)

Of course, if a chromosome contains more than one gene

then this operator can be applied to each gene with a
probability of Pblx-0.0.
Genetic Algorithms:
BLX-
Developed by Eshelman & Schaffer (1992)
Given two parents where X and Y represent a floating point
number, and where X < Y:
◦ Parent 1: X
◦ Parent 2: Y
◦ Let  = (Y-X), where  = 0.5
◦ Offspring: rnd(X-, Y+ )

Of course, if a chromosome contains more than one gene

then this operator can be applied to each gene with a
probability of Pblx-.
Genetic Algorithms:
Mutation (Binary-Coded)
In Binary-Coded GAs, each bit in the chromosome is
mutated with probability pbm known as the mutation rate.

Parent1 1 0 0 0 0 1 0
Parent2 1 1 1 0 0 0 1

Child1 1 0 0 1 0 0 1
Child2 0 1 1 0 1 1 0

An Exa mple of Sing le-point Crossover Between the

Third and Fourth Genes with a Mutation Rate of
0.01 Applied to Bina ry Coded Ch ro mosomes
Genetic Algorithms:
Mutation (Real-Coded)
In real-coded GAs, Gaussian mutation can be used.
For example, BLX-0.0 Crossover with Gaussian mutation.
Given two parents where X and Y represent a floating point
number:
◦ Parent 1: X
◦ Parent 2: Y
◦ Offspring: rnd(X,Y) + N(0,1)
Genetic Algorithm:
Selecting Who Survives
An Example Genetic Algorithm
Procedure GA{

t = 0;

Initialize P(t);

Evaluate P(t);

While (Not Done)

Parents(t) = Select_Parents(P(t));

Offspring(t) = Procreate(Parents(t));

Evaluate(Offspring(t));

P(t+1)= Select_Survivors(P(t),Offspring(t));

t = t + 1;

}
Genetic Algorithms:
Selection Who Survives
Basically, there are two types of GAs commonly used.
These GAs are characterized by the type of replacement
strategies they use.
A Generational GA uses a (,) replacement strategy where
the offspring replace the parents.
A Steady-State GA usually will select two parents, create 1-2
offspring which will replace the 1-2 worst individuals in the
current population even if the offspring are worse than the
individuals they replace.
This slightly different than (+1) or (+2) replacement.
Genetic Algorithm:
Example by Hand
Now that we have an understanding of the various parts of
a GA let’s evolve a simple GA (SGA) by hand.
A SGA is :
◦ binary-coded,
◦ Uses proportionate selection
◦ uses single-point crossover (with a crossover usage rate between
0.6-1.0),
◦ uses a small mutation rate, and
◦ is generational.
Genetic Algorithms:
Example
The SGA for our example will use:
◦ A population size of 6,
◦ A crossover usage rate of 1.0, and
◦ A mutation rate of 1/7.

Let’s try to solve the following problem

◦ f(x) = x2, where -2.0  x  2.0,
◦ Let l = 7, therefore our mapping function will be
◦ d(2,-2,7,c) = 4*decode(c)/127 - 2
Genetic Algorithms:
An Example Run (by hand)
Randomly Generate an Initial Population
Genotype Phenotype Fitness
Person 1: 1001010 0.331 Fit: ?
Person 2: 0100101 - 0.835 Fit: ?
Person 3: 1101010 1.339 Fit: ?
Person 4: 0110110 - 0.300 Fit: ?
Person 5: 1001111 0.488 Fit: ?
Person 6: 0001101 - 1.591 Fit: ?
Genetic Algorithms:
An Example Run (by hand)
Evaluate Population at t=0
Genotype Phenotype Fitness
Person 1: 1001010 0.331 Fit: 0.109
Person 2: 0100101 - 0.835 Fit: 0.697
Person 3: 1101010 1.339 Fit: 1.790
Person 4: 0110110 - 0.300 Fit: 0.090
Person 5: 1001111 0.488 Fit: 0.238
Person 6: 0001101 - 1.591 Fit: 2.531
Genetic Algorithms:
An Example Run (by hand)
Select Six Parents Using the Roulette Wheel
Genotype Phenotype Fitness
Person 6: 0001101 - 1.591 Fit: 2.531
Person 3: 1101010 1.339 Fit: 1.793
Person 5: 1001111 0.488 Fit: 0.238
Person 6: 0001101 - 1.591 Fit: 2.531
Person 2: 0100101 - 0.835 Fit: 0.697
Person 1: 1001010 0.331 Fit: 0.109
Genetic Algorithms:
An Example Run (by hand)
Create Offspring 1 & 2 Using Single-Point Crossover
Genotype Phenotype Fitness
Person 6: 00|01101 - 1.591Fit: 2.531
Person 3: 11|01010 1.339 Fit: 1.793
Child 1 : 0001010 - 1.685 Fit: ?
Child 2 : 1101101 1.433 Fit: ?
Genetic Algorithms:
An Example Run (by hand)
Create Offspring 3 & 4
Genotype Phenotype Fitness
Person 5: 1001|111 0.488 Fit: 0.238
Person 6: 0001|101 - 1.591Fit: 2.531
Child 3 : 1011100 0.898 Fit: ?
Child 4 : 0001011 - 1.654 Fit: ?
Genetic Algorithms:
An Example Run (by hand)
Create Offspring 5 & 6
Genotype Phenotype Fitness
Person 2: 010|0101 - 0.835Fit: 0.697
Person 1: 100|1010 0.331 Fit: 0.109
Child 5 : 1101010 1.339 Fit: ?
Child 6 : 1010101 0.677 Fit: ?
Genetic Algorithms:
An Example Run (by hand)
Evaluate the Offspring
Genotype Phenotype Fitness
Child 1 : 0001010 - 1.685 Fit: 2.839
Child 2 : 1101101 1.433 Fit: 2.054
Child 3 : 1011100 0.898 Fit: 0.806
Child 4 : 0001011 - 1.654 Fit: 2.736
Child 5 : 1101010 1.339 Fit: 1.793
Child 6 : 1010101 0.677 Fit: 0.458
 Genetic Algorithms:
An Example Run (by hand)
Population at t=0

Genotype Phenotype Fitness

Person 1: 1001010 0.331 Fit: 0.109

Person 2: 0100101 - 0.835 Fit: 0.697

Person 3: 1101010 1.339 Fit: 1.793

Person 4: 0110110 - 0.300 Fit: 0.090

Person 5: 1001111 0.488 Fit: 0.238

Person 6: 0001101 - 1.591 Fit: 2.531

Is Replaced by:

Genotype Phenotype Fitness

Child 1 : 0001010 - 1.685 Fit: 2.839

Child 2 : 1101101 1.433 Fit: 2.053

Child 3 : 1011100 0.898 Fit: 0.806

Child 4 : 0001011 - 1.654 Fit: 2.736

Child 5 : 1101010 1.339 Fit: 1.793

Child 6 : 1010101 0.677 Fit: 0.458

Genetic Algorithms:
An Example Run (by hand)
Population at t=1
Genotype Phenotype Fitness
Person 1: 0001010 - 1.685 Fit: 2.839
Person 2: 1101101 1.433 Fit: 2.054
Person 3: 1011100 0.898 Fit: 0.806
Person 4: 0001011 - 1.654 Fit: 2.736
Person 5: 1101010 1.339 Fit: 1.793
Person 6: 1010101 0.677 Fit: 0.458
Genetic Algorithms:
An Example Run (by hand)
The Process of:
◦ Selecting six parents,
◦ Allowing the parents to create six offspring,
◦ Mutating the six offspring,
◦ Evaluating the offspring, and
◦ Replacing the parents with the offspring

Is repeated until a stopping criterion has been reached.

Genetic Algorithms:
An Example Run (Steady-State GA)
Randomly Generate an Initial Population
Genotype Phenotype Fitness
Person 1: 1001010 0.331 Fit: ?
Person 2: 0100101 - 0.835 Fit: ?
Person 3: 1101010 1.339 Fit: ?
Person 4: 0110110 - 0.300 Fit: ?
Person 5: 1001111 0.488 Fit: ?
Person 6: 0001101 - 1.591 Fit: ?
Genetic Algorithms:
An Example Run (Steady-State GA)
Evaluate Population at t=0
Genotype Phenotype Fitness
Person 1: 1001010 0.331 Fit: 0.109
Person 2: 0100101 - 0.835 Fit: 0.697
Person 3: 1101010 1.339 Fit: 1.790
Person 4: 0110110 - 0.300 Fit: 0.090
Person 5: 1001111 0.488 Fit: 0.238
Person 6: 0001101 - 1.591 Fit: 2.531
Genetic Algorithms:
An Example Run (Steady-State GA)
Select 2 Parents and Create 2 Using Single-Point Crossover
Genotype Phenotype Fitness
Person 6: 00|01101 - 1.591Fit: 2.531
Person 3: 11|01010 1.339 Fit: 1.793
Child 1 : 0001010 - 1.685 Fit: ?
Child 2 : 1101101 1.433 Fit: ?
Genetic Algorithms:
An Example Run (Steady-State GA)
Evaluate the Offspring
Genotype Phenotype Fitness
Child 1 : 0001010 - 1.685 Fit: 2.839
Child 2 : 1101101 1.433 Fit: 2.054
Genetic Algorithms:
An Example Run (Steady-State GA)
Find the two worst individuals to be replaced
Genotype Phenotype Fitness
Person 1: 1001010 0.331 Fit: 0.109
Person 2: 0100101 - 0.835 Fit: 0.697
Person 3: 1101010 1.339 Fit: 1.790
Person 4: 0110110 - 0.300 Fit: 0.090
Person 5: 1001111 0.488 Fit: 0.238
Person 6: 0001101 - 1.591 Fit: 2.531
Genetic Algorithms:
An Example Run (Steady-State GA)
Replace them with the offspring
Genotype Phenotype Fitness
Person 1: 1001010 0.331 Fit: 0.109
Child 1 : 0001010 - 1.685 Fit: 2.839
Person 3: 1101010 1.339 Fit: 1.790
Child 2 : 1101101 1.433 Fit: 2.054
Person 5: 1001111 0.488 Fit: 0.238
Person 6: 0001101 - 1.591 Fit: 2.531
Genetic Algorithms:
An Example Run (Steady-State GA)
This process of:
◦ Selecting two parents,
◦ Allowing them to create two offspring, and
◦ Immediately replacing the two worst individuals in the population with the
offspring
Is repeated until a stopping criterion is reached
Notice that on each cycle the steady-state GA will make two function
evaluations while a generational GA will make P (where P is the
population size) function evaluations.
Therefore, you must be careful to count only function evaluations when
comparing generational GAs with steady-state GAs.
Genetic Algorithms:
Additional Properties
Generation Gap: The fraction of the population that is
replaced each cycle. A generation gap of 1.0 means that the
whole population is replaced by the offspring. A generation
gap of 0.01 (given a population size of 100) means
______________.
Elitism: The fraction of the population that is guaranteed to
survive to the next cycle. An elitism rate of 0.99 (given a
population size of 100) means ___________ and an elitism
rate of 0.01 means _______________.
Genetic Algorithms:
“Wake-up Neo, It’s Schema Theorem
Time!”
The Schema Theorem was developed by John Holland in an
attempt to explain the quickness and efficiency of genetic
search (for a Simple Genetic Algorithm).
His explanation was that GAs operate on large number of
schemata, in parallel. These schemata can be seen as
building-blocks. Thus, GAs solves problems by assembling
building blocks similar to the way a child build structures
with building blocks.
This explanation is known as the “Building-Block
Hypothesis”.
Genetic Algorithms:
The Schema Theorem
Schema Theorem Terminology:
◦ A schema is a similarity template that represents a number of
genotypes.
◦ Let H = #1##10 be a schema.
◦ Schemata have a base which is the cardinality of their largest domain
of values (alleles).
◦ Binary coded chromosomes have a base of 2. Therefore the alphabet
for a schema is taken from the set {#,0,1} where # represents the
don’t care symbol.
◦ Schema H represents 8 unique individuals. How do we know this?
Genetic Algorithms:
The Schema Theorem
Schema Theorem Terminology (Cont.):
◦ if H = #1##10,
◦ Let (H) represent the defining length of H, which is measured by the
outermost non-wildcard values.
◦ Therefore, (H) = 6-2 = 4.
◦ Let o(H) represent the order of H, which is the number of non-
wildcard values.
◦ Thus, o(H) = 3.
◦ Therefore schema H will represent 2l-o(H) individuals.
Genetic Algorithms:
The Schema Theorem
Schema Theorem Terminology (Cont.):
◦ Let m(H,t) denoted the number of instances of H that are in the
population at time t.
◦ Let f(H,t) denote the average fitness of the instances of H that are in
the population at time t.
◦ Let favg(t) represent the average fitness of the population at time t.
◦ Let pc and pm represent the single-point crossover and mutation
rates.
◦ According to the Schema Theorem there will be:
◦ m(H,t+1) = m(H,t) f(H,t)/favg(t) instances of H in the next population if H has
an above average fitness.
Genetic Algorithms:
The Schema Theorem
Schema Theorem Terminology (Cont.):
◦ According to the Schema Theorem there will be:
◦ m(H,t+1) = m(H,t) f(H,t)/favg(t)
◦ instances of H in the next population if H has an above average
fitness.
◦ If we let f(H,t) = favg(t) + c favg(t), for some c > 0, then
◦ m(H,t+1) = m(H,t)(1+c), and
◦ If m(H,0) > 0 then we can rewrite the equation as
◦ m(H,t) = m(H,0)(1+c)t
◦ What this says is that proportionate selection allocates an
exponentially increasing number of trials to above average
schemata.
Genetic Algorithms:
The Schema Theorem
Schema Theorem Terminology (Cont.):
◦ m(H,t+1) = m(H,t) f(H,t)/favg(t)
◦ Although the above equation seems to say that above average
schemata are allowed an exponentially increasing number of trials,
instances may be gained or lost through the application of single-
point crossover and mutation.
◦ Thus we need to calculate the probability that schema H survives:
◦ Single-Point Crossover: Sc(H) =1- [pc (H)/(l-1)]
◦ Mutation: Sm(H) = (1- pm )o(H)
Genetic Algorithms:
The Schema Theorem
Schema Theorem:
m(H,t+1)  m(H,t) f(H,t)/favg(t) Sc(H) Sm(H)
It proposes that the type of schemata to gain instances will be
those with:
◦ Above average fitness,
◦ Low defining length, and
◦ Low order

But does this really tell us how SGAs search?

Do SGAs allow us to get something (implicit parallelism) for
nothing (perhaps a Free Lunch)?
This lecture was based on: G. Dozier, A. Homaifar, E. Tunstel, and D. Battle, "An Introduction
to Evolutionary Computation" (Chapter 17), Intelligent Control Systems Using Soft Computing Methodologies, A.
Zilouchian & M. Jamshidi (Eds.), pp. 365-380, CRC press. (can be found at:
www.eng.auburn.edu/~gvdozier/chapter17.doc)
 GA Requirements
A typical genetic algorithm requires two things to be defined:
a genetic representation of the solution domain, and
a fitness function to evaluate the solution domain.

A standard representation of the solution is as an array of bits.

Arrays of other types and structures can be used in essentially
the same way.
The main property that makes these genetic representations
convenient is that their parts are easily aligned due to their fixed
size, that facilitates simple crossover operation.
Variable length representations may also be used, but crossover
implementation is more complex in this case.
Tree-like representations are explored in Genetic programming.
Representation
Chromosomes could be:
◦ Bit strings (0101 ... 1100)
◦ Real numbers (43.2 -33.1 ... 0.0 89.2)
◦ Permutations of element (E11 E3 E7 ... E1 E15)
◦ Lists of rules (R1 R2 R3 ... R22 R23)
◦ Program elements (genetic programming)
◦ ... any data structure ...
 GA Requirements
The fitness function is defined over the genetic representation
and measures the quality of the represented solution.
The fitness function is always problem dependent.
For instance, in the knapsack problem we want to maximize the
total value of objects that we can put in a knapsack of some fixed
capacity.
A representation of a solution might be an array of bits, where
each bit represents a different object, and the value of the bit (0
or 1) represents whether or not the object is in the knapsack.
Not every such representation is valid, as the size of objects may
exceed the capacity of the knapsack.
The fitness of the solution is the sum of values of all objects in the
knapsack if the representation is valid, or 0 otherwise. In some
problems, it is hard or even impossible to define the fitness
expression; in these cases, interactive genetic algorithms are
used.
A fitness function
 Basics of GA
The most common type of genetic algorithm works like this:
a population is created with a group of individuals created
randomly.
The individuals in the population are then evaluated.
The evaluation function is provided by the programmer and gives
the individuals a score based on how well they perform at the
given task.
Two individuals are then selected based on their fitness, the
higher the fitness, the higher the chance of being selected.
These individuals then "reproduce" to create one or more
offspring, after which the offspring are mutated randomly.
This continues until a suitable solution has been found or a
certain number of generations have passed, depending on the
needs of the programmer.
 General Algorithm for
GA
Initialization
Initially many individual solutions are randomly
generated to form an initial population. The population
size depends on the nature of the problem, but typically
contains several hundreds or thousands of possible
solutions.
Traditionally, the population is generated randomly,
covering the entire range of possible solutions (the
search space).
Occasionally, the solutions may be "seeded" in areas
where optimal solutions are likely to be found.
 General Algorithm for
GA
Selection
During each successive generation, a proportion of the existing
population is selected to breed a new generation.
Individual solutions are selected through a fitness-based process,
where fitter solutions (as measured by a fitness function) are
typically more likely to be selected.
Certain selection methods rate the fitness of each solution and
preferentially select the best solutions. Other methods rate only
a random sample of the population, as this process may be very
time-consuming.
Most functions are stochastic and designed so that a small
proportion of less fit solutions are selected. This helps keep the
diversity of the population large, preventing premature
convergence on poor solutions. Popular and well-studied
selection methods include roulette wheel selection and
tournament selection.
 General Algorithm for
GA
In roulette wheel selection, individuals are given a probability of being
selected that is directly proportionate to their fitness.

Two individuals are then chosen randomly based on these probabilities and
produce offspring.
 General Algorithm for
GA
Roulette Wheel’s Selection Pseudo Code:

for all members of population

sum += fitness of this individual

end for

for all members of population

probability = sum of probabilities + (fitness / sum)

sum of probabilities += probability

end for

loop until new population is full

do this twice

number = Random between 0 and 1

for all members of population

if number > probability but less than next probability then

you have been selected

end for

end

create offspring

end loop
 General Algorithm for
GA
Reproduction
The next step is to generate a second generation population of
solutions from those selected through genetic operators:
crossover (also called recombination), and/or mutation.
For each new solution to be produced, a pair of "parent"
solutions is selected for breeding from the pool selected
previously.
By producing a "child" solution using the above methods of
crossover and mutation, a new solution is created which typically
shares many of the characteristics of its "parents". New parents
are selected for each child, and the process continues until a new
population of solutions of appropriate size is generated.
 General Algorithm for
GA
These processes ultimately result in the next generation population of
chromosomes that is different from the initial generation.

Generally the average fitness will have increased by this procedure for the
population, since only the best organisms from the first generation are
selected for breeding, along with a small proportion of less fit solutions, for
reasons already mentioned above.
 Crossover
the most common type is single point crossover. In single point
crossover, you choose a locus at which you swap the remaining
alleles from on parent to the other. This is complex and is best
understood visually.
As you can see, the children take one section of the chromosome
from each parent.
The point at which the chromosome is broken depends on the
randomly selected crossover point.
This particular method is called single point crossover because
only one crossover point exists. Sometimes only child 1 or child 2
is created, but oftentimes both offspring are created and put into
the new population.
Crossover does not always occur, however. Sometimes, based on
a set probability, no crossover occurs and the parents are copied
directly to the new population. The probability of crossover
occurring is usually 60% to 70%.
Crossover
 Mutation
After selection and crossover, you now have a new population
full of individuals.
Some are directly copied, and others are produced by crossover.
In order to ensure that the individuals are not all exactly the
same, you allow for a small chance of mutation.
You loop through all the alleles of all the individuals, and if that
allele is selected for mutation, you can either change it by a small
amount or replace it with a new value. The probability of
mutation is usually between 1 and 2 tenths of a percent.
Mutation is fairly simple. You just change the selected alleles
based on what you feel is necessary and move on. Mutation is,
however, vital to ensuring genetic diversity within the
population.
Mutation
 General Algorithm for
GA
Termination

This generational process is repeated until a termination

condition has been reached.
Common terminating conditions are:
◦ A solution is found that satisfies minimum criteria
◦ Fixed number of generations reached
◦ Allocated budget (computation time/money) reached
◦ The highest ranking solution's fitness is reaching or has
reached a plateau such that successive iterations no longer
produce better results
◦ Manual inspection
◦ Any Combinations of the above
GA Pseudo-code
Choose initial population
Evaluate the fitness of each individual in the population
Repeat

Select best-ranking individuals to reproduce

Breed new generation through crossover and mutation (genetic

operations) and give birth to offspring

Evaluate the individual fitnesses of the offspring

Replace worst ranked part of population with offspring

Until <terminating condition>

 Symbolic AI VS. Genetic
Algorithms
Most symbolic AI systems are very static.
Most of them can usually only solve one given specific
problem, since their architecture was designed for
whatever that specific problem was in the first place.
Thus, if the given problem were somehow to be
changed, these systems could have a hard time
adapting to them, since the algorithm that would
originally arrive to the solution may be either incorrect
or less efficient.
Genetic algorithms (or GA) were created to combat
these problems; they are basically algorithms based on
natural biological evolution.
 Symbolic AI VS. Genetic
Algorithms
The architecture of systems that implement genetic algorithms (or GA)
are more able to adapt to a wide range of problems.
A GA functions by generating a large set of possible solutions to a given
problem.
It then evaluates each of those solutions, and decides on a "fitness
level" (you may recall the phrase: "survival of the fittest") for each
solution set.
These solutions then breed new solutions.
The parent solutions that were more "fit" are more likely to reproduce,
while those that were less "fit" are more unlikely to do so.
In essence, solutions are evolved over time. This way you evolve your
search space scope to a point where you can find the solution.
Genetic algorithms can be incredibly efficient if programmed correctly.
 Genetic Programming
In programming languages such as LISP, the mathematical
notation is not written in standard notation, but in prefix
notation. Some examples of this:
+21 : 2+1
*+212 : 2 * (2+1)
* + - 2 1 4 9 : 9 * ((2 - 1) + 4)
Notice the difference between the left-hand side to the right?
Apart from the order being different, no parenthesis! The prefix
method makes it a lot easier for programmers and compilers
alike, because order precedence is not an issue.
You can build expression trees out of these strings that then can
be easily evaluated, for example, here are the trees for the above
three expressions.
Genetic Programming
 Genetic Programming
You can see how expression evaluation is thus a lot
easier.
What this have to do with GAs? If for example you have
numerical data and 'answers', but no expression to
conjoin the data with the answers.
A genetic algorithm can be used to 'evolve' an
expression tree to create a very close fit to the data.
By 'splicing' and 'grafting' the trees and evaluating the
resulting expression with the data and testing it to the
answers, the fitness function can return how close the
expression is.
 Genetic Programming
The limitations of genetic programming lie in the huge search space the GAs
have to search for - an infinite number of equations.
Therefore, normally before running a GA to search for an equation, the user
tells the program which operators and numerical ranges to search under.
Uses of genetic programming can lie in stock market prediction, advanced
mathematics and military applications .
Evolving Neural
Networks
Evolving the architecture of neural network is slightly more
complicated, and there have been several ways of doing it. For small
nets, a simple matrix represents which neuron connects which, and
then this matrix is, in turn, converted into the necessary 'genes', and
various combinations of these are evolved.
 Evolving Neural
Networks
Many would think that a learning function could be
evolved via genetic programming. Unfortunately,
genetic programming combined with neural networks
could be incredibly slow, thus impractical.
As with many problems, you have to constrain what you
are attempting to create.
For example, in 1990, David Chalmers attempted to
evolve a function as good as the delta rule.
He did this by creating a general equation based upon
the delta rule with 8 unknowns, which the genetic
algorithm then evolved.
 Other Areas
Genetic Algorithms can be applied to virtually any problem that has a
large search space.

Al Biles uses genetic algorithms to filter out 'good' and 'bad' riffs for jazz
improvisation.

The military uses GAs to evolve equations to differentiate between

different radar returns.

Stock companies use GA-powered programs to predict the stock

market.
Example
f(x) = {MAX(x2): 0 <= x <= 32 }
Encode Solution: Just use 5 bits (1 or 0).
Generate initial population.
A 0 1 1 0 1
B 1 1 0 0 0
C 0 1 0 0 0
D 1 0 0 1 1

Evaluate each solution against objective.

Sol. String Fitness % of Total
A 01101 169 14.4
B 11000 576 49.2
C 01000 64 5.5
D 10011 361 30.9
Example Cont’d
Create next generation of solutions
◦ Probability of “being a parent” depends on the fitness.
Ways for parents to create next generation
◦ Reproduction
◦ Use a string again unmodified.
◦ Crossover
◦ Cut and paste portions of one string to another.
◦ Mutation
◦ Randomly flip a bit.
◦ COMBINATION of all of the above.
 Checkboard example
◦ We are given an n by n checkboard in which every field
can have a different colour from a set of four colors.
◦ Goal is to achieve a checkboard in a way that there are
no neighbours with the same color (not diagonal)

1 1

2 2

3 3

4 4

5 5

6 6

7 7

8 8

9 9

10 10

1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10
Checkboard example
Cont’d
◦ Chromosomes represent the way the checkboard is colored.
◦ Chromosomes are not represented by bitstrings but by
bitmatrices
◦ The bits in the bitmatrix can have one of the four values 0, 1, 2
or 3, depending on the color.
◦ Crossing-over involves matrix manipulation instead of point
wise operating.
◦ Crossing-over can be combining the parential matrices in a
horizontal, vertical, triangular or square way.
◦ Mutation remains bitwise changing bits in either one of the
other numbers.
Checkboard example
Cont’d
• This problem can be seen as a graph with n nodes and (n-1) edges,
so the fitness f(x) is defined as:

f(x) = 2 · (n-1) ·n
 Checkboard example
Cont’d
• Fitnesscurves for different cross-over rules:

Lower-Triangular Crossing Over Square Crossing Over

180 180

170 170

160 160
Fitness

150 150

140 140

130 130
0 100 200 300 400 500 0 200 400 600 800

Horizontal Cutting Crossing Over Verical Cutting Crossing Over

180 180

170 170

160 160
Fitness

150 150

140 140

130 130
0 200 400 600 800 0 500 1000 1500
Generations Generations
Questions

??
THANK YOU