ECEG-6311

Power System Optimization

and AI
Modern Methods of Optimization
Yoseph Mekonnen (Ph.D.)

Page 1
Outlines
Introduction
Genetic Algorithm
Simulated Annealing
Particle Swarm
Ant Colony
Fuzzy Logic
Neural Network

Page 2
 Introduction to AI
In recent years, some optimization methods that are
conceptually different from the traditional mathematical
programming techniques have been developed.
These methods are labeled as modern or nontraditional
methods of optimization.
Most of these methods are based on certain
characteristics and behavior of biological, molecular, swarm
of insects, and neurobiological systems.
The genetic algorithms are based on the principles of
natural genetics and natural selection.
Simulated annealing is based on the simulation of thermal
annealing of critically heated solids.

Page 3
 ..Contd..
Both genetic algorithms and simulated annealing are
stochastic methods that can find the global minimum with a
high probability and are naturally applicable for the solution
of discrete optimization problems.
The particle swarm optimization is based on the behavior
of a colony of living things, such as a swarm of insects, a
flock of birds, or a school of fish.
Ant colony optimization is based on the cooperative
behavior of real ant colonies, which are able to find the
shortest path from their nest to a food source.
In many practical systems, the objective function,
constraints, and the design data are known only in vague and
linguistic terms. Fuzzy optimization methods have been
developed for solving such problems.
Page 4
 ..Contd..
In neural-network-based methods, the problem is modeled
as a network consisting of several neurons, and the network
is trained suitably to solve the optimization problem
efficiently.

Page 5
 GENETIC ALGORITHMS
Many practical optimum design problems are characterized by
mixed continuous–discrete variables, and discontinuous and non
convex design spaces.
If standard non linear programming techniques are used for this
type of problem they will be inefficient, computationally
expensive, and, in most cases, find a relative optimum that is
closest to the starting point.
Genetic algorithms (GAs) are well suited for solving such
problems, and in most cases they can find the global optimum
solution with a high probability.
GAs were first presented systematically by Holland, the basic
ideas of analysis and design based was described by of
Rechenberg.
Philosophically, GAs are based on Darwin’s theory of survival of
the fittest.
Page 6
 ..Contd..
Genetic algorithms are based on the principles of natural
genetics and natural selection.
The basic elements of natural genetics reproduction,
crossover, and mutation are used in the genetic search
procedure.
GAs differ from the traditional methods of optimization in
the following respects:
1. A population of points (trial design vectors) is used for
starting the procedure instead of a single design point.
If the number of design variables is n, usually this is
taken as 2n to 4n.
Since several points are used as candidate solutions, GAs
are less likely to get trapped at a local optimum.

Page 7
 ..Contd..
2. GAs use only the values of the objective function.
The derivatives are not used in the search procedure.
3. In GAs the design variables are represented as strings of
binary variables that correspond to the chromosomes in
natural genetics.
Thus the search method is naturally applicable for solving
discrete and integer programming problems.
For continuous design variables, the string length can be
varied to achieve any desired resolution.
4.The objective function value corresponding to a design
vector plays the role of fitness in natural genetics.

Page 8
 ..Contd..
5. In every new generation, a new set of strings is produced
by using randomized parents selection and crossover
from the old generation (old set of strings).
They efficiently explore the new combinations with the
available knowledge to find a new generation with better
fitness or objective function value.

Page 9
 Representation of Design Variables
In GAs, the design variables are represented as strings of
binary numbers, 0 and 1.
For example, if a design variable xi is denoted by a string
of length four (or a four-bit string)as 0 1 0 1, its integer
(decimal equivalent) value will be:

If each design variable xi, i =1,2, . . . , n is coded in a string

of length q, a design vector is represented using a string of
total length nq.
For example, if a string of length 5 is used to represent
each variable, a total string of length 20 describes a design
vector with n=4.

Page 10
 ..Contd..
The following string of 20 binary digits denote the vector
(x1=18, x2=3, x3=1, x4=4):

In general, if a binary number is given by bqbq−1· · · b2b1b0,

where bk=0 or 1, k=0,1,2, . . . , q , then its equivalent decimal
numbery (integer) is given by:

This indicates that a continuous design variable x can only

be represented by a set of discrete values if binary
representation is used.

Page 11
 ..Contd..
If a variable x (whose bounds are given by x(l) and x(u)) is
represented by a string of q binary numbers its decimal
value can be computed as:

If a continuous variable is to be represented with high

accuracy, we need to use a large value of q in its binary
representation.
In fact, the number of binary digits needed (q) to
represent a continuous variable in steps (accuracy) of x can
be computed from the relation:

Page 12
 ..Contd..
For example, if a continuous variable x with bounds 1 and 5
is to be represented with an accuracy of 0.01, we need to
use a binary representation with q digits where:

Page 13
 Example
Steel plates are available in thicknesses (in inches) from a
manufacturer.

If the thickness of the steel plate, to be used in the

construction of a pressure vessel, is considered as a discrete
design variable, determine the size of the binary string to be
used to select a thickness from the available values.
The lower and upper bounds on the steel plate (design
variable, x) are given by 1/32 and ½ in., respectively, and the
resolution or difference between any two adjacent
thicknesses is 1/32in. Hence:

from which the size of the binary string to be used can be

obtained as q=4.
Page 14
Representation of Objective Function and Constraints

Since genetic algorithms are based on the survival-of-the-

fittest principle of nature, they try to maximize a function
called the fitness function.
Thus GAs are naturally suitable for solving unconstrained
maximization problems.
The fitness function, F(X), can be taken to be same as the
objective function f(X) of an unconstrained maximization
problem so that F(X)=f(X).
A minimization problem can be transformed into a
maximization problem before applying the GAs.
Usually the fitness function is chosen to be nonnegative.

Page 15
 ..Contd..
The commonly used transformation to convert an
unconstrained minimization problem to a fitness function is
given by:

It can be seen that the above equation does not alter the
location of the minimum of f(X) but converts the
minimization problem into an equivalent maximization
problem.

Page 16
 ..Contd..
A general constrained minimization problem can be stated
as:

This problem can be converted into an equivalent

unconstrained minimization problem by using the concept of
penalty function as:

The penalty will be proportional to the square of the amount

of violation of the inequality and equality constraints at the
design vector X.

Page 17
 ..Contd..

where ri and Rj are the penalty parameters associated with

the constraints gi(X) and hj(X), whose values are usually
kept constant throughout the solution process. In the
equation, the function〈gi(X)〉, called the bracket function,
is defined as:

In most cases, the penalty parameters associated with all

the inequality and equality constraints are assumed to be
the same constants as:

where r and R are constants.

Page 18
 ..Contd..
The fitness function, F(X), to be maximized in the GAscan
be obtained:

Page 19
 Genetic Operators
The solution of an optimization problem by GAs starts with
a population of random strings denoting several (population
of) design vectors.
The population size in GAs(n)is usually fixed.
Each string (or design vector) is evaluated to find its
fitness value.
The population is operated by three operators:
Reproduction
Crossover
Mutation
to produce a new population of points (designs).

Page 20
 ..Contd..
The new population is further evaluated to find the fitness
values and tested for the convergence of the process.
One cycle of reproduction, crossover, and mutation and the
evaluation of the fitness values is known as a generation in
GAs.
If the convergence criterion is not satisfied, the
population is iteratively operated by the three operators
and the resulting new population is evaluated for the fitness
values.
The procedure is continued through several generations
until the convergence criterion is satisfied and the process
is terminated.

Page 21
 Reproduction.
Reproduction is the first operation applied to the
population to select good strings (designs) of the population
to form a mating pool.
The reproduction operator is also called the selection
operator because it selects good strings of the population.
The reproduction operator is used to pick above-average
strings from the current population and insert their
multiple copies in the mating pool based on a probabilistic
procedure.
In a commonly used reproduction operator, a string is
selected from the mating pool with a probability
proportional to its fitness.

Page 22
 ..Contd..
Thus if Fi denotes the fitness of the ith string in the
population of size n, the probability for selecting the ith
string for the mating pool (pi) is given by:

The sum of the probabilities of the strings of the

population being selected for the mating pool is one.
the average fitness of the population:

Page 23
 Roulette Wheel Example.

The population size is assumed to be 6 with fitness values

of the strings 1, 2, 3, 4, 5, and 6 given by 12, 4, 16, 8, 36,
and 24, respectively.
Since the fifth string (individual) has the highest value, it is
expected to be selected most of the time (36% of the time,
probabilistically) when the roulette wheel is spun n times
(n=6)
Page 24
 ..Contd..
The selection scheme, based on the spinning of the roulette
wheel, can be implemented numerically during computations
as follows:
The of selecting different strings based on their fitness
values are calculated using:

These probabilities are used to determine the cumulative

probability of string i being copied to the mating pool, Pi, by
adding the individual probabilities of strings 1 through i as:

Page 25
 ..Contd..
To generate the mating pool of size n during numerical
computations, n random numbers, each in the range of zero
to one, are generated (or chosen).
By treating each random number as the cumulative
probability of the string to be copied to the mating pool, n
strings corresponding to the n random numbers are selected
as members of the mating pool.
By this process, the string with a higher (lower) fitness
value will be selected more (less) frequently to the mating
pool because it has a larger (smaller) range of cumulative
probability.
Thus strings with high fitness values in the population,
probabilistically, get more copies in the mating pool.

Page 26
 ..Contd..
It is to be noted that no new strings are formed in the
reproduction stage; only the existing strings in the
population get copied to the mating pool.
The reproduction stage ensures that highly fit individuals
(strings) live and reproduce, and less fit individuals (strings)
die.
Thus the GAs simulate the principle of ―survival-of-the-
fittest‖ of nature.

Page 27
 ..Contd..
Table

Page 28
 Example
Consider six strings with fitness values 12, 4, 16, 8, 36, and 24
with the corresponding roulette wheel as shown in Fig.

Find the levels of contribution of the various strings to the

mating pool using the roulette-wheel selection process with
the following 12 random numbers: 0.41, 0.65, 0.42, 0.80, 0.67,
0.39, 0.63, 0.53, 0.86,0.88, 0.75, 0.55. Although the original
population consists of only 6 strings, the mating pool is
assumed to be composed of12 strings to illustrate the
roulette-wheel selection process.
Page 29
 Solution
If the given random numbers are assumed to represent
cumulative probabilities, the string numbers to be copied to
the mating pool can be determined from the cumulative
probability ranges listed in the last column of the previous
table as follows:

Page 30
 ..Contd..
This indicates that the mating pool consists of 1 copy of
string 4, 8 copies of string 5,and 3 copies of string 6.
This shows that less fit individuals (strings 1, 2, and 3) did
not contribute to the next generation (or died) because
they could not contribute to the mating pool.
String 4, although has a small fitness value, contributed 1
copy to the mating pool based on the random selection
process used.

Page 31
 Crossover
After reproduction, the crossover operator is
implemented.
The purpose of crossover is to create new strings by
exchanging information among strings of the mating pool.
Many crossover operators have been used in the literature
of GAs.
In most crossover operators, two individual strings
(designs) are picked (or selected) at random from the
mating pool generated by the reproduction operator and
some portions of the strings are exchanged between the
strings.

Page 32
 ..Contd..
In the commonly used process, known as a single-point
crossover operator, a crossover site is selected at random
along the string length, and the binary digits (alleles) lying
on the right side of the crossover site are swapped
(exchanged) between the two strings.
The two strings selected for participation in the crossover
operators are known as parent strings and the strings
generated by the crossover operator are known as child
strings.

Page 33
 ..Contd..
For example, if two design vectors (parents), each with a
string length of 10, are given by:

The result of crossover, when the crossover site is 3, is

given by:

Since the crossover operator combines substrings from

parent strings (which have good fitness values), the
resulting child strings created are expected to have better
fitness values provided an appropriate (suitable) crossover
site is selected.

Page 34
 ..Contd..
Hence the crossover site is usually chosen randomly.
The child strings generated using a random crossover site
may or may not be as good or better than their parent
strings in terms of their fitness values.
 If they are good or better than their parents, they will
contribute to a faster improvement of the average fitness
value of the new population.
On the other hand, if the child strings created are worse
than their parent strings, it should not be of much concern
to the success of the GAs because the bad child strings will
not survive very long as they are less likely to be selected in
the next reproduction stage (because of the survival-of-
the-fittest strategy used).

Page 35
 Mutation
The mutation operator is applied to the new strings with a
specific small mutation probability, pm.
The mutation operator changes the binary digit (allele’s
value) 1 to 0 and vice versa.
In the single-point mutation, a mutation site is selected at
random along the string length and the binary digit at that
site is then changed from 1 to 0 or 0 to 1 with a probability
of pm.
In the bit-wise mutation, each bit (binary digit) in the
string is considered one at a time in sequence, and the digit
is changed from 1 to 0 or 0 to 1 with a probability pm.

Page 36
 ..Contd..
Numerically a random number between 0 and 1 is
generated/chosen.
If the random number is smaller than pm, then the binary
digit is changed. Otherwise, the binary digit is not changed.
The purpose of mutation is:
(1) to generate a string (design point) in the neighbor hood
of the current string, thereby accomplishing a local search
around the current solution,
(2) to safeguard against a premature loss of important
genetic material at a particular position, and
(3) to maintain diversity in the population.

Page 37
 ..Contd..
As an example, consider the following population of size n=5
with a string length 10:

Here all the five strings have a 1 in the position of the first
bit. If the true optimum solution of the problem requires a
0 as the first bit. The required 0 cannot be created by
either the reproduction or the crossover operators.
However, when the mutation operator is used, the binary
number will be changed from 1 to 0 in the location of
thefirst bit with a probability of npm.

Page 38
 ..Contd..
Note that the three operators—reproduction, crossover,
and mutation—are simple to implement.
The reproduction operator selects good strings for the
mating pool, the crossover operator recombines the
substrings of good strings of the mating pool to create
strings (next generation of population), and the mutation
operator alters the string locally.
The use of these three operators successively yields new
generations with improved values of average fitness of the
population.
The process has been found to converge to the optimum
fitness value of the objective function.

Page 39
 ..Contd..
Algorithmm

Page 40
Thank You!

Page 41