Degree Program MSc in Electrical Power and Control Engineering

(Control Engineering)
Course Title: Advanced Optimization Techniques (Course Code
EPCE6302) Credits 3; Contact Hours/week Lecture 2, Tutorial
Practice/Laboratory 3

Course Instructor: Dr Endalew Ayenew, Address EPCE Building

E-mail:[email protected]
Adama Science and Technology University School of Electrical
Engineering and Computing Department of Electrical Power and
Control Engineering
LEARNING OBJECTIVES:
 Understand Nature-Inspired Optimization knowledge.
 Design Advanced Optimization Techniques based applications in controller systems.
30/03/2024 Dr Endalew Ayenew 1
 Introduction
Optimization is the study of planning, designing, and solving complex problems in day-
to-day life in recent years. The optimization problems are of high significance in
knowledge-based domain like scientific sector and economy based domain like Industrial
Sector. Right from vehicle logistics management to constraint job scheduling and from
data repository integration, Vision tracking to Hard Problem solving, optimization
management is subjected to various constraints. The main objective in optimization is
to optimize the physical parameter depending on the problem considered.

30/03/2024 Dr Endalew Ayenew 2

 Introduction
The categorization of optimization algorithms is broadly classified as Deterministic and
Probabilistic algorithms. The deterministic algorithms are classical optimization
technique which has limited scope in the practical scenario as they can solve only
single variable functions and multivariable with or without constraints. The stochastic
(or) probabilistic algorithms are an advanced optimization technique which inherits
the essential numerical techniques of classical optimization and also has many optimal
input parameters like randomness, robustness, constraint functions, dimension
factor, equalities, scalability, etc., to ease the computation.

30/03/2024 Dr Endalew Ayenew 3

 Introduction

Advanced Optimization Techniques also sometimes called non-traditional

optimization techniques /numerical optimization, have emerged as powerful and
popular methods for solving complex engineering optimization problems in recent
years.
Classifications of Advanced Optimization Techniques is presented in the following picture.

30/03/2024 Dr Endalew Ayenew 4

Introduction Generalized Classification of optimization methods

30/03/2024 Dr Endalew Ayenew 5

 Introduction
In this course we will focus on

1. Evolutionary Computing based Optimization Techniques: Evolutionary Computing is a sub-

field of Soft Computing and is used extensively in complex optimization problems and for
continuous optimization. E.g.
Genetic algorithms are computerized search and optimization algorithms based on the
mechanics of natural genetics and natural selection.
Particle swarm optimization mimics the behavior of social organisms such as a colony or swarm
of insects. and
Ant colony optimization is based on the cooperative behavior of ant colonies, which are able to
find the shortest path from their nest to a food source.
Simulated annealing is based on the mechanics of the cooling process of molten metals
through annealing.

30/03/2024 Dr Endalew Ayenew 6

Introduction

Figure: Architecture of Evolutionary computing

30/03/2024 Dr Endalew Ayenew 7
 Introduction
2. Nature-Inspired based Optimization Techniques: Reference: Omid Bozorg-Haddad Editor (2018), Advanced
Optimization by Nature-Inspired Algorithms, Springer Nature Singapore Pte Ltd.
-Cat Swarm Optimization (CSO) Algorithm;
-League Championship Algorithm (LCA);
-Anarchic Society Optimization (ASO) Algorithm;
- Cuckoo Optimization Algorithm (COA);
-Teaching-Learning-Based Optimization (TLBO) Algorithm;
-Flower Pollination Algorithm (FPA);
-Krill Herd Algorithm (KHA);
-Grey Wolf Optimization (GWO) Algorithm;
-Shark Smell Optimization (SSO) Algorithm;
-Ant Lion Optimizer (ALO) Algorithm;
-Gradient Evolution (GE) Algorithm;
-Moth-Flame Optimization (MFO) Algorithm;
-Crow Search Algorithm (CSA); Dragonfly Algorithm (DA).
30/03/2024 Dr Endalew Ayenew 8
Introduction Recently introduced Physics-Inspired
Optimization Algorithms
1. Archimedes Optimization Algorithm (AOA), 2021

2. Atomic Orbital Search (AOS), 2021

3. Flow Direction Algorithm (FDA), 2021

4. Equilibrium Optimizer (EO), 2020

5. Henry Gas Solubility Optimization (HGSO), 2019

6. Atom Search Optimization (ASO), 2019

7. Nuclear Reaction Optimization (NRO), 2019

30/03/2024 Dr Endalew Ayenew 9

Introduction

30/03/2024 Dr Endalew Ayenew 10

 Introduction
3. Soft Computing based optimization Techniques: (not to be covered in this course)
Neural network-based optimizations are based on the immense computational power of the
nervous system to solve exceptional problems in the presence of massive amount of sensory
data through its parallel processing capability. and
Fuzzy optimizations are developed to solve optimization problems involving design data,
objective function, and constraints stated in imprecise form involving vague and linguistic
descriptions

30/03/2024 Dr Endalew Ayenew 11

 Introduction
Demonstration: Let us assume we deal with a Advanced Optimization technique /numerical
optimization problem/ modern optimization technique as formed below.

30/03/2024 Dr Endalew Ayenew 12

Chapter I: Evolutionary Computing based Optimization Techniques
Reference: Singiresu S. Rao (2009), Engineering Optimization Theory and Practice, Fourth Edition, John Wiley &
Sons, Inc., pp. 693-717.

1.1 Genetic algorithm (GA)

Sex-determination: The X, Y’s of sex chromosomes

30/03/2024 Dr Endalew Ayenew 13
Cont.GA
 Genetic Algorithms are search algorithms based on the mechanics of natural selection and
natural genetics.
 They start with a group of knowledge structures which are usually coded into binary
strings (chromosomes).
 These structures are evaluated within some environment and the strength (fitness) of a
structure is defined.
 The fitness of each chromosome is calculated and a new set of chromosomes is then
formulated by random selection and reproduction.

30/03/2024 Dr Endalew Ayenew 14

Cont. GA
 Each chromosome is selected with a probability determined by it’s fitness and, hence,
chromosomes with the higher fitness values will tend to survive and those with lower
fitness values will tend to become extinct.
 The selected chromosomes then undergo certain genetic operations such as:
crossover- where chromosomes are paired and randomly exchange information, and
mutation- where individual chromosomes are altered.
 The resulting chromosomes are re-evaluated and the process is repeated until no further
improvement in overall fitness is achieved.
 In addition, there is often a mechanism to preserve the current best chromosome
(elitism). “Survival of the fittest”
30/03/2024 Dr Endalew Ayenew 15
Elitism
It is possible sometimes to lose the best solution that the algorithm has already
found after performing crossover and mutation for several iterations. To
prevent this, some of the best chromosomes from the current population are
directly copied into the next generation without any modification, i.e. with
out being affected by crossover and mutation. This ensures the existence of
good chromosomes in each generation even if low quality chromosomes emerge
for one reason or another.

30/03/2024 Dr Endalew Ayenew 16

 Definitions

30/03/2024 Dr Endalew Ayenew 17

Components of a Genetic Algorithm (GA)
 a genetic representation
 a way to create an initial population of potential solutions
 an evaluation function rating solutions in terms of their “fitness”
 genetic operators that alter the composition of children during reproduction
 values of various parameters (population size, probabilities of applying genetic
operators, etc)
Differences from Conventional Optimization
 GAs work with a coding of the parameter set, not the parameters themselves
 GAs search from a population of points, not a single point
 GAs use probabilistic transition rules, not deterministic rules
 GAs have the capability of finding a global optimum within a set of local optima
30/03/2024 Dr Endalew Ayenew 18
How do we use GAs to optimize the parameters we’ re interested in?

30/03/2024 Dr Endalew Ayenew 19

 Genetic Algorithm Flow Diagram

30/03/2024 Dr Endalew Ayenew 20

30/03/2024 Dr Endalew Ayenew 21
Initial Population and Coding

solutions.
30/03/2024 Dr Endalew Ayenew 22
 Selection (roulette wheel principle)

We mathematically construct a “roulette wheel‟ with slots sized

according to fitness values. Spinning this wheel will then select a
new population according to these fitness values with the
chromosomes with the highest fitness having the greatest chance of
selection. The procedure is:

30/03/2024 Dr Endalew Ayenew 23

Cont. Selection

i
qi   p j
j 1

30/03/2024 Dr Endalew Ayenew 24

Example 1:

Segment area proportional to pi , i=1,...,12

30/03/2024 Dr Endalew Ayenew 25
Cont. Selection

30/03/2024 Dr Endalew Ayenew 26

Crossover:
We choose a parameter value pc as the probability of crossover. Then the
expected number of chromosomes to undergo the crossover operation will
be pc *pop_size. We proceed as follows:- (for each chromosome in the new
population)
1. Generate a random number ( r )from the range [0,1].
2. If r < pc , then select the given chromosome for crossover. ensuring that
an even number is selected. Now we mate the selected chromosomes
randomly:-
30/03/2024 Dr Endalew Ayenew 27
Crossover:

30/03/2024 Dr Endalew Ayenew 28

Crossover
Replaces two
parent solutions
with two children
solutions.

Mechanism for
covering large
area of search
space.

30/03/2024 Dr Endalew Ayenew 29

30/03/2024 Dr Endalew Ayenew 30
Mutation types
Mutation
Operates on a single
chromosome.

Mechanism to improve
local search space.

30/03/2024 Dr Endalew Ayenew 31

30/03/2024 Dr Endalew Ayenew 32
Summary of Advantages of Genetic Algorithms

30/03/2024 Dr Endalew Ayenew 33

Applications area of GA

30/03/2024 Dr Endalew Ayenew 34

30/03/2024 Dr Endalew Ayenew 35
Example 2 Cont.:

30/03/2024 Dr Endalew Ayenew 36

Example
2 cont.:

30/03/2024 Dr Endalew Ayenew 37

Example 2 cont.:

30/03/2024 Dr Endalew Ayenew 38

 Old chromosomes
Example 2 cont.:

became new
30/03/2024 Dr Endalew Ayenew 39
 Example 2 cont.:

In the range [1, m-1], for m = 9.

30/03/2024 Dr Endalew Ayenew 40

Example 2 cont.:

30/03/2024 Dr Endalew Ayenew 41

Example 2
cont.:

30/03/2024 Dr Endalew Ayenew 42

Example 2 r £ [0 1] 0.8 0.01 0.9 0.4 0.5 0.7 0.9 0.3 0.02 0.06
cont.: v1 v2 v3 v4 v5 v6 v7 v8 v9 v10

But, actually bit to be

mutated is after
Pm*m*N = 0.04*9*10
= 3.6 ~ 4

30/03/2024 Dr Endalew Ayenew 43

 (the weakest- V8 is replaced by the fittest- original V3)
Example 2
cont.:

30/03/2024 Dr Endalew Ayenew 44

Example 2
cont.:

30/03/2024 Dr Endalew Ayenew 45

Example 2
cont.:

30/03/2024 Dr Endalew Ayenew 46

Example 2
cont.:

30/03/2024 Dr Endalew Ayenew 47

Representation of Objective Function and Constraints

30/03/2024 Dr Endalew Ayenew 48

cont.:

30/03/2024 Dr Endalew Ayenew 49

cont.:

30/03/2024 Dr Endalew Ayenew 50

Problem Related to Application of GA
Problem 1.

30/03/2024 Dr Endalew Ayenew 51

Problem 2.

30/03/2024 Dr Endalew Ayenew 52

Problem 3.

Note: use Matlab

30/03/2024 Dr Endalew Ayenew 53
 1.2 PARTICLE SWARM OPTIMIZATION
 Particle swarm optimization (PSO) is based on the behavior of a colony
or swarm of insects, such as ants, termites, bees, and wasps, a flock of
birds; or a school of fish.
 The PSO algorithm was originally 1st proposed by James Kennedy (a social
psychologist) and Russell Eberhart (an electrical engineer) in 1995
(independently).
 The particle swarm optimization algorithm mimics the behavior of these
social organisms.

30/03/2024 Dr Endalew Ayenew 54

PSO Cont.

Colony of Termites (ምስጥ) Colony of wasps (ተርብ)

30/03/2024 Dr Endalew Ayenew 55
Cont. PSO

Flock of birds
30/03/2024 Dr Endalew Ayenew 56
1.2 PARTICLE SWARM OPTIMIZATION

30/03/2024 Dr Endalew Ayenew 57

1.2 PARTICLE SWARM OPTIMIZATION

A search technique used in computing to find

near optimal solution to optimization problem.
Thus PSO algorithm do not guarantee an
optimal solution is ever found

30/03/2024 Dr Endalew Ayenew 58

30/03/2024 Dr Endalew Ayenew 59
30/03/2024 Dr Endalew Ayenew 60
30/03/2024 Dr Endalew Ayenew 61
 Cont. PSO
The particles behave as a swarm through global communication. By adjusting the
flight direction of each particle in a quasi-stochastic manner, the swarm is able to
search the space of an objective function for near optimal solutions.

30/03/2024 Dr Endalew Ayenew 62

30/03/2024 Dr Endalew Ayenew 63
30/03/2024 Dr Endalew Ayenew 64
 +1

are cognitive and social parameters

30/03/2024 Dr Endalew Ayenew 65

Note: Inertia weight (w)
The role of the inertia weight w is considered important for the PSO's convergence behavior. It

is employed to control the impact of the previous history of velocities on the current velocity.

Thus, the parameter w regulates the trade off between the global (wide—ranging) and the local

(nearby) exploration abilities of the swarm. A large w facilitates exploration (searching new

areas), while a small one tends to facilitate exploitation, i.e. fine tuning the current search area.

A proper value for the inertia weight w provides balance between the global and local

exploration ability of the swarm, and, thus results in better solutions. Experimental results imply

that it is preferable to initially set the inertia to a large value, to promote global exploration of the

search space, and gradually decrease it to obtain refined solutions. Particularly, 0.9 ≧ w ≧ 0.1

30/03/2024 Dr Endalew Ayenew 66

30/03/2024 Dr Endalew Ayenew 67
30/03/2024 Dr Endalew Ayenew 68
 +1

30/03/2024 Dr Endalew Ayenew 69

30/03/2024 Dr Endalew Ayenew 70
Example:

30/03/2024 Dr Endalew Ayenew 71

PSO Algorithm
% PSO Parameters
M=3; n=5; Wmax=0.9; Wmin =0.4, c1=0.4; c2=2;
% population initialization (swarm)
LB=[0 0 0]; UB=[15 15 15];

PARTICLES
Maxt=50;
for i =1:n
for j =1:m
Xo(I,j) = round(LB(j)+rand(0,1)*(UB(j)-LB(j)))
end
end
30/03/2024 Dr Endalew Ayenew 72
30/03/2024 Dr Endalew Ayenew 73
 +1

, for t= 1
V1 V2 V3

Initial Velocity

30/03/2024 Dr Endalew Ayenew 74

F(X)

30*(1.1-3)^2 = 1.3226

30/03/2024 Dr Endalew Ayenew 75

 (Aim is minimization
of obj. fun., so select
Aim: Follow the Bird which is Nearest to the Food smallest fitness
value)
Step 2: Evaluate Fitness f(xti) for each particles, and then
choose the best value as gBest
gBest= [x02(1) =9.9, x02(2) =6.6, x02(3) =1.1 ]

+1
UPDATE

30/03/2024 Dr Endalew Ayenew 76

 Best01 - x01) , for r2 =0.1576 and gBest01 = x02(1) =9.9
, for r1 and r2 in rand (0, 1)
Use
for X1 , for r2 =0.1576 matlab
Note: using the following initial values of position and velocity
for X2 , for r2 =0.800
V1 V2 V3
for X3

Initial Velocity
30/03/2024 Dr Endalew Ayenew 77
 V1 for X1 V1 for X2 V1 for X3

+1

Updated velocity

30/03/2024 Dr Endalew Ayenew 78

 (7.14-)^2+30

30/03/2024 Dr Endalew Ayenew 79

30/03/2024 Dr Endalew Ayenew 80
30/03/2024 Dr Endalew Ayenew 81
 New pBest value
1.0512
1.3236
2.2179
2.9208
2.2542

30/03/2024 Dr Endalew Ayenew 82

 Updated position for each Particles

30/03/2024 Dr Endalew Ayenew 83

For Constrained Optimization Problem using PSO
Referee to the book:
Engineering Optimization: Theory and Practice, Fourth Edition Singiresu S. Rao Copyright ©
2009 by John Wiley & Sons, Inc.

30/03/2024 Dr Endalew Ayenew 84

 Matlab Constrained Objective Function by PSO
(Example)

30/03/2024 Dr Endalew Ayenew 85

 Notes: To use PSO matlab code;
Change Maximization to minimization as seen below

And in the main code change < to > or > to <.

30/03/2024 Dr Endalew Ayenew 86
 Multi-objective optimization
A multi-objective optimization problem with a number of competing objectives
can be defined as follows:

𝐺 objective space, as formulated in equations on the next page.

30/03/2024 Dr Endalew Ayenew 87
𝑍, which is an element of 𝑋, that dominates any solution 𝑌, as formulated in the
following equation

Note: One of its drawbacks is that PSO does not have memory. It does not store the historical trajectories of
each particle in each iteration. This makes it hard for the algorithm to avoid local optima
30/03/2024 Dr Endalew Ayenew 88
Problem 1

Problem 2
Design PSO Optimized PID Controller for a continuous stirred tank reactor (CSTR) System. Use
minimum integral of time multiplied by absolute error (ITAE) as objective function.

30/03/2024 Dr Endalew Ayenew 89

Model of CSTR System

30/03/2024 Dr Endalew Ayenew 90

 Use matlab to generate random number r1 and r2 rand ().

30/03/2024 Dr Endalew Ayenew 91

 Problem 3: Consider a DC motor speed control system. The
PID controller is given as
U (s) Ki
 Kp   Kd s
E (s) s
and the dc motor dynamics is described by
Bm Km Fc
x1   x1  x2  sgn( x1 )
J J J
Ke Rm 1
x2   x1  x2  u
L L L
where x1(t)= (t), x2(t)= i(t), and Fc denotes the Coulomb friction
coefficient and the other parameters are in the table.

30/03/2024 Dr Endalew Ayenew 92

 DC Motor Parametrs
Parameters Values(Unit)
Armature resistance, Rm 4.0 ()
Armature inductance, L 0.072 (H)
Torque constant, Km 1.26 (Nm/A)
Back EMF constant, Ke 1.26 (V/rad/s)
Friction coefficient, Bm 0.087 (Nm/rad/s)
Motor inertia, J 0.061 (kgm2)
Coulomb friction Fc 0.001 (Nm)

30/03/2024 Dr Endalew Ayenew 93

(a) Tune the PID controller parameters using PSO such that the IAE
performance index is minimized:

IAEU  0
(| e | 0.1* | u  u0 |) dt

where the reference input is r= 1000rpm, e= r-y and u0= (Ke+RmBm/Km)r.

(b) Simulate the response of the closed-loop system for 1sec

30/03/2024 Dr Endalew Ayenew 94

1.3 ANT COLONY OPTIMIZATION (ACO)
- Basic Concept Ant colony optimization (ACO) is based on the cooperative
behavior of real ant colonies, which are able to find the shortest path from their
nest to a food source. The method was developed by Italian PhD candidate
‘Morco Dorigo’ in 1992. He proposed Ant System for the PhD thesis and he used
ACO. It used for discrete optimization problem. The ACO process can be
explained by representing the optimization problem as a multi-layered graph as
shown in the next Figure; where the number of layers is equal to the number of
design variables and the number of nodes in a particular layer is equal to the
number of discrete values permitted for the corresponding design variable.
30/03/2024 Dr Endalew Ayenew 95
Cont. ACO

Figure: a colony of ants is confronted with the choice of reaching their food via two different routes
30/03/2024 Dr Endalew Ayenew 96
Cont. ACO

Graphical representation of the ACO process in the

form of a multi-layered network.
30/03/2024 Dr Endalew Ayenew 97
 Cont. ACO
Thus each node is associated with a permissible discrete value of a design variable. The
Figure denotes a problem with six design variables with eight permissible discrete values
for each design variable.

The ACO process can be explained as follows

1. Let the colony consist of N ants
2. The ants start at the home node, travel through the various layers from the first layer
to the last or final layer, and end at the destination node in each cycle or iteration.
3. Each ant can select only one node in each layer in accordance with the state.
transition rule given by Eq. (13.32), pheromone decay (p) and global updating (ξ)
factors.
4. The nodes selected along the path visited by an ant represent a candidate solution.
Pheromone: substance produced by an animal as a chemical signal to attract another of the same species.
(ዝባድ- የጥርኝ የላበት ግግር - ሽቶ ለመስራት የሚያገለግል)
30/03/2024 Dr Endalew Ayenew 98
30/03/2024 Dr Endalew Ayenew 99
Cont. ACO

30/03/2024 Dr Endalew Ayenew 100

30/03/2024 Dr Endalew Ayenew 101
Cont. ACO

30/03/2024 Dr Endalew Ayenew 102

30/03/2024 Dr Endalew Ayenew 103
 ACO

30/03/2024 Dr Endalew Ayenew 104

 ACO Algorithm

30/03/2024 Dr Endalew Ayenew 105

 ACO Algorithm

30/03/2024 Dr Endalew Ayenew 106

 e solution the problem
ze

30/03/2024 Dr Endalew Ayenew 107

 of discrete value m)

Variables equal amounts of pheromone r

30/03/2024 Dr Endalew Ayenew 108

30/03/2024 Dr Endalew Ayenew 109
Cont. Example

30/03/2024 Dr Endalew Ayenew 110

Cont. Example

30/03/2024 Dr Endalew Ayenew 111

Cont. Example

30/03/2024 Dr Endalew Ayenew 112

Cont. Example

30/03/2024 Dr Endalew Ayenew 113

Cont. Example

30/03/2024 Dr Endalew Ayenew 114

Cont. Example

30/03/2024 Dr Endalew Ayenew 115

Cont. Example

30/03/2024 Dr Endalew Ayenew 116

Cont. Example

30/03/2024 Dr Endalew Ayenew 117

Problem 1 (ACO)

Problem 2 (ACO)

Note: consider pheromone decay factor equal to 0.6 and global updating parameter is 2.

30/03/2024 Dr Endalew Ayenew 118

 Design Optimal LQR and PID Controller tuning their Parameters using ACO
considering Problem 3 under PSO a DC motor speed control system
Referee: Aman Jacknoon and M. A. Abido, (2017), Ant Colony Based LQR and PID tuned Parameters for Controlling Inverted
Pendulum, ICCCCEE, Khartoum, Sudan

DC motor

PID parameters tuning using ACO

PID (Parameters Kp, Ki, Kd) and LQR (Parameters Q and R, and then K ) are to be optimally tuned by ACO.
30/03/2024 Dr Endalew Ayenew 119