PUE 4110: Optimization Techniques

Lecturer: Prof. Philip Ngare

(Contact: [email protected])

Pan African University

Institute for Science, Technology and Innovations
Ph.D in Electrical Engineering

May 21, 2021

Prof. P. Ngare (PAUSTI) PUE 4110: Optimization Techniques May 21, 2021 1 / 15
Course Purpose

The aim of this course is to enable the student to have a solid foundation
in the general concepts of optimization theory and be equipped with an
array of techniques to solve electrical engineering problems.

Prof. P. Ngare (PAUSTI) PUE 4110: Optimization Techniques May 21, 2021 2 / 15
Expected Learning Outcomes

At the end of the course, the student will be able to;

1 Apply various optimization algorithms in solving electrical engineering
problems
2 Design and implement appropriate optimization approaches for
specific applications

Prof. P. Ngare (PAUSTI) PUE 4110: Optimization Techniques May 21, 2021 3 / 15
Course description

1 Linear Programming
I Classical Optimization Techniques: One-Variable Unconstrained
Optimization, Multivariable Unconstrained Optimization
I Linear Programming; Simplex method of solving LPP, Duality
2 Non-linear programming
I Types of Nonlinear Programming Problems
I Integer programming, geometric programming
I The Karush-Kuhn-Tucker (KKT) Conditions for Constrained
Optimization
I Quadratic Programming
I Set Constrained Optimization
I Convex Programming
3 Dynamic Programming

Prof. P. Ngare (PAUSTI) PUE 4110: Optimization Techniques May 21, 2021 4 / 15
Course description

4 Decision Processes
I Decision Analysis
I Markov Decision Processes

5 Stochastic optimization/Simulation methods:

I Metropolis-Hastings algorithm and Gibbs sampling

6 Emerging Optimization Techniques with applications to electrical

engineering
I Genetic algorithm,
I Simulated annealing
I Tabu search and Swarm optimization algorithms
I Linear Matrix Inequalities (LMI)- Based Optimization
I Artificial Intelligence and Evolutionary Algorithms-Based Optimization;
Artificial neural network, Machine learning with applications to electrical
engineering.

Prof. P. Ngare (PAUSTI) PUE 4110: Optimization Techniques May 21, 2021 5 / 15
Introduction
Over the years, making the best decision among a number of possible
choices has been a challenging issue.
The issue arises from the complexity of the decision criteria and the
extent of possible choices.
Optimization is the knowledge of decision-making.
In the area of engineering, the first step towards optimization is to
model the decision-making process mathematically into an
optimization problem.
Depending on the structure of the decision criteria and possible
choices, optimization problems are generally categorized into linear
and nonlinear programming problems.
linear programming is a mature area and mostly considered as a
technology.
On the other hand, the area of nonlinear programming remains a
challenging domain due the diversity and complexity in the form of
functions deployed in the problem.
Prof. P. Ngare (PAUSTI) PUE 4110: Optimization Techniques May 21, 2021 6 / 15
Introduction
Nonlinear programming can be divided into different categories:
integer programming, convex programming and geometric
programming are more common due to the solution procedures
proposed in the literature.
solving a nonlinear programming problem analytically is not trivial in
general
Alternatively, artificial intelligence-based tools such as artificial neural
network, particle swarm optimization, and genetic algorithm can be
used.
Inspired by the human and nature, these tools are mostly based on
search methods.
They employ intelligent techniques to explore the whole possible
solutions of the given problem and therefore to overcome some
limitations of traditional optimization approaches.
The challenging issue is in the modeling of nonlinear programming
forms is recognize whether a given problem follows the required forms
or not then reliable and efficient algorithms is implemented.
Prof. P. Ngare (PAUSTI) PUE 4110: Optimization Techniques May 21, 2021 7 / 15
Optimization in Electrical Engineering
Engineering is the knowledge of analysis and design.
The most important requirement of designing is the simplicity in the
design and the optimality in the performance.
The knowledge of optimization helps engineers to design more
efficient systems.
They would be able to carry out systematic modeling, designing, and
analysis of complex networks and systems considering required
constraints and parameters.
For example, communication and electrical power infrastructures are
getting more intelligent, a higher number of electronic devices are
placed on integrated circuits, and more data is generated and
processed by the information technology.
Design and analysis in all of these areas need appropriate tools of
optimization.

Prof. P. Ngare (PAUSTI) PUE 4110: Optimization Techniques May 21, 2021 8 / 15
Optimization in Electrical Engineering: Signal Processing
One known application of optimization in the domain of signal
processing was the design of linear phase finite impulse response
filters in which filter coefficients are optimized.
In array signal processing, multiple sensor outputs are linearly
combined together to form a composite array.
The output weights are then optimized to form a desired pattern.
The parameters of classification algorithms in statistical and machine
learning are determined with optimization methods.
Image and speech processing applications take advantage of
optimization techniques for noise removal, recognition, and
automation.
Parameter estimation and data fitting are also optimized using fitting
moving average (MA) or autoregressive moving average (ARMA)
models.
The performance of many functional blocks in communication
systems and networks, is improved using optimization methods;
equalization, pulse shape design, demodulation, antenna selection.
Prof. P. Ngare (PAUSTI) PUE 4110: Optimization Techniques May 21, 2021 9 / 15
Optimization in Electrical Engineering: Circuit Design
With the development of VLSI(Very large scale integration)
technology, integration of mixed analog–digital circuits as a complete
system on a chip is in progress.
Although the analog part forms a small part of the entire circuit, its
design is much more difficult due to the complex nature of analog
circuit design
This design is often done in two stages. In the first stage, the circuit
topology and structure is constructed.
In the second stage, the circuit parameters such as the size of
elements are determined in order to optimize performance measures
such as area, gain, and consumption power.
Optimization methods and tools not only reduce the time of circuit
design but also result in more efficient, robust, and cost- effective
circuits i.e
I evolutionary optimization methods in the design of analog integrated
circuits,
I A particle swarm optimization method is used towards multi-objective
circuit design and extracting Pareto feasible points
Prof. P. Ngare (PAUSTI) PUE 4110: Optimization Techniques May 21, 2021 10 / 15
 Furthermore, the implementation of radio frequency energy harvesting
circuits is also done using optimization methods

Prof. P. Ngare (PAUSTI) PUE 4110: Optimization Techniques May 21, 2021 11 / 15
Resource Allocation in Communication Networks
Due to the diversity in the number of supported users and services,
communication networks have demonstrated their importance and
progressing role for the community over the years.
The performance of these networks is highly impacted by the
methodologies employed to design and to allocate communication
resources among the network entities
The resources appear in a number of forms including time,
geographical space, transmitted power by antennas, frequency
spectrum provided by the network utility, buffer spaces in the network
equipment, etc.
The purpose of resource allocation in communication networks is to
intelligently assign the limited available resources among the network
entities in an efficient way to satisfy the user’s service requirements
With the fast development of communication networks and diversity
in the number of transmission technologies, resource allocation
becomes a challenging issue.
To satisfy this requires, optimization theory, control and game theory.
Prof. P. Ngare (PAUSTI) PUE 4110: Optimization Techniques May 21, 2021 12 / 15
Power Dispatching
Power generation units in electric power systems face a number of
constraints such as fuel, generation levels, ramping rate, output
uncertainties, and the system load demand.
The process of power dispatching is to determine and to allocate
generation levels to generation units in order to satisfy the load and
power demand in the network while considering the units and the
network constraints.
The objective in power dispatching is to minimize the cost of power
production, which is known as economic power dispatching problem.
Due to the constraints of the network and generation units,
transmission losses, and generation and load uncertainties, power
dispatching problem has been a challenging issue for decades.
To overcome the complexity of power dispatching, optimization and
control tools have been employed widely by the researchers.
Intelligence-based optimization tools have been employed widely for
dispatching problems.
Prof. P. Ngare (PAUSTI) PUE 4110: Optimization Techniques May 21, 2021 13 / 15
 The uncertainty from both sides of generation and demand in
next-generation power grids needs new techniques on stochastic
modeling and optimization.

Prof. P. Ngare (PAUSTI) PUE 4110: Optimization Techniques May 21, 2021 14 / 15
References and recent research publications

1 D.G. Luenberger, Y. Ye, Linear and Nonlinear Programming

(Springer, Berlin, 2016)
2 Kumar, S., Jha, R.K. Noise-Induced Resonance and Particle Swarm
Optimization-Based Weak Signal Detection. Circuits Syst Signal
Process 38, 2677–2702 (2019).
https://doi.org/10.1007/s00034-018-0987-1
3 Sun, Ying, Prabhu Babu, and Daniel P. Palomar.
”Majorization-minimization algorithms in signal processing,
communications, and machine learning.” IEEE Transactions on Signal
Processing 65.3 (2016): 794-816.
4 Tazi, Khadija, Fouad Mohamed Abbou, and Farid Abdi. ”Multi-agent
system for microgrids: design, optimization and performance.”
Artificial Intelligence Review 53.2 (2020): 1233-1292.

Prof. P. Ngare (PAUSTI) PUE 4110: Optimization Techniques May 21, 2021 15 / 15