Review of Math

Basics of Optimization
Power systems basics
Review of Matlab
Examples of Optimization in Power systems
Graphical Optimization
Dr. Muhammad Naeem Optimization Types Constraint and Unconstraint
Optimization Problem Types Linear NonLinear etc
Linear Optimization and Power Systems Applications
Non Linear Optimization and Power Systems Applications
Integer and Mixed integer programming and Power Systems Applications
Complexity Analysis

Sets A well-defined collection of objects Sets A well-defined collection of objects

Subset Subset
A={1,3,5,7,9}, B={x|x is odd}, C={1,3,5,7,9,...} A={1,3,5,7,9}, B={x|x is odd}, C={1,3,5,7,9,...}

Cardinality of A=5 (|A|=5) Cardinality of A=5 (|A|=5)

A⊂ B A is a proper subset of B. (If A is a subset of B, but A is not A⊂ B A is a proper subset of B. (If A is a subset of B, but A is not
equal to B (i.e. there exists at least one element of B which is not an equal to B (i.e. there exists at least one element of B which is not an
element of A) element of A)
A ⊆ B ⇔ ∀x[ x ∈ A x ∈ B]
C⊆B C is a subset of B. A ⊄ B ⇔ ¬∀x[ x ∈ A x ∈ B] C⊆B C is a subset of B.

⇔ ∃x[ x ∈ A ∧ x ∉ B ] The power set is the set of all subsets of a given set.
The power set is the set of all subsets of a given set. For the set S = {1,2,3} this means:
For the set S = {1,2,3} this means: P(S) = {0, {1}, {2}, {3}, {1,2}, {1,3}, {2,3}, {1,2,3} }
P(S) = {0, {1}, {2}, {3}, {1,2}, {1,3}, {2,3}, {1,2,3} } |S|=n then |P(S)| = 2n
|S|=n then |P(S)| = 2n
Conceptually, sets may be infinite (i.e., not finite, without end). For real numbers a,b with a<b
Symbols for some special infinite sets: [ a, b] = {x ∈ R | a ≤ x ≤ b} closed interval
N = {0, 1, 2, …} The natural numbers.
Z = {…, -2, -1, 0, 1, 2, …} The integers. ( a, b) = {x ∈ R | a < x < b} open interval
R = The “real” numbers, such as 374.1828… [ a, b) = {x ∈ R | a ≤ x < b} half-open interval
( a, b] = {x ∈ R | a < x ≤ b} half-open interval
For real numbers a,b with a<b
[ a, b] = {x ∈ R | a ≤ x ≤ b} closed interval x∈S (“x is in S”) is the proposition that object x is an element or member
of set S.
( a, b) = {x ∈ R | a < x < b} open interval e.g. 3∈N, “a”∈{x | x is a letter of the alphabet}
Can define set equality in terms of ∈ relation:
[ a, b) = {x ∈ R | a ≤ x < b} half-open interval ∀S,T: S=T ↔ (∀x: x∈S ↔ x∈T)
“Two sets are equal iff they have all the same members.”
( a, b] = {x ∈ R | a < x ≤ b} half-open interval x∉S :≡ ¬(x∈S) “x is not in S”

! !
Let X and Y be two nonempty sets. Let X and Y be two nonempty sets.
A function from X into Y is a relation that associates with each element of A function from X into Y is a relation that associates with each element of
X exactly one element of Y. X exactly one element of Y.

Domain: In a set of ordered pairs, (x, y), the domain is the set of all x- Domain: In a set of ordered pairs, (x, y), the domain is the set of all x-
coordinates. coordinates.
Range: In a set of ordered pairs, (x, y), the range is the set of all y- Range: In a set of ordered pairs, (x, y), the range is the set of all y-
coordinates. coordinates.

Example: f (x) = x−5 Example: f (x) = x−5

Domain: {x: x 5} Domain: {x: x 5}
Range: {y: y 0} Range: {y: y 0}

Example: f ( x) = x 2 − 5 Example: f ( x ) = x2 − 5
Domain: All Real
Range: {y: y -5}
 ! ! " #
Let X and Y be two nonempty sets.
For n a natural number, There are 8 choices for the first position,
A function from X into Y is a relation that associates with each element of n! = n(n - 1)(n - 2)·...·3·2·1 leaving 7 choices for the second slot, 6
X exactly one element of Y. 0! = 1 choices for the third slot and so on.
n! = n(n - 1)! The number of different orderings is
Domain: In a set of ordered pairs, (x, y), the domain is the set of all x- 1! = 1
coordinates. 2! = 2 8(7)(6)(5)(4)(3)(2)(1)=8! =40,320
Range: In a set of ordered pairs, (x, y), the range is the set of all y- 3! = 6
coordinates. 4!= 3!*4 = 24
What about 10! =
Example: f (x) = x−5
Domain: {x: x 5} Permutations are “arrangements”.
Range: {y: y 0} In a permutation, the order of the books is important.
Each different permutation is a different arrangement.
Example: f ( x ) = x2 − 5
Domain: All Real The arrangement ABC is different from the arrangement CBA, even
though they are the same 3 books.
Range: {y: y -5}
The number of r-permutations of a set S with n=|S| elements is
P(n,r) = n(n−1)…(n−r+1) = n!/(n−r)

! " # ! " #
! "# $ %&'&
There are some problems where the order of the items is NOT
important. These are called combinations.
You are just making selections, not making different arrangements. n n!
C (n, r ) = =
Example: A committee of 3 students must be selected from a group of 5 r r !(n − r )!
people. How many possible different committees could be formed?
Let’s call the 5 people A,B,C,D,and E.
Suppose the selected committee consists of students E, C, and A. If Essentially unordered permutations …
you re-arrange the names to C, A, and E, it’s still the same group of
people. This is why the order is not important. P (n, r ) = C (n, r ) P (r , r )
Because we’re not going to use all the possible combinations of ECA, like EAC,
CAE, CEA, ACE, and AEC, there will be a lot fewer committees. Therefore instead of Note that C(n,r) = C(n, n−r)
using only 5x4x3, to get the fewer committees, we must divide n P (n, r ) n! /(n − r )! n!
(Always divide by the C ( n, r ) = = = =
5x4x3 Answer: factorial of the number of r P(r , r ) r! r!(n − r )!
3x2x1 10 committees digits on top of the fraction.)
 $ % # $ % #
“Algebra” means, roughly, “relationships”. Vector in Rn is an ordered 1
“Linear Algebra” means, roughly, “line-like relationships”. set of n real numbers. 6
e.g. v = (1,6,3,4) is in R4 3
If 3 feet forward has a 1-foot rise, then going 10x as far should give a 10x
“(1,6,3,4)” is a column
rise (30 feet forward is a 10-foot rise) 4
vector:
Think of a vector as a directed line segment y a as opposed to a row vector:
in N-dimensions! (has “length” and m-by-n matrix is an object
(1 6 3 4)
“direction”) v v= b with m rows and n columns,
Basic idea: convert geometry in higher each entry fill with a real
dimensions into algebra! x
c number:
1 2 8
Once you define a “nice” basis along 4 78 6
each dimension: x-, y-, z-axis …
Vector becomes a 1 x N matrix!
9 3 2
v = [a b c]T
Geometry starts to become linear
algebra on vectors like v!

$ % # $ % #
Line equation: y = mx + c use the head-to-tail method A & '( )$ * ) *
Matrix equation: y = Mx + c to combine vectors

Second order equations:

C
B v = ( x1 , x 2 , , xn ) T
xTMx = c
B n
y = (xTMx)u + Mx v = x i2 Magnitude or “2-
(Magnitude
i =1
… involves quadratic forms like xTMx norm”)
A If v = 1, v is a unit vecto r
Vector Addition: A+B
A + B = ( x1 , x2 ) + ( y1 , y2 ) = ( x1 + y1 , x2 + y2 ) Alternate representations:
(unit vector => pure direction)
Scalar Product: A+B av = a ( x1 , x2 ) = (ax1 , ax2 ) Polar coords: (||v||, θ)
y
Complex numbers: ||v||ejθθ
Change only the length (“scaling”), but keep direction fixed.
||v||
Matrix operation (Av) can change length, direction and also dimensionality!
θ
“phase”
x
 $ % # $ % #
& 5 0 1 5 (stretching)
=
For an n-dimensional vector x = [ x1 x2 ... xn ] 0 5 1 5
1/ p

the vector norm: x p ≡ x xi

p
; p = 1, 2,... 0 −1 1 −1
p = (rotation)
i 1 0 1 1
Special case : x ∞ max xi
i

Most commonly used L2 − norm : x 2 = x = x12 + x2 2 + ...xn 2 0 1 1 0 (reflection)

=
1 0 0 1
+, ) *-%! " " # ( ) *
Name Symbol value 1 0 1 1 (projection)
L1 – norm |v|1 6
1. x > 0 when x ≠ 0; x = 0 iff x = 0 =
2. kx = k x ∀ scalar k
0 0 1 0
L2 – norm |v|2 141/2 ≅ 3.74
L3 – norm |v|3 361/3≅3.3 3. x + y ≤ x + y
1 c x x + cy (shearing)
L∞ – norm |v|∞ 3 =
0 1 y y

$ % # $ % #
x
( x+ y+ z =1 1 1 1 1 .
y =
2 −1 1 2
2x − y + z = 2 z Example +- /

a 0 0 a b c a b 0 0 1 0 0
a 0 0
c d e 0 0 1 0
3 5 3 5
0 b 0 0 d e
b c 0 0 f g h
A= det A = = (3)( 2) − (5)(1) = 6 − 5 = 1
0 0 c 0 0 f d e f
0 0 1 1 2 1 2
0 0 i j

diagonal upper-triangular lower-triangular tri-diagonal I (identity matrix) Def: Minors

Let A =[aij] be an nxn matrix . The ijth minor of A ( or the minor of
Transpose of A: Matrix obtained by interchanging rows and columns of A. aij) is the determinant Mij of the (n-1)x(n-1) submatrix after you
Denoted by A’ or AT. AT = [aji] delete the ith row and the jth column of A.
1 4
AT = 2 5
3 6
Example Find M 23 , M 32 , M 33 , 1 0 2
Symmetric Matrix: = A AT A= 4 3 1
Identity Matrix: A square matrix whose diagonal elements are all 1 and off-
diagonal elements are all zero. Denoted by I. 3 5 1
Null Matrix: A matrix whose all elements are zero.

19 20
 $ % # $ % #
Def: Cofactors
Let A =[aij] be an nxn matrix . The ijth cofactor of A ( or the
0 2 1
cofactor of aij) is defined to be Aij = ( −1) i + j M ij A = 3 −1 2
Example Find A23 , A32 , A33 , signs 4 0 1
1 0 2 + − +
A= 4 3 1 − + −
3 5 1 + − +
−1 2 3 2 3 −1
M 11 = = −1, M 12 = = −5, M 13 = =4
Def: Cofactor matrix
0 1 4 1 4 0
C11 = −1 C12 = 5 C13 = −4
Let A =[aij] be an nxn matrix . The cofactor matrix = [Aij] M 21 = 2, M 22 = −4, M 23 = −8, C21 = −2, C22 = −4, C23 = 8,
Example Find the cofactor matrix signs M 31 = 5, M 32 = −3, M 33 = −6. C31 = 5, C32 = 3, C33 = −6.
+ − +
1 0 2
− + − A = a11C11 + a12C12 + a13C13 = 0(−1) + 2(5) + 1(4) = 14
A= 4 3 1
+ − + = a21C21 + a22C22 + a23C23 = 3(−2) + (−1)(−4) + 2(8) = 14
3 5 1
= a11C11 + a21C21 + a31C31 = 0(−1) + 3(−2) + 4(5) = 14
22

$ % # $ % #
Principal Minor Leading Principal Minor
The leading principal minor of order k of an n x n matrix is obtained
Principal minor of order k: A sub-matrix obtained by deleting any n-k by deleting the last n-k rows and their corresponding columns.
rows and their corresponding columns from an n x n matrix Q. 1 2 3
1 2 3 Q= 4 5 6
Consider
Q= 4 5 6 7 8 9
7 8 9
Leading principal minor of order 1 is 1.
Leading principal minor of order 2 is 1 2
Principal minors of order 1 are diagonal elements 1, 5, and 9.
1 2 1 3 5 6 4 5
Principal minors of order 2 are
, and Leading principal minor of order 3 is Q itself.
4 5 7 9 8 9 No. of leading principal determinants of an n x n matrix is n.
Principal minor of order 3 is Q.

Determinant of a principal minor is called principal determinant.

There are 2n-1 principal determinants for an n x n matrix.

 $ % # $ % #
Tests for positive definite matrices
All diagonal elements must be positive.
All the leading principal determinants must be positive.
Tests for positive semi-definite matrices
All diagonal elements are non-negative.
All the principal determinants are non-negative.
Tests for negative definite and negative semi-definite matrices
Test the negative of the matrix for positive definiteness or positive
semi-definiteness.
Test for indefinite matrices
At least two of its diagonal elements are of opposite sign.

$ % # $ % #
 $ % # $ % #
Eigenvalues Computation of eigenvalues, and the characteristic equation
If the action of a matrix on a (nonzero) vector changes its magnitude but When a transformation is represented by a square matrix A, the
not its direction, then the vector is called an eigenvector of that matrix. A eigenvalue equation can be expressed as
vector which is "flipped" to point in the opposite direction is also
considered an eigenvector. Each eigenvector is, in effect, multiplied by a
scalar, called the eigenvalue corresponding to that eigenvector. The AX − λ IX = 0
eigenspace corresponding to one eigenvalue of a given matrix is the set of
all eigenvectors of the matrix with that eigenvalue Solve det(A − I) = 0.
Given a linear transformation A, a non-zero vector x is defined to be an
eigenvector of the transformation if it satisfies the eigenvalue equation

AX = λ X
for some scalar . In this situation, the scalar is called an eigenvalue of
A corresponding to the eigenvector x.

$ % # $ % #
$ % # $ % #

$ % # $ % #
 +
/ - - " / - / !,# / / 0!,#
/ / ,
f ( x + h) − f ( x )
f `( x) = lim If k ( x) = f ( x).g ( x) , then: k `( x) = f `( x) g ( x) + g `( x) f ( x)
h→0 h
2 If y = f ( x).g ( x), then:
1 dy df dg dy
= .g ( x ) + . f ( x ) 45 = f `g + g ` f
dx dx dx dx
f ( x + h) − f ( x )
( f ( x + h) − f ( x ) 1. Differentiate y = x 2 sin x
mPQ =
h
f ( x) = x 2 g ( x) = sin x dy
, = f `g + g ` f = 2 x sin x + x 2 cos x
, ,3 dx
f `( x) = 2 x g `( x) = cos x
δy dy
Leibniz Notation: f `( x) = lim =
h→o δx dx

1 5 A secant line is a straight line joining two points on a function

dy f `g − g ` f
=
dx g2

d x3
1. Find
dx sin x

f ( x) = x3 g ( x) = sin x

f `( x) = 3 x 2 g `( x) = cos x A tangent line is a straight line that touches a function at only one point.
The tangent line represents the instantaneous rate of change of the
function at that one point. The slope of the tangent line at a point on the
dy f `g − g ` f 3 x 2 sin x − x 3 cos x function is equal to the derivative of the function at the same point.
= =
dx g2 sin 2 x
 y

A stationary point is an input to a / *

4
function where the derivative is zero
Stationary points (red circles) and
x2
3

inflection points (blue squares).

− 2 ≤ x <1
2
The stationary points in this graph f ( x) = x 1≤ x < 2
are all relative maxima or relative 1
minima. 1
An inflection point, is a point on a curve x +1 2 ≤ x < 4 – 2 – 1 1 2 3 4 x

at which the curvature or concavity 2

– 1

changes sign from plus to minus or from

minus to plus. Domain [−2, 4) – 2

A critical point of a function is

any point (a, f(a)) where f `(a) =
0 or where f `(a) does not exist.
2x − 2 ≤ x <1
f(x) = x2 + 2x + 3 is differentiable everywhere
f `( x) = 1 1≤ x < 2
f(x) = x2/3 is defined for all x and differentiable
for x 0, with the derivative f (x) = 2x−1/3/3. 1
2≤ x<4
2

x2 − 2 ≤ x <1 2x − 2 ≤ x <1
Local extreme values occur either at the end points of the function, turning
f ( x) = x 1≤ x < 2 f `( x) = 1 1≤ x < 2 points or critical points within the interval of the domain.
y

1 1 Consider the function, 5

x +1 2 ≤ x < 4 2≤ x<4
2 2 x2 + 2 x − 3 ≤ x < −1 4

y f ( x) = x 3 −1 ≤ x < 1 3

5
2
A (-2,4) f `(-2) does not exist. −2 x 2 + 8 x − 5 1 ≤ x < 3
6 4
(Right differentiable at x = -2) 1

3
+ Domain [−3,3)
–3 –2 –1 1 2 3 x

. B (0,0) f `(0) = 0. (Turning point) 3 is the local maximum value –1

2

-1 is the local minimum value –2

1 C (1,1) f `(1) does not exist. Left
derivative =2, right derivative = 1 If extrema occurs at end points then they are end point maximums or end
7 point minimums.
– 2 – 1 1 2 3 4 x

– 1
D (2,2) f `(2) does not exist. Left (i) Local maximum / minimum turning points
– 2 derivative = 1, right derivative = 0.5 (ii) End point values
(iii) Critical points
E is not a critical point as it is not in the domain of f.
 -
Network = graph
The Nature of Stationary Points. 1 2 3 1 2 3
Informally a graph is a set of nodes joined
If f `(a) = 0 then a table of values over a suitable interval centred at a by a set of lines or arrows.
provides evidence of the nature of the stationary point that must exist at a. Nodes and edges 4 5 6
4 5 6
G(V, E)
A simpler test does exist. Undirected graph
Directed graph
It is the second derivative test.
V:={1,2,3,4,5,6}
E:={{1,2},{1,5},{2,3},{2,5},{3,4},{4,5},{4,6}}
f `( x ) = 0 and f ``( x ) > 0 then minimum turning point. Many problems can be stated in terms of a graph
The properties of graphs are well-studied
f `( x ) = 0 and f ``( x ) < 0 then maximum turning point.
Many algorithms exists to solve graph problems
f `( x ) = 0 and f ``( x ) = 0 then draw a table of signs. Many problems are already known to be intractable
If the second derivative test is easier to determine than making a table of By reducing an instance of a problem to a standard graph problem, we may
signs then this provides an efficient technique to finding the nature of be able to use well-known graph algorithms to provide an optimal solution
stationary points. Graphs are excellent structures for storing, searching, and retrieving large
amounts of data
Graph theoretic techniques play an important role in increasing the
storage/search efficiency of computational techniques.

- -
Undirected graph Directed graph

loop loop

G=(V,E)
, 2
isolated vertex path: no vertex can be repeated
adjacent example path: a-b-c-d-e
multiple
trail: no edge can be repeated
edges
example trail: a-b-c-d-e-b-d
incidence: an edge (directed or undirected) is incident to a vertex that is walk: no restriction
one of its end points.
example walk: a-b-d-a-b-c /
degree of a vertex: number of edges incident to it
Nodes of a digraph can also be said to have an indegree and an closed: if starting vertex is also ending
outdegree vertex
adjacency: two vertices connected by an edge are adjacent length: number of edges in the path, trail,
or walk

circuit: a closed trail (ex: a-b-c-d-b-e-d-a)

cycle: closed path (ex: a-b-c-d-a)
 - -
acyclic graph (forest): a graph with no cycles
simple graph: an undirected graph with no loops or multiple edges tree: a connected, acyclic graph
between the same two vertices
rooted tree: a tree with a “root” or “distinguished” vertex
multi-graph: any graph that is not simple
leaves: the terminal nodes of a rooted tree
connected graph: all vertex pairs are joined by a path
directed acyclic graph (DAG): a digraph with no cycles
disconnected graph: at least one vertex pairs is not joined by a path
weighted graph: any graph with weights associated with the edges
complete graph: all vertex pairs are adjacent (edge-weighted) and/or the vertices (vertex-weighted)
Kn: the completely connected graph with n vertices
b 10 a
Simple graph a
a e 5
b d 8 c
e
b
K5 2 -3

d e f
6
c d
Disconnected graph
with two components c