1

System Planning 2013

Lecture 7: Optimization
Appendix A
Contents:
General about optimization
Formulating optimization problems
Linear Programming (LP)
Mixed Integer Linear Programming (MILP)
2
Optimization General
Discipline in applied mathematics

Used when:
Want to maximize or minimize something
(e.g. profit or cost) under various conditions
(e.g. generation capacities, transmission
limits)


3

Optimization General
Typical problems
In what sequence should parts be produced on a
machine in order to minimize the change-over time?
How can a dress manufacturer lay out its patterns on
rolls of cloth to minimize wasted material?
How many elevators should be installed in a new office
building to achieve an acceptable expected waiting
time?
What is the lowest-cost formula for chicken food which
will provide required quantities of necessary minerals
and other nutrients?
What is the generation plan for the power plants during
the next six hours?




4
Optimization - General
The general optimization problem:
min ( )
s.t. ( )
( )
f x
g x b
h x c
x x x
s
=
s s
x
x
f(x): objective function
x: optimization variables (vector)
g(x), h(x): constraints (vectors)
and : variabel limits (vector)
Defines the feasible set
5
Optimization Formulation
Formulation of optimization problems:

1. Describe the problem in words:
Think through the problem
2. Define symbols
Review the parameters and variables
involved
3. Write the problem mathematically
Translate the problem defined in words
to mathematics
6
Optimization Formulation
The problem:

A manufacturer owns a number of factories. How
should the manufacturer ship his goods to the
customers?
?
7
Optimization Formulation
1. The problem in words:

How should the merchandise be shipped in
order to minimize the transportation costs? The
following must be fulfilled:

The factories cannot produce more than
their production capacities.
The customers must at least receive their
demand
8
2. Define symbols:

Index and parameters:
m factories, factory i has capacity a
i
n customers, customer j demand b
j
units of the
commodity
The transportation cost from factory i to customer
j is c
ij
per unit
Variables
Let x
ij
, i = 1...m, j = 1...n denote the number of units
of the commodity shipped from factory i to
customer j
Optimization Formulation
9
3. Mathematical formulation:
Objective function

Transportation cost:

Constraints

Production capacity:

Demand:

Variable limits:

= =
m
i
n
j
ij ij
x c
1 1
m i a x
i
n
j
ij
... 1
1
= s

=
n j b x
j
m
i
ij
... 1
1
= >

=
n j m i x
ij
... 1 , ... 1 0 = = >
Optimization Formulation
10
The optimization problem:
n j m i x
n j b x
m i a x
x c
ij
j
m
i
ij
i
n
j
ij
m
i
n
j
ij ij
... 1 , ... 1 , 0
... 1 ,
... 1 , s.t.
min
1
1
1 1
= = >
= s
= s

=
=
= =
Optimization Formulation
11
LP General
Linear Programming problems (LP problems)
Class of optimization problems with linear objective
function and constraints
All LP problems can be written as (standard form):
0
s.t.
min
>
=
x
b Ax
x c
T
12
LP General
LP problems will here be explained through
examples (similar as in Appendix A)
Will have to look at the following:
Extreme points
Slack variables
Non existing solution
Not active constraints
No finite solution or unbounded problems
Degenerated solution
Flat optimum
Duality
Solution methods
13
LP Example
The problem:





0 , 0
12 12 4
2 s.t.
10 5 min
2 1
2 1
2 1
2 1
> >
> +
> +
+ =
x x
x x
x x
x x z
14
LP Example
Optimum in:
x
1
+x
2
> 2
4x
1
+12x
2
> 12
x
1
> 0
x
2

>

0

z = 20
z = 30
z = 40
Objective:
5 . 0
5 . 1
2
1
=
=
x
x
5 . 12 = z
x
2
1
2
3
4
x
1
1 2 3 4
15
LP Extreme points
The corners of the
feasible set are called
extreme points
Optimum is always
reached in one or more
extreme points!
x
1
+x
2
> 2
4x
1
+12x
2
> 12
x
1
> 0
x
2

>

0

z = 20
z = 30
z = 40
x
2
1
2
3
4
x
1
1 2 3 4
16
LP Slack variables
LP problem on standard form:





Introduce slack variables in order to write
inequality constraints using equality
0
s.t.
min
>
=
x
b Ax
x c
T
17
LP Slack variables
Without slack variables:
(Ax > b)
0 , 0
12 12 4
2
2 1
2 1
2 1
> >
> +
> +
x x
x x
x x
0 , 0 , 0 , 0
12 12 4
2
4 3 2 1
4 2 1
3 2 1
> > > >
= +
= +
x x x x
x x x
x x x
With slack variables:
(Ax = b)
18
Can happen that the constraints makes no
solution feasible! (Bad formulation of problem)
0 , 0
1
12 12 4
2 s.t.
10 5 min
2 1
2 1
2 1
2 1
2 1
> >
s +
> +
> +
+ =
x x
x x
x x
x x
x x z
New constraint
LP Non existing solution
19
B: Feasible set
defined by (c)
A: Feasible set defined
by (a) & (b)

x
1
+x
2
> 2
4x
1
+12x
2
> 12
x
1
> 0
x
2

>

0

x
2
1
2
3
4
x
1
1 2 3 4
(a)
(b)
x
1
+x
2
s 1
(c)
Feasible
set is
empty!
LP Non existing solution
A B =C
20
LP Not active constraints
Some constraints might not be active in
optimum.

0 , 0
7
12 12 4
2 s.t.
10 5 min
2 1
2 1
2 1
2 1
2 1
> >
s +
> +
> +
+ =
x x
x x
x x
x x
x x z
New constraint
21

x
2
x
1
1
2
3
4
1 2 3 4
x
1
+x
2
> 2
x
2

>

0

4x
1
+12x
2
> 12
x
1
> 0
z = 20
z = 30
z = 40
x
1
+x
2
s 7
Not active constraints
Active
constraints
LP Not active constraints
22
LP No finite solution
If the constraints dont limit the objective: | z|

0 , 0
12 12 4
2 s.t.
min
2 1
2 1
2 1
2 1
> >
> +
> +
=
x x
x x
x x
x x z
New objective function
23

x
1
+x
2
> 2
x
2

>

0

4x
1
+12x
2
> 12
x
1
> 0
x
2
x
1
1
2
3
4
1 2 3 4
z = -5
z = -4
z = -3
min z -
LP No finite solution
24
LP Degenerated solution
More than one extreme point can be optimal. All
linear combinations of these points are then also
optimal

0 , 0
12 12 4
2 s.t.
10 10 min
2 1
2 1
2 1
2 1
> >
> +
> +
+ =
x x
x x
x x
x x z
New objective
25

x
1
+x
2
> 2
x
2

>

0

4x
1
+12x
2
> 12
x
1
> 0
x
2
1
2
3
4
1 2 3 4
z = 30
z = 40
z = 50
The line between
the extreme
points are also
optimal solutions
x
1
LP Degenerated solution
26
LP Flat optimum
A number of extreme points can have almost the
same objective function value.

0 , 0
12 12 4
2 s.t.
99 . 9 10 min
2 1
2 1
2 1
2 1
> >
> +
> +
+ =
x x
x x
x x
x x z
New objective
27
LP Flat optimum
x
1
+x
2
> 2
x
2

>

0

4x
1
+12x
2
> 12
x
1
> 0
x
2
1
2
3
4
1 2 3 4
z = 30
z = 40
z = 50
x
1
z = 19.980
z = 19.995
28
LP Duality
All LP problems (primal problem) have a corresponding dual
problem








are called dual variables
0
s.t.
min
>
=
x
b Ax
x c
T
0
s.t.
max
>
s

c A
b
T
T
Primal problem: Dual problem:
29
LP Duality
Theorem (strong duality):
If the primal problem has an optimal
solution, then also the dual problem
has an optimal solution and the
objective values of these solutions are
the same.
30
LP Duality
Q: What does the dual variables describe?
A: Dual variables = Marginal value of the
constraint that the dual variable represents, i.e.
how the objective function value changes when
the right-hand side of the constraint changes
0
s.t.
min
>
=
x
b Ax
x c
T
One dual variable
for each constraint!
31
LP Duality

x
2
x
1
1
2
3
4
1 2 3 4
x
1
+x
2
> 2
x
2

>

0

4x
1
+12x
2
> 12
x
1
> 0
z = 20
z = 30
z = 40

2
x
1
+x
2
s 7

3
In optimum:

1
> 0 (active)

2
> 0 (active)

3
= 0 (not active)
32
Dual variables
Small changes in right-hand side Ab


Change in objective function value Az:
LP Duality
Az =
T
Ab
33
MILP General
Mixed Integer Linear Programming problems (MILP
problems)
Class of optimization problems with linear objective
function and constraints
Some variables can only take integer values
{ }
1
2
1
2
1
2
min
s.t.
0
0,1, 2,...
T
x
c
x
x
A b
x
x
x
(
(

(
=
(

>
e
34
MILP Example
Optimum in:
x
1
+x
2
> 2
4x
1
+12x
2
> 12
x
1
> 0
x
2

>

0

Objective:
) 5 . 0 ( 1
) 5 . 1 ( 1
2
1
=
=
x
x
) 5 . 12 ( 15 = z
x
2
1
2
3
4
x
1
1 2 3 4
z = 20
z = 30
z = 40
35
MILP Solving
Easy to implement integer variables.
Hard to solve the problems.
Execution times can increase exponentially
with the number of integer variables
Avoid integer variables if possible!

Special case of integer variables: Binary variables
x e {0,1}
36
MILP Example
Minimize the cost for buying something
Variable, x: Quantity of something, x > 0.
Not constant cost per unit - cost function.
For example: Discount when buying more than a
specified quantity:
Cost [SEK]
Number
of units
x
b
x
1
x
2
Split x into two
different variables,
x
1
and x
2
. Observe
that
x
1
x
b
. Also note that
both x
1
and x
2
are > 0.
37
MILP Example
Cheaper to buy in segment 2 Must force the
problem to fill the first segment before entering
segment 2.
Can be performed by introducing a binary
variable.
Cost [SEK]
Number
of units
x
b
x
1
x
2
38
MILP Example
Introduce the following
constraints:
0
0
2
1
> +
>
Ms x
s x x
b
s=0 s=1
Cost [SEK]
Number
of units
x
b
x
1
x
2
Check if s=0: 0 0 0 , 0
2 2 2 1
= s > > x x x x
Check if s=1:
M x M x
x x x x x x
b b b
s > +
= > >
2 2
1 1 1
0
, 0
where M is an arbitrarely large number.
Let s be an integer variable
indicating whether being in
segment 1 or 2.
39
LP Solution methods
Simplex:
Returns optimum in extreme point (also when
degenerated solution)

Interior point methods:
If degenerated solution: Returns solution between two
extreme points
40
LP Solution methods

x
1
+x
2
> 2
x
2

>

0

4x
1
+12x
2
> 12
x
1
> 0
x
2
1
2
3
4
1 2 3 4
z = 30
z = 40
z = 50
Simplex
Interior point method
41
PROBLEM 11 - Solutions to
LP problems

Assume that readymade software is used to
solve an LP problem. In which of the following
cases do you get an optimal solution to the LP
problem and in which cases do you have to
reformulate the problem (or correct an error in
the code).

a) The problem has no feasible solution.
b) The problem is degenerated.
c) The problem does not have a finite solution.