0% found this document useful (0 votes)
3 views27 pages

Optimization23 12

Uploaded by

jashmer2099
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
3 views27 pages

Optimization23 12

Uploaded by

jashmer2099
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 27

Optimization

Techniques
for AI
Lecture
CS/AI 12
2101
Recap
 Linear Programming
 Simplex Method

 Start with a BFS

 Go to adjacent BFS with better value

 Repeat till optimality reached

 Duality

 Complementary Slackness Condition

 Checking Optimality

 Transportation Problem
 u-v Method
Integer Programming

Optimization Problems with


integer variables
Integer Programming
 Integer Linear Programming
 Pure ILP- All variables are required to be

integer.
 Mixed ILP- Some variables are restricted to be

integers; the others can take any value.


 (0, 1)-Programming-
 Special case of ILP
 All variables are restricted to be 0 or 1.
 Integer Quadratic Programming-
 Quadratic Prog with integer restriction
 Quadratic (0, 1)-Programming
Integer Linear Programming
… Objective Function

…+ Inequalit
y
…+ Constrai
….. nts
n >>
…+ m

…, NonNegative Constraints

…, are integers Integer Constraints


ILP and LP- Relationship

PLP PILP
Max z = 5x1 + Max z = 5x1 +
8x2 8x2
s.t. x1 + x2  6 s.t. x1 + x2  6
5x1 + 9x2  45 5x1 + 9x2  45
x1 , optimal
the x2 ≥ 0 x1 , x2 ≥ function
value of the objective 0 of PLP
P
z*ILP the optimal value of theinteger
objective
function of PILP
Can we find a technique to solve PILP if we
know the solution technique of PLP
Linear ProgrammingP
LP
Max z = 5x1 + 8x2
6
x1 + x2 = 6 s.t. x1 + x2  6
5 5x1 + 9x2  45
x1 , x2 ≥ 0
4 (2.25, 3.75)

Z=41.25
2 5x1 + 9x2 = 45

1
Z=20

0 1 2 3 4 5 6 7 8
Integer Linear Programming
PILP
Max z = 5x1 + 8x2
6
x1 + x2 = 6 s.t. x1 + x2  6
5 5x1 + 9x2  45
x1 , x2 ≥ 0 integer
4 (2.25, 3.75)

Z=41.25
2 5x1 + 9x2 = 45

1
Z=20

0 1 2 3 4 5 6 7 8
Strategy I
Given an Integer Linear Programming
Problem PILP to find optimal solution x*ILP
and optimal value z*ILP
PPILP PPLP x*
x*LP and
andz*z*
LP
ILP Ignore Relx Solve Relx Relx
Relax any Solve
integer PLP
constraint PRelx
constraints
Known Algo
Relaxati
on
Yes
x*ILP = x*LP Is x*LPx*
Does integer?
Relx satisfy

z*ILP = z*LP relaxed constraints?

N
o
Our Example
PILP PLP
Max z = 5x1 + Max z = 5x1 +
8x2 8x2
s.t. x1 + x2  6 s.t. x1 + x2  6
5x1 + 9x2  45 5x1 + 9x2  45
x1 , x 2 ≥ 0 x1 , x 2 ≥ 0
x* LP =(x1,
integer x2) = (2.25, 3.75) is the
optimal solution of the PLP
It is not integral.
z*LP =41.25 .
Strategy II

Generate 2 or more subproblems P1ILP and P2ILP such


that no feasible solution of PILP is missed.

It is assumed that the subproblems are


simpler/smaller.

If we know z1*ILP and z2*ILP, then


z*ILP = Max (z1*ILP , z2*ILP )
Subproble
m1 P1ILP
Max z = 5x1 + z1*LP =41 and z2*LP =
8x2 39
Z=41
s.t. x1 + x2  6 x1*LP =(1.8, 4) and x2*LP =
5x1 + 9x2  45
5 (3, 3)
x2 ≥ 4

4 x1 , x 2 ≥ 0
(1.8, 4)integer NOTE- x2*Subproblem
LP = x *ILP
2
2 and
z2*LP = z2*ILP
P2ILP
3 (3, 3)
Max z = 5x1 +
8x2
2 s.t. x1 + x2  6
5x1 + 9x2  45
1 Z=39 x2  3
x1 , x 2 ≥ 0
0 integer
1 2 3 4 5 6 7 8
3 Important Concepts for
solving
 Relaxation
 Solving Relaxed problem by simpler
method
 Subproblem Generation
Relaxation and its effect-
Bound
Any optimization problem P – Maximize z= f(x) subject to
constraints C
 In a general sense, Constraints C consists of several kind of
constraints; linear/nonlinear, inequality constraints, non-
negativity constraints, integer constraints.
 A feasible set- set of values of x satisfying the constraints, say
Feasible(C)
 Relaxation – to ignore some of the constraints in C, relaxing
the set of constraints resulting in another optimization
problem.
 We get a different feasible set when C is relaxed to get C’;

Feasible (C )  Feasible(C’) .

The optimal(maximization) value of z with C, (say z*(C )) cannot


exceed the same with C’,
z*(C ) ≤ z*(C’)
LP Relaxation
 The most commonly used relaxation consists in dropping
the constraint that variables be integer.
 For instance, we replace xi {0,1} by 0 ≤ xi ≤ 1.
 Then, optimizing the relaxed problem calls for solving a
linear program – and we know how to optimize LPs
quickly! LP Relaxation
Maximize z = 8x1 + 11x2 + 6x3
+ 4x4
5x1 + 7x2 + 4x3 + 3x4 ≤ 14
Relax it xi ∈ { 0, 1} for all i = 1, 2, 3,
to: 4

Maximize z = 8x1 + 11x2 + 6x3


+ 4x4
5x1 + 7x2 + 4x3 + 3x4 ≤ 14
xi ∈ [0, 1] for all i = 1, 2, 3, 4
Is LP Relaxation the only kind
of relaxation ?

No- We shall learn other relaxations too


Branch & Bound Algorithm
for Integer Linear
Programming
(0, 1)-Knapsack Problem

Maximize z = 8x1 + 11x2 +


P(0,1) 6x3 + 4x4
5x1 + 7x2 + 4x3 + 3x4 ≤ 14
xi ∈ { 0, 1} for all i = 1, 2,
3, 4
(0, 1)-Knapsack Problem

Maximize z = 8x1 + 11x2 +


P(0,1) 6x3 + 4x4
5x1 + 7x2 + 4x3 + 3x4 ≤ 14
xi Relaxation
∈ { 0, 1} for all i = 1, 2, (0,1) constraints
is relaxed, so the
3, A4 primary variables can
assumption is that take any value
the relaxed between 0 and 1.
problem is easier
Maximize
to solve z = 8x1 + 11x2 +
PLP 6x3 + 4x4
5x1 + 7x2 + 4x3 + 3x4 ≤ 14
0 ≤ xi ≤1 for all i = 1, 2, 3,
4
General (0,1) Knapsack
Problem
P(0,1)
Maximize

∈ { 0, 1} for all i = 1, 2, …, n (0,1) constraints


is relaxed, so the
variables can
PLP take any value
between 0 and 1.
Maximize

∈ [ 0, 1] for all i = 1, 2, …, n
Relaxation
A primary
assumption is that
the relaxed
problem is easier
to solve
Simple Technique to Solve
Psimple
A LP technique to solve LP-relaxed Knapsack
Problem PLP
Technique to solve
1. Arrange the variables in the decreasing order of
the ratio as follows.
2. Choose the largest k such that
3. Assign , for i =1, 2, .., k
PLP
Maximize z = 8x1 + 11x2 + 6x3 + 4x4
5x1 + 7x2 + 4x3 + 3x4 ≤ 14 STEP 3a
xi ∈ [ 0, 1] for all i = 1, 2, 3, 4
Assign , for i =1, 2, .., k ,

STEP 1
STEP 3b
,,,
Check that the variables
are arranged as

STEP 2
Choose the largest k such that
k = 1, means (=5) is less than (=14)
k = 2, means , 5+7 is less than 14
k=3, means , 5+7+4 is not less than 14
PLP
Maximize z = 15x1 + 100x2 + 90x3 + 60x4 +40x5 + 15x6 + 10x7 + 1x8
2x1 + 20x2 + 20x3 + 30x4 +40x5 + 30x6 + 60x7 + 10x8 ≤ 102
xi ∈ [ 0, 1] for all i = 1, 2, …, 8

,,,,,,,

Check that the variables are arranged as

Choose the largest k such that


k = 4, and

Assign , for i =1, 2, .., k ,


Strategy I

Ignore
integer Solve
P(0,1) constraints PLP PLP x*LP and z*LP
Max z = 8x1 + 11x2 + Max z = 8x1 + 11x2 +
6x3 + 4x4 6x3 + 4x4
5x1 + 7x2 + 4x3 + 3x4 ≤ 5x1 + 7x2 + 4x3 + 3x4 ≤
14 14
xi ∈ { 0, 1} for all i = xi ∈ [ 0, 1] for all i =
1, 2, 3, 4 1, 2, 3, 4

Is x*LP integer?

N
o
Strategy II P1(0, 1)

Generate 2 or more Max z = 8x1 + 11x2 + 6x3


subproblems P1(0,1) and + 4x4
5x1 + 7x2 + 4x3 + 3x4 ≤
P2(0,1) such that no feasible 14
solution of P(0,1) is missed. x3=1
P(0, xi ∈ { 0, 1} for all i = 1,
2, 3, 4
Max 1) z = 8x + 11x + 6x
1 2 3
+ 4x4
5x1 + 7x2 + 4x3 + 3x4 ≤
P2(0, 1)
14
xi ∈ { 0, 1} for all i = 1, Max z = 8x1 + 11x2 + 6x3
2, 3, 4 + 4x4
5x1 + 7x2 + 4x3 + 3x4 ≤
14
x3=0
xi ∈ { 0, 1} for all i = 1,
2, 3, 4
P1(0, 1) P2(0, 1)
Max z = 8x1 + 11x2 + 6x3 Max z = 8x1 + 11x2 + 6x3
+ 4x4 + 4x4
5x1 + 7x2 + 4x3 + 3x4 ≤ 5x1 + 7x2 + 4x3 + 3x4 ≤
14 14
x3=1 x3=0
xi ∈ { 0, 1} for all i = 1, xi ∈ { 0, 1} for all i = 1,
2, 3, z4 = 6+8x1 + 11x2 +
Max 2, 3, z4 = 8x1 + 11x2 + 4x4
Max
4x4 5x1 + 7x2 + 3x4 ≤ 14
5x1 + 7x2 + 3x4 ≤ 10 xi ∈ { 0, 1} for all i = 1,
xi ∈ { 0, 1} for all i = 1, 2, 4
2, 4
P(0, 1)

x3 =
x3 = 0
1

P2(0, 1)
P1
(0, 1)

x2 =
x2 = 0
1

You might also like