Optimization

Techniques
for AI
Lecture
CS/AI 12
2101
Recap
 Linear Programming
 Simplex Method

 Start with a BFS

 Go to adjacent BFS with better value

 Repeat till optimality reached

 Duality

 Complementary Slackness Condition

 Checking Optimality

 Transportation Problem
 u-v Method
Integer Programming

Optimization Problems with

integer variables
Integer Programming
 Integer Linear Programming
 Pure ILP- All variables are required to be

integer.
 Mixed ILP- Some variables are restricted to be

integers; the others can take any value.

 (0, 1)-Programming-
 Special case of ILP
 All variables are restricted to be 0 or 1.
 Integer Quadratic Programming-
 Quadratic Prog with integer restriction
 Quadratic (0, 1)-Programming
Integer Linear Programming
… Objective Function

…+ Inequalit
y
…+ Constrai
….. nts
n >>
…+ m

…, NonNegative Constraints

…, are integers Integer Constraints

 ILP and LP- Relationship

PLP PILP
Max z = 5x1 + Max z = 5x1 +
8x2 8x2
s.t. x1 + x2  6 s.t. x1 + x2  6
5x1 + 9x2  45 5x1 + 9x2  45
x1 , optimal
the x2 ≥ 0 x1 , x2 ≥ function
value of the objective 0 of PLP
P
z*ILP the optimal value of theinteger
objective
function of PILP
Can we find a technique to solve PILP if we
know the solution technique of PLP
Linear ProgrammingP
LP
Max z = 5x1 + 8x2
6
x1 + x2 = 6 s.t. x1 + x2  6
5 5x1 + 9x2  45
x1 , x2 ≥ 0
4 (2.25, 3.75)

Z=41.25
2 5x1 + 9x2 = 45

1
Z=20

0 1 2 3 4 5 6 7 8
Integer Linear Programming
PILP
Max z = 5x1 + 8x2
6
x1 + x2 = 6 s.t. x1 + x2  6
5 5x1 + 9x2  45
x1 , x2 ≥ 0 integer
4 (2.25, 3.75)

Z=41.25
2 5x1 + 9x2 = 45

1
Z=20

0 1 2 3 4 5 6 7 8
Strategy I
Given an Integer Linear Programming
Problem PILP to find optimal solution x*ILP
and optimal value z*ILP
PPILP PPLP x*
x*LP and
andz*z*
LP
ILP Ignore Relx Solve Relx Relx
Relax any Solve
integer PLP
constraint PRelx
constraints
Known Algo
Relaxati
on
Yes
x*ILP = x*LP Is x*LPx*
Does integer?
Relx satisfy

z*ILP = z*LP relaxed constraints?

N
o
Our Example
PILP PLP
Max z = 5x1 + Max z = 5x1 +
8x2 8x2
s.t. x1 + x2  6 s.t. x1 + x2  6
5x1 + 9x2  45 5x1 + 9x2  45
x1 , x 2 ≥ 0 x1 , x 2 ≥ 0
x* LP =(x1,
integer x2) = (2.25, 3.75) is the
optimal solution of the PLP
It is not integral.
z*LP =41.25 .
 Strategy II

Generate 2 or more subproblems P1ILP and P2ILP such

that no feasible solution of PILP is missed.

It is assumed that the subproblems are

simpler/smaller.

If we know z1*ILP and z2*ILP, then

z*ILP = Max (z1*ILP , z2*ILP )
 Subproble
m1 P1ILP
Max z = 5x1 + z1*LP =41 and z2*LP =
8x2 39
Z=41
s.t. x1 + x2  6 x1*LP =(1.8, 4) and x2*LP =
5x1 + 9x2  45
5 (3, 3)
x2 ≥ 4

4 x1 , x 2 ≥ 0
(1.8, 4)integer NOTE- x2*Subproblem
LP = x *ILP
2
2 and
z2*LP = z2*ILP
P2ILP
3 (3, 3)
Max z = 5x1 +
8x2
2 s.t. x1 + x2  6
5x1 + 9x2  45
1 Z=39 x2  3
x1 , x 2 ≥ 0
0 integer
1 2 3 4 5 6 7 8
3 Important Concepts for
solving
 Relaxation
 Solving Relaxed problem by simpler
method
 Subproblem Generation
Relaxation and its effect-
Bound
Any optimization problem P – Maximize z= f(x) subject to
constraints C
 In a general sense, Constraints C consists of several kind of
constraints; linear/nonlinear, inequality constraints, non-
negativity constraints, integer constraints.
 A feasible set- set of values of x satisfying the constraints, say
Feasible(C)
 Relaxation – to ignore some of the constraints in C, relaxing
the set of constraints resulting in another optimization
problem.
 We get a different feasible set when C is relaxed to get C’;

Feasible (C )  Feasible(C’) .

The optimal(maximization) value of z with C, (say z*(C )) cannot

exceed the same with C’,
z*(C ) ≤ z*(C’)
LP Relaxation
 The most commonly used relaxation consists in dropping
the constraint that variables be integer.
 For instance, we replace xi {0,1} by 0 ≤ xi ≤ 1.
 Then, optimizing the relaxed problem calls for solving a
linear program – and we know how to optimize LPs
quickly! LP Relaxation
Maximize z = 8x1 + 11x2 + 6x3
+ 4x4
5x1 + 7x2 + 4x3 + 3x4 ≤ 14
Relax it xi ∈ { 0, 1} for all i = 1, 2, 3,
to: 4

Maximize z = 8x1 + 11x2 + 6x3

+ 4x4
5x1 + 7x2 + 4x3 + 3x4 ≤ 14
xi ∈ [0, 1] for all i = 1, 2, 3, 4
Is LP Relaxation the only kind
of relaxation ?

No- We shall learn other relaxations too

Branch & Bound Algorithm
for Integer Linear
Programming
(0, 1)-Knapsack Problem

Maximize z = 8x1 + 11x2 +

P(0,1) 6x3 + 4x4
5x1 + 7x2 + 4x3 + 3x4 ≤ 14
xi ∈ { 0, 1} for all i = 1, 2,
3, 4
(0, 1)-Knapsack Problem

Maximize z = 8x1 + 11x2 +

P(0,1) 6x3 + 4x4
5x1 + 7x2 + 4x3 + 3x4 ≤ 14
xi Relaxation
∈ { 0, 1} for all i = 1, 2, (0,1) constraints
is relaxed, so the
3, A4 primary variables can
assumption is that take any value
the relaxed between 0 and 1.
problem is easier
Maximize
to solve z = 8x1 + 11x2 +
PLP 6x3 + 4x4
5x1 + 7x2 + 4x3 + 3x4 ≤ 14
0 ≤ xi ≤1 for all i = 1, 2, 3,
4
 General (0,1) Knapsack
Problem
P(0,1)
Maximize

∈ { 0, 1} for all i = 1, 2, …, n (0,1) constraints

is relaxed, so the
variables can
PLP take any value
between 0 and 1.
Maximize

∈ [ 0, 1] for all i = 1, 2, …, n
Relaxation
A primary
assumption is that
the relaxed
problem is easier
to solve
 Simple Technique to Solve
Psimple
A LP technique to solve LP-relaxed Knapsack
Problem PLP
Technique to solve
1. Arrange the variables in the decreasing order of
the ratio as follows.
2. Choose the largest k such that
3. Assign , for i =1, 2, .., k
 PLP
Maximize z = 8x1 + 11x2 + 6x3 + 4x4
5x1 + 7x2 + 4x3 + 3x4 ≤ 14 STEP 3a
xi ∈ [ 0, 1] for all i = 1, 2, 3, 4
Assign , for i =1, 2, .., k ,

STEP 1
STEP 3b
,,,
Check that the variables
are arranged as

STEP 2
Choose the largest k such that
k = 1, means (=5) is less than (=14)
k = 2, means , 5+7 is less than 14
k=3, means , 5+7+4 is not less than 14
 PLP
Maximize z = 15x1 + 100x2 + 90x3 + 60x4 +40x5 + 15x6 + 10x7 + 1x8
2x1 + 20x2 + 20x3 + 30x4 +40x5 + 30x6 + 60x7 + 10x8 ≤ 102
xi ∈ [ 0, 1] for all i = 1, 2, …, 8

,,,,,,,

Check that the variables are arranged as

Choose the largest k such that

k = 4, and

Assign , for i =1, 2, .., k ,

 Strategy I

Ignore
integer Solve
P(0,1) constraints PLP PLP x*LP and z*LP
Max z = 8x1 + 11x2 + Max z = 8x1 + 11x2 +
6x3 + 4x4 6x3 + 4x4
5x1 + 7x2 + 4x3 + 3x4 ≤ 5x1 + 7x2 + 4x3 + 3x4 ≤
14 14
xi ∈ { 0, 1} for all i = xi ∈ [ 0, 1] for all i =
1, 2, 3, 4 1, 2, 3, 4

Is x*LP integer?

N
o
 Strategy II P1(0, 1)

Generate 2 or more Max z = 8x1 + 11x2 + 6x3

subproblems P1(0,1) and + 4x4
5x1 + 7x2 + 4x3 + 3x4 ≤
P2(0,1) such that no feasible 14
solution of P(0,1) is missed. x3=1
P(0, xi ∈ { 0, 1} for all i = 1,
2, 3, 4
Max 1) z = 8x + 11x + 6x
1 2 3
+ 4x4
5x1 + 7x2 + 4x3 + 3x4 ≤
P2(0, 1)
14
xi ∈ { 0, 1} for all i = 1, Max z = 8x1 + 11x2 + 6x3
2, 3, 4 + 4x4
5x1 + 7x2 + 4x3 + 3x4 ≤
14
x3=0
xi ∈ { 0, 1} for all i = 1,
2, 3, 4
 P1(0, 1) P2(0, 1)
Max z = 8x1 + 11x2 + 6x3 Max z = 8x1 + 11x2 + 6x3
+ 4x4 + 4x4
5x1 + 7x2 + 4x3 + 3x4 ≤ 5x1 + 7x2 + 4x3 + 3x4 ≤
14 14
x3=1 x3=0
xi ∈ { 0, 1} for all i = 1, xi ∈ { 0, 1} for all i = 1,
2, 3, z4 = 6+8x1 + 11x2 +
Max 2, 3, z4 = 8x1 + 11x2 + 4x4
Max
4x4 5x1 + 7x2 + 3x4 ≤ 14
5x1 + 7x2 + 3x4 ≤ 10 xi ∈ { 0, 1} for all i = 1,
xi ∈ { 0, 1} for all i = 1, 2, 4
2, 4
 P(0, 1)

x3 =
x3 = 0
1

P2(0, 1)
P1
(0, 1)

x2 =
x2 = 0
1