FUNDAMENTALS OF

OPTIMIZATION
Integer Linear Programming
CONTENT
• Relaxation and Bound
• Branch and Bound
• Cutting plan
• Gomory Cut
• Branch and Cut
Relaxation and Bound

• Given an Integer Program (IP)

z = max{cx: x X  Zn}
• Find decreasing sequence of upper bounds
𝑧ഥ1 > 𝑧ഥ1 > . . . > 𝑧ഥ𝑠  z
• Find increasing sequence of lower bounds
𝑧1 < 𝑧1 < . . . < 𝑧𝑡 ≤ z
• Algorithm stop when 𝑧ഥ𝑠 - 𝑧𝑡 ≤ 
Relaxation and Bound
• Primal bounds
• Every feasible solution x*  X provides a lower bound of the
maximization problem: 𝑧 = cx* ≤ z
• Example: in TSP, every close tour is a upper bound of the objective
function (as TSP is a minimization problem)
Relaxation and Bound
• Dual bounds
• Finding upper bounds for a maximization problem (or lower bounds
for a minimization problem) gives dual bounds of the objective
• Definition A problem (RP) zR = max{f(x): xT Rn} is a relaxation of (IP)
z = max{cx: xX  Zn} if:
• XT
• f(x)  cx,  xX
Relaxation and Bound
• Linear Relaxation
• ZLP = max{cx: x P} with P = {x Rn: Ax ≤ b} is a linear relaxation
program of the IP max{cx: x  P  Zn}
Branch and Bound
• Feasible region of P0 is divided into feasible regions of P1 and P2: X(P0)
= X(P1)  X(P2)

𝑧ഥ0
P0
𝑧0

𝑧ഥ01 𝑧ഥ02
P1 P2
𝑧01 𝑧02
Branch and Bound

𝑧ഥ0 = 30 𝑧ഥ0 = 20

𝑧0 = 10 𝑧0 = 15

𝑧ഥ01 = 15 𝑧ഥ02 = 20 𝑧ഥ01 = 15 𝑧ഥ02 = 20

𝑧01 = 15 𝑧02 = 13 𝑧01 = 15 𝑧02 = 13

Branch and Bound

𝑧ഥ0 = 30 𝑧ഥ0 = 25

𝑧0 = 10 𝑧0 = 13

𝑧ഥ01 = 20 𝑧ഥ02 = 25 𝑧ഥ01 = 20 𝑧ഥ02 = 25

𝑧01 = 15 𝑧02 = 23 𝑧01 = 15 𝑧02 = 23

LP-based Branch and Bound
Initial problem S with formulation P on a list L
Incumbent x* is initialized with primal bound z = -INF
while L not empty do{
Select a problem Si with formulation Pi from L
Solve LP relaxation over Pi got dual bound 𝑧ഥ𝑖 and solution xi(LP)
if 𝑧ഥ𝑖 ≤ z then continue; // prune by dual bound
if xi(LP) integer then{
z = 𝑧ഥ𝑖
x* = xi(LP)
}else{
select a component xi of xi(LP) whose value i is fractional
𝑃1𝑖 = Pi  (xi ≤ i), 𝑃2𝑖 = Pi  (xi  i)
add 𝑃1𝑖 and 𝑃2𝑖 to L
}
}
Return x*
Cutting Plane
• Given a MIP max{cz: z X}
• Inequality z ≤ 0 is called a valid inequality if z ≤ 0 is true for all z  X
• Finding valid inequalities allows us to narrow the search space,
transform MIP to corresponding LP in which an optimal solution to LP is
an optimal solution to the original MIP
Cutting Plane

Feasible points
 Example, consider a MIP with X x2
= {(x1,x2): x1 ≤ 10x2, 0 ≤ x1 ≤ 14, .
x2  𝑍+1 } . Valid inequality
 Red lines represent X
.
 x1 ≤ 6 + 4x2 is a valid inequality
(dashed line) x1 ≤ 10x2
3

14
 Example Integer Rounding

• Consider feasible region X = P  Z3 where P = {x𝑅+3 : 5x1 + 9x2 + 13x3 

19}
9 13 19
• From 5x1 + 9x2 + 13x3  19 we have x1 + x2 + x 
5 5 3 5
19
→ x1 + 2x2 + 3x3 
5
• As x1, x2, x3 are integers, so we have
19
x1 + 2x2 + 3x3    = 4 (this is a valid inequality for X)
5
Gomory Cut
• (IP) max {cx: Ax = b, x  0 and integer}
• Solve corresponding linear programming relaxation (LP) max {cx: Ax =
b, x  0}
• Suppose with an optimal basis, the LP is rewritten in the form
𝑎00 + σ𝑗∈𝐽𝑁 𝑎0𝑗𝑥𝑗 → max
𝑥𝐵𝑢 + σ𝑗∈𝐽𝑁 𝑎𝑢𝑗𝑥𝑗 = 𝑎𝑢0, u = 1, 2,…, m
x  0 and integer
with 𝑎0𝑗 ≤ 0 (as these coefficients corresponds to a maximizer), and 𝑎𝑢0  0
Gomory Cut
• If the basic optimal solution x* is not integer, then there exists some row
u with 𝑎𝑢0 is not integer
→ Create a Gomory cut 𝑥𝐵𝑢 + σ𝑗∈𝐽𝑁 𝑎𝑢𝑗𝑥𝑗 ≤ 𝑎𝑢0 (1)

→ Rewriting this inequality (as 𝑥𝐵𝑢 is integer)

σ𝑗∈𝐽𝑁(𝑎𝑢𝑗 − 𝑎𝑢𝑗)𝑥𝑗  𝑎𝑢0 − 𝑎𝑢0
or
σ𝑗∈𝐽𝑁 𝑓𝑢 𝑗 𝑥𝑗  𝑓𝑢 0 (2)
, ,
with 𝑓𝑢, 𝑗 = 𝑎𝑢𝑗 − 𝑎𝑢𝑗 and 𝑓𝑢, 0 = 𝑎𝑢0 − 𝑎𝑢0
• As 0 ≤ 𝑓𝑢, 𝑗 < 1 and 0 < 𝑓𝑢, 0 < 1 and 𝑥𝑗∗ = 0,  j  JN → (2) cuts off x*.
Gomory Cut

• Difference between the left-hand side (LHS) and right-hand side (RHS)
of (1) is integral (as x is integral) → the difference between LHS and
RHS of (2) is also integral

→ rewrite (2) in the form s = σ𝑗∈𝐽𝑁 𝑓𝑢, 𝑗 𝑥𝑗 - 𝑓𝑢, 0 where the slack variable s is
nonnegative integer
Gomory Cut

Consider the Integer Program (IP)

f(x1, x2, x3, x4, x5) = x1 + x2 → max
2x1 + x2 + x3 = 8
3x1 + 4x2 + x4 = 24
x1 - x2 + x5 = 2
x1, x2, x3, x4, x5  0 and integer
Gomory Cut

Solve the LP relaxation

f(x1, x2, x3, x4, x5) = x1 + x2 → max
2x1 + x2 + x3 = 8
3x1 + 4x2 + x4 = 24
x1 - x2 + x5 = 2
x1, x2, x3, x4, x5  0
→ Optimal solution (1.6, 4.8, 0, 0, 5.2) with JB = (1,2,5) and JN = (3, 4)
Rewrite the original IP
f(x1, x2, x3, x4, x5) = 6.4 – 0.2x3 – 0.2x4 → max
x1 + 0.8x3 – 0.2x4 = 1.6
x2 – 0.6x3 + 0.4 x4 = 4.8
x5 – 1.4x3 + 0.6x4 = 5.2
x1, x2, x3, x4, x5  0 and integer
 Gomory Cut

We obtain an equivalent IP
f(x1, x2, x3, x4, x5,x6,x7,x8) = x1 + x2 → max
2x1 + x2 + x3 = 8
3x1 + 4x2 + x4 = 24
x1 - x2 + x5 =2
0.8x3 + 0.8x4 – x6 = 0.6
0.4x3 + 0.4x4 – x7 = 0.8
0.6x3 + 0.6x4 – x8 = 0.2
x1, x2, x3, x4, x5, x6, x7, x8  0 and integer
Gomory Cut

Solve the corresponding LP relaxation

f(x1, x2, x3, x4, x5,x6,x7,x8) = x1 + x2 → max
2x1 + x2 + x3 = 8
3x1 + 4x2 + x4 = 24
x1 - x2 + x5 =2
0.8x3 + 0.8x4 – x6 = 0.6
0.4x3 + 0.4x4 – x7 = 0.8
0.6x3 + 0.6x4 – x8 = 0.2
x1, x2, x3, x4, x5, x6, x7, x8  0
→ Optimal solution x* = (0, 6, 2, 0, 8, 1, 0, 1) which is integer.
→ So (0,6,2,0,8) is an optimal solution to the original problem with
the objective value = 0+6 = 6
 Branch and Cut [Wolsey, 98]
INITIALIZATION z = max{cx: xX} with formulation P, 𝑧 = -INF, incumbent x* empty, init Nodelist

NODE if Nodelist empty, goto EXIT else choose and

remove node i from Nodelist and goto RESTORE

RESTORE formulation Pi of the set Xi, set k = 1 and Pi,1 = Pi

LP RELAXATION Iteration k. Solve 𝑧ҧi,k = max{cx: x  Pi,k }. If

infeasible, prune and goto NODE Else solution xi,k and goto CUT

CUT Iteration k. Try to cut off xi,k. If not cuts found, goto PRUNE Else add
cuts to Pi,k giving Pi,k+1. Increase k by 1, and goto LP RELAXATION

PRUNE If 𝑧ҧi,k ≤ z, goto NODE. If xX, set z = 𝑧ҧi,k, update

incumbent x* = xi,k and goto NODE Else goto BRANCHING

BRANCHING Create two or more new problems and add them to the Nodelist

EXIT Incumbent x* with optimal value z

22
Thank you
for your
attentions!