Computational Integer Programming

Lecture 11: Cutting Plane Methods

Dr. Ted Ralphs

Computational MILP Lecture 11 1

Describing conv(S)

• We have seen that, in theory, conv(S) is a polyhedron and has a finite

description.
• If we “simply” construct that description, we could turn our MILP into
an LP.
• So why aren’t IPs easy to solve?
– The size of the description is generally HUGE!
– The number of facets of the TSP polytope for an instance with 120
nodes is more than 10100 times the number of atoms in the universe.
– It is physically impossible to write down a description of this polytope.
– Not only that, but it is very difficult in general to generate these facets
(this problem is not polynomially solvable in general).
Computational MILP Lecture 11 2

For Example

• For a TSP of size 15

– The number of subtour elimination constraints is 16,368.
– The number of comb inequalities is 1, 993, 711, 339, 620.
– These are only two of the know classes of facets for the TSP.
• For a TSP of size 120
– The number of subtour elimination constraints is 0.6 × 1036!
– The number of comb inequalities is approximately 2 × 10179!
Computational MILP Lecture 11 3

Basic Bounding Methods

• Our discussions of branch and bound has so far focused on the use of
three basic bounding methods.
– LP relaxation
– Lagrangian relaxation
– Combinatorial relaxation
• Branch and bound is fundamentally based on the dynamic generation
and imposition of valid disjunctions.
• We will now show how disjunctions can also be exploited to generate
inequalities valid for conv(S).
Computational MILP Lecture 11 4

Cutting Planes

• Recall that the inequality denoted by (π, π0) is valid for a polyhedron P
if πx ≤ π0 ∀x ∈ P.
• The term cutting plane usually refers to an inequality valid for conv(S),
but which is violated by the solution obtained by solving the (current)
LP relaxation.
• Cutting plane methods attempt to improve the bound produced by the LP
relaxation by iteratively adding cutting planes to the initial LP relaxation.
• Adding such inequalities to the LP relaxation may improve the bound
(this is not a guarantee).
• Note that when π and π0 are integer, then π, π0 is a split disjunction for
which X2 = ∅.
Computational MILP Lecture 11 5

The Separation Problem

• Formally, the problem of generating a cutting plane can be stated as

follows.
Separation Problem: Given a polyhedron Q ⊆ Rn and x∗ ∈ Rn,
determine whether x∗ ∈ Q and if not, determine (π, π0), a valid inequality
for Q such that πx∗ > π0.

• This problem is stated here independent of any solution algorithm.

• However, it is typically used as a subroutine inside an iterative method
for improving the LP relaxation.
• In such a case, x∗ is the solution to the LP relaxation (of the current
formulation, including previously generated cuts).
• We will see later that the difficulty of solving this problem exactly is
strongly tied to the difficulty of the optimization problem itself.
Computational MILP Lecture 11 6

Generic Cutting Plane Method

Let P = {x ∈ Rn | Ax ≤ b} be the initial formulation for

max{c>x | x ∈ S}, (MILP)

where S = P ∩ Zr+ × Rn−p

+ , as defined previously.

Cutting Plane Method

P0 ← P
k←0
while TRUE do
Solve the LP relaxation max{c>x | x ∈ Pk } to obtain a solution xk
Solve the problem of separating xk from conv(S)
if xk ∈ conv(S) then
STOP
else
Determine an inequality (π k , π0k ) valid for conv(S) but for which
π >xk > π0k .
end if
Pk+1 ← Pk ∩ {x ∈ Rn | (π k )>x ≤ π0k }.
k ←k+1
end while
Computational MILP Lecture 11 7

Questions to be Answered

• How do we solve the separation problem?

• Will this algorithm terminate?
• If it does terminate, are we guaranteed to obtain an optimal solution?
Computational MILP Lecture 11 8

Methods for Generating Cutting Planes

• Methods for generating cutting planes attempt to solve separation

problem.
• In most cases, the separation problems that arises cannot be solved
exactly, so we either
– solve the separation problem heuristically, or
– solve the separation problem exactly, but for a relaxation.
• The template paradigm for separation consists of restricting the class of
inequalities considered to just those with a specific form.
• This is equivalent, in some sense, to solving the separation problem for
a relaxation.
• Separation algorithm can be generally divided into two classes
– Algorithms that do not assume any specific structure.
– Algorithms that only work in the presence of specific structure.
Computational MILP Lecture 11 9

Generating Cutting Planes: Two Basic Viewpoints

• There are a number of different points of view from which one can derive
the standard methods used to generate cutting planes for general MILPs.
• As we have seen before, there is an algebraic point of view and a
geometric point of view.
• Algebraic:
– Take combinations of the known valid inequalities.
– Use rounding to produce stronger ones.
• Geometric:
– Use a disjunction (as in branching) to generate several disjoint
polyhedra whose union contains S.
– Generate inequalities valid for the convex hull of this union.
• Although these seem like very different approaches, they turn out to be
very closely related.
Computational MILP Lecture 11 10

Generating Valid Inequalities: Algebraic Viewpoint

• Consider the polyhedron P = {x ∈ Rn | Ax ≤ b}.

• Valid inequalities for P can be obtained by taking nonnegative linear
combinations of the rows of (A, b).
• Except for one pathological case1, all valid inequalities for P are either
equivalent to or dominated by an inequality of the form

uAx ≤ ub, u ∈ Rm
+.

• We are simply taking combinations of inequalities existing in the

description, so any such inequalities will be redundant for the LP
relaxation.

1
The pathological case occurs when one or more variables have no explicit upper bound and both the
primal and dual problems are infeasible.
Computational MILP Lecture 11 11

Generating Valid Inequalities for conv(S)

• All inequalities valid for P are also valid for conv(S), but they are not
cutting planes.
• We can do better.
• We need the following simple principle: if a ≤ b and a is an integer, then
a ≤ bbc.
• Believe it or not, this simple fact is all we need to generate all valid
inequalities for conv(S)!
Computational MILP Lecture 11 12

Back to the Matching Problem

Recall again the matching problem.

X
min cexe
e={i,j}∈E
X
s.t. xij = 1, ∀i ∈ N
{j|{i,j}∈E}

xe ∈ {0, 1}, ∀e = {i, j} ∈ E.

Computational MILP Lecture 11 13

Generating the Odd Cut Inequalities

• Recall that each odd cutset induces a possible valid inequality.

X
xe ≥ 1, S ⊂ N, |S| odd.
e∈δ(S)

• Let’s derive these another way.

– Consider an odd set of nodes UP.
– Sum the (relaxed) constraints {j|{i,j}∈E} xij ≤ 1 for i ∈ U .
P P
– This results in the inequality 2 e∈E(U ) xe + e∈δ(U ) xe ≤ |U |.
1
Dividing through by 2, we obtain e∈E(U ) xe + 2 e∈δ(U ) xe ≤ 12 |U |.
P P
–
– We can drop the second term of the sum to obtain
X 1
xe ≤ |U |.
2
e∈E(U )

– What’s the last step?

Computational MILP Lecture 11 14

Chvátal Inequalities

• Suppose we can find a u ∈ Rm

+ such that π = uA is integer and
π0 = ub 6∈ Z.
• In this case, we have π >x ∈ Z for all x ∈ S, and so π >x ≤ bπ0c for all
x ∈ S.
• In other words, (π, bπ0c) is both a valid inequality and a split disjunction.
Computational MILP Lecture 11 15

Chvátal-Gomory Inequalities

• If we allow the non-negativity constraints to be combined with the

constraints of P (with weight vector v), then integrality of π requires

uAI − vI ∈ Zp
uAC − vC = 0

so
vi ≥ αi − bαic for 0 ≤ i ≤ p
v i = πi ≥ 0 for p + 1 ≤ i ≤ n

• We then obtain that X

buAicxi ≤ bubc (C-G)
0≤i≤p
is valid for all u ∈ Rm
+ such that uAC ≥ 0.

• This is the Chvátal-Gomory Inequality.

Computational MILP Lecture 11 16

The Chvátal-Gomory Procedure

1. Choose a weight vector u ∈ Rm+ such that uAC ≥ 0.

P
2. Obtain the valid inequality 0≤i≤p(uAi)xi ≤ ub.
P
3. Round the coefficients down to obtain 0≤i≤p (buAi c)xi ≤ ub. Why
can we do this?
4. Finally, round the right hand side down to obtain the valid inequality
X
(buAic)xi ≤ bubc
0≤i≤p

• This procedure is called the Chvátal-Gomory rounding procedure, or

simply the C-G procedure.
• Surprisingly, for pure ILPs (p = n), any inequality valid for conv(S) can
be produced by a finite number of iterations of this procedure!
• This is not true for the general mixed case.
Computational MILP Lecture 11 17

Assessing the Procedure

• Although it is theoretically possible to generate any valid inequality using

the C-G procedure, this is not true in practice.
• The two biggest challenges are numerical errors and slow convergence.
• The inequalities produced may be very weak—we may not even obtain a
supporting hyperplane.
• This is is because the rounding only “pushes” the inequality until it meets
some point in Zn, which may or may not even be in S.
• The coefficients of the generated inequality must be relatively prime to
ensure the generated hyperplane even includes an integer point!

n
P
Proposition 1. Let S = {x ∈ Z | j∈N aj xj ≤ b}, where aj ∈ Z
∈ N , and let k = gcd{a1, . . . , an}. Then conv(S) = {x ∈
for j P
Rn | j∈N (aj /k)xj ≤ bb/kc}.
Computational MILP Lecture 11 18

Gomory Inequalities
• Let’s consider T , the set of solutions to a pure ILP with one equation:
 
 Xn 
T = x ∈ Zn+ aj xj = a0
 
j=1

• For each j, let fj = aj − baj c. Then equivalently

 
 Xn 
T = x ∈ Zn+ fj xj = f0 + k for some integer k
 
j=1

Pn
• Since j=1 fj xj ≥ 0 and f0 < 1, then k ≥ 0 and so
n
X
fj xj ≥ f0
j=1

is a valid inequality for S called a Gomory inequality.

• Note that this has been derived as a split disjunction with X2 = ∅.
Computational MILP Lecture 11 19

Gomory Cuts from the Tableau

• Gomory cutting planes can also be derived directly from the tableau while
solving an LP relaxation.
• Consider the set

(x, s) ∈ Zn+m

+ | Ax + Is = b

in which the LP relaxation of an ILP is put in standard form.

• We assume for now that A has integral coefficients so that the slack
variables also have integer values implicitly.
• The tableau corresponding to basis matrix B is

B −1Ax + B −1s = B −1b

• Each row of this tableau corresponds to a weighted combination of the

original constraints.
• The weight vectors are the rows of B −1.
Computational MILP Lecture 11 20

Gomory Cuts from the Tableau (cont.)

• A row of the tableau is obtained by combining the equations in the

standard representation with weight vector λ = Bj−1 to obtain

n
X m
X
(λAj )xj + λisi = λb,
j=1 i=1

where Aj is the j th column of A and λ is a row of B −1.

• Applying the previous procedure, we can obtain the valid inequality

n
X m
X
(λAj − bλAj c)xj + (λi − bλic)si ≥ λb − bλbc.
j=1 i=1

• We will show that this Gomory cut is equivalent to the C-G inequality
with weights ui = λi − bλic.
Computational MILP Lecture 11 21

Gomory Versus C-G

• To show the Gomory cut is a C-G cut, we first apply the C-G procedure
directly to the tableau row, resulting in the inequality

n
X m
X
bλAj cxj + bλicsi ≤ bλbc.
j=1 i=1

• Note that we can also obtain this inequality by adding the Gomory cut
and the original tableau row.
• Now, we substitute out the slack variables using the equation

s = b − Ax.

to obtain
n m
! m
X X X
bλAj c − bλicaij xj ≤ bλbc − bλicbi,
j=1 i=1 i=1
Computational MILP Lecture 11 22

Gomory Versus C-G (cont.)

• The final inequality from the previous slide can be re-written as

n m
! m
X X X
(λi − bλic)aij xj ≤ (λi − bλic)bi,
j=1 i=1 i=1

which is a C-G inequality.

• The substitution of slack variables is more than just a textbook procedure
to show the Gomory cut is a C-G cut.
• In practice, the slack variables are substituted out in this fashion in order
to derive a cut in terms of the original variables.
Computational MILP Lecture 11 23

Strength of Gomory Cuts from the Tableau

• Consider a row of the tableau in which the value of the basic variable is
not an integer.
• Applying the procedure from the last slide, the resulting inequality will
only involve nonbasic variables and will be of the form
X
fj xj ≥ f0
j∈N B

where 0 ≤ fj < 1 and 0 < f0 < 1.

• The left-hand side of this cut has value zero with respect to the solution
to the current LP relaxation.
• We can conclude that the generated inequality will be violated by the
current solution to the LP relaxation.
Computational MILP Lecture 11 24

A Finite Cutting Plane Procedure

• Under mild assumptions on the algorithm used to solve the LP, this yields
a general algorithm for solving (pure) ILPs.
Computational MILP Lecture 11 25

Example: Gomory Cuts

Consider the polyhedron P described by the constraints
4x1 + x2 ≤ 28 (1)
x1 + 4x2 ≤ 27 (2)
x1 − x 2 ≤ 1 (3)
x1, x2 ≥ 0 (4)

Graphically, it can be easily determined that the facet-inducing valid

inequalities describing conv(S = P ∩ Z2) are
x1 + 2x2 ≤ 15 (5)
x1 − x 2 ≤ 1 (6)
x1 ≤ 5 (7)
x2 ≤ 6 (8)
x1 ≥ 0 (9)
x2 ≥ 0 (10)
Computational MILP Lecture 11 26

Example: Gomory Cuts (cont.)

Figure 1: Convex hull of S

Computational MILP Lecture 11 27

Example: Gomory Cuts (cont.)

Consider the optimal tableau of the LP relaxation of the ILP

max{2x1 + 5x2 | x ∈ S},

shown in Table 2.
Basic var. x1 x2 s1 s2 s3 RHS
x2 0 1 -2/30 8/30 0 16/3
s3 0 0 -1/3 1/3 1 2/3
x1 1 0 8/30 -2/30 0 17/3
Table 1: Optimal tableau of the LP relaxation

The associated optimal solution to the LP relaxation is also shown in

Figure 2.
Computational MILP Lecture 11 28

Example: Gomory Cuts (cont.)

The Gomory cut from the first row is

28 8 1
s 1 + s2 ≥ ,
30 30 3

In terms of x1 and x2, we have

4x1 + 2x2 ≤ 33, (G-C1)

Computational MILP Lecture 11 29

Example: Gomory Cuts (cont.)

The Gomory cut from the second row is

2 1 2
s1 + s2 ≥ ,
3 3 3

In terms of x1 and x2, we have

3x1 + 2x2 ≤ 27, (G-C2)

Computational MILP Lecture 11 30

Example: Gomory Cuts (cont.)

The Gomory cut from the third row is

8 28 2
s 1 + s2 ≥ ,
30 30 3

In terms of x1 and x2, we have

x1 + 2x2 ≤ 16, (G-C3)

Computational MILP Lecture 11 31

Example: Gomory Cuts (cont.)

This picture shows the effect of adding all Gomory cuts in the first round.
Computational MILP Lecture 11 32

Generating All Valid Inequalities

• Any valid inequality that can be obtained through iterative application

of the C-G procedure (or is dominated by such an inequality) is a C-G
inequality.
• For pure integer ILPs, all valid inequalities are C-G inequalities.

Theorem 1. Let (π, π0) ∈ Zn+1 be a valid inequality for S = {x ∈

Zn+ | Ax ≤ b} =
6 ∅. Then (π, π0) is a C-G inequality for S.

• The C-G rank denoted r(π, π0) of an inequality (π, π0) valid for P is
defined recursively as follows.
– All inequalities valid for the elementary closure P 1 = e(P) are rank 1.
– The polyhedron P 2 = e(P 1) is the rank 2 closure—inequalities valid
for it that are not rank 1 are rank 2 inequalities.
– An inequality is rank k if it is valid for the rank k closure P k = e(P k−1)
and not for P k−1.
• The C-G rank of P is the maximum rank of any facet-defining inequality
of conv(S).
Computational MILP Lecture 11 33

Valid Inequalities from Disjunctions

• Valid inequalities for conv(S) can also be generated based on disjunctions.
• Let Pi = {x ∈ Rn+ | Aix ≤ bi} for i = 1, . . . , k be such that S ⊆ ∪ki=1Pi.
• Then inequalities valid for ∪ki=1Pi are also valid for conv(S).
• The following procedure shows how to generate such inequalities.

Proposition 2. If (π 1, π01) is valid for S1 ⊆ Rn+ and (π 2, π02) is valid for

S2 ⊆ Rn+, then
n
X
min(πj1, πj2)xj ≤ max(π01, π02) (11)
j=1

for x ∈ S1 ∪ S2.

• In fact, all valid inequalities for the union of two polyhedra can be
obtained in this way.

Proposition 3. If P i = {x ∈ Rn+ | Aix ≤ bi} for i = 1, 2 are nonempty

polyhedra, then (π, π0) is a valid inequality for conv(P 1 ∪ P 2) if and only
if there exist u1, u2 ∈ Rm such π ≤ uiAi and π0 ≥ uibi for i = 1, 2.
Computational MILP Lecture 11 34

Gomory Mixed Integer Inequalities

• Let’s consider again the set of solutions T to an IP with one equation.

• This time, we write T equivalently as
 
 X X 
T = x∈ Zn+ fj x j + (fj − 1)xj = f0 + k for some integer k
 
j:fj ≤f0 j:fj >f0

• Since k ≤ −1 or k ≥ 0, we have the disjunction

X fj X (1 − fj )
xj − xj ≥ 1
f0 f0
j:fj ≤f0 j:fj >f0

OR
X fj X (1 − fj )
− xj + xj ≥ 1
(1 − f0) (1 − f0)
j:fj ≤f0 j:fj >f0
Computational MILP Lecture 11 35

The Gomory Mixed Integer Cut

• Applying Proposition 2, we get
X fj X (1 − fj )
xj + xj ≥ 1
f0 (1 − f0)
j:fj ≤f0 j:fj >f0

• This is called a Gomory mixed integer (GMI) cut.

• GMI cuts dominate the associated Gomory cut and can also be obtained
easily from the tableau.
• In the case of the mixed integer set
 
 n
X 
T = x ∈ Zp+ × Rn−p+ aj xj = a0 ,
 
j=1

the GMI cut is

X fj X (1 − fj ) X aj X aj
xj + xj + xj − xj ≥ 1
f0 (1 − f0) f0 (1 − f0)
0≤j≤p 0≤j≤p p+1≤j≤n p+1≤j≤n
fj ≤f0 fj >f0 aj >0 aj <0
Computational MILP Lecture 11 36

Gomory Mixed Integer Cuts from the Tableau

• Let’s consider how to generate mixed integer Gomory cuts from the
tableau when solving an MILP of the form

Q = {x ∈ Zp+ × Rn−p
+ | Ax ≤ b}.

• We first introduce a slack variable for each inequality in the formulation.

• Solving the LP relaxation, we look for a row in the tableau in which an
integer variable is basic and has a fractional variable.
• We apply the GMI procedure to produce a cut.
• Finally, we substitute out the slack variables in order to express the cut
in terms of the original variables only.
Computational MILP Lecture 11 37

Example: GMI Cuts versus Gomory Cuts

Recall our example from last time.

max 2x1 + 5x2 (12)

s.t. 4x1 + x2 ≤ 28 (13)
x1 + 4x2 ≤ 27 (14)
x1 − x2 ≤ 1 (15)
x1, x2 ≥ 0 (16)

The optimal tableau for the LP relaxation is:

Basic var. x1 x2 s1 s2 s3 RHS
x2 0 1 -2/30 8/30 0 16/3
s3 0 0 -1/3 1/3 1 2/3
x1 1 0 8/30 -2/30 0 17/3
Table 2: Optimal tableau of the LP relaxation

The associated optimal solution to the LP relaxation is also shown in

Figure 2.
Computational MILP Lecture 11 38

Example: Gomory Cuts (cont.)

Figure 2: Convex hull of S

Computational MILP Lecture 11 39

Example: GMI Cuts versus Gomory Cuts (cont.)

The GMI cut from the first row is

1 8
s1 + s2 ≥ 1,
10 10

In terms of x1 and x2, we have

12x1 + 33x2 ≤ 234, (GMI-C1)

Computational MILP Lecture 11 40

Example: GMI Cuts versus Gomory Cuts (cont.)

The GMI cut from the third row is

4 2
s1 + s2 ≥ 1,
10 10

In terms of x1 and x2, we have

3x1 + 2x2 ≤ 26, (GMI-C3)

Computational MILP Lecture 11 41

Geometric Interpretation of GMI Cuts

• To understand the geometric interpretation of GMI cuts, we consider a

relaxation of (??) associated with a basis of the LP relaxation.
• We simply relax the non-negativity constraints on the basic variables to
obtain

T = (x, s) ∈ Zn+m | Ax + Is = b, xN ≥ 0, sN ≥ 0 ,


where xN and sN are the non-basic variables associated with basis B.

• This is equivalent to relaxing the non-binding constraints.
• The convex hull of T is the so-called corner polyhedron associated with
the basis B.
Computational MILP Lecture 11 42

Example: Corner Polyhedron

Figure 3: The corner polyhedron associated with the optimal basis of the
LP relaxation of the earlier example.
Computational MILP Lecture 11 43

GMI Cuts in Practice

Here is an example of the slow convergence sometimes seen in practice.

min 20x1 + 15x2

−2x1 − 3x2 ≤ −5
−4x1 − 2x2 ≤ −15
−3x1 − 4x2 ≤ 20
0 ≤ x1 ≤ 9
0 ≤ x2 ≤ 6
x1, x2 ∈ Z

We will solve this using the naive implementation in CuPPy.

Computational MILP Lecture 11 44

The Polyhedra in Example

Figure 4: Feasible region of Example MILP

Computational MILP Lecture 11 45

First Iteration

• The solution to the LP relaxation is (2, 3.5).

• The tableau row in which x2 is basic is

x2 + 0.3s2 − 0.4s3

• Note that for purposes of illustration, we are explicitly included the bound
constraints in the tableau.
• The GMI is
0.6s2 + 0.8s3 ≥ 1

• In terms of the original variables, this is

−4.8x1 − 4.4x2 ≤ −26

Computational MILP Lecture 11 46

Second Iteration

Figure 5: Feasible region of Example MILP after adding cut

The solution in the second iteration is (1.75, 4) and the cut is

−10.4x1 − 5.8667x2 ≤ −42.6667.
Computational MILP Lecture 11 47

Third Iteration

Figure 6: Feasible region of Example MILP after two cuts

The solution in the third iteration is (2, 3.7273) and the cut is −14.3x1 −
11.7333x2 ≤ −73.3333.
Computational MILP Lecture 11 48

Further Iterations

Figure 7: Feasible region of Example MILP after 100 cuts

Computational MILP Lecture 11 49

Further Iterations

• Note the slow convergence rate.

• Not much progress is being made with each cut.
• After 100 iteration, the solution is (1.9979, 4), which may be “close
enough,” but would not be considered optimal by most solvers.
• It is surprising that such a small MILP would have such a high rank.
• This is at least partly due to numerical errors and the fact that our
implementation is naive.
• We will delve further into these topics later in the course.
Computational MILP Lecture 11 50

Split Inequalities

• Let (α, β) be a split disjunction and define

P1 = P ∩ {x ∈ Rn | α>x ≤ β}
P2 = P ∩ {x ∈ Rn | α>x ≥ β + 1}

• Any inequality valid for conv(P1 ∪ P2) is valid for S and is called a split
inequality.
Computational MILP Lecture 11 51

Separation Problem for Split Inequalities

• The LP (??) can be generalized straightforwardly to produce the most
violated split cut.

max πx̂ − π 0
s.t. π ≤ uA + u0α,
π ≤ vA − v 0α,
π 0 ≥ ub + u0β,
π 0 ≥ vb − v0(β + 1), (SCGLP)
m
X m
X
ui + u0 + vi + v0 = 1
i=1 i=1

u, u0, v, v0 ≥ 0
α ∈ Zn
β∈Z
• The separation problem is a mixed integer nonlinear optimization
problem, however, and is not easy to solve.
Computational MILP Lecture 11 52

Exercise: Experiment with Cutting Planes

• Download http://miplib.zib.de/download/miplib2010-1.1.2-benchmark
zip
• Try solving some instances with and without cutting planes using Cbc.
• Experient with particular classes of cuts.
• A link to instructions for how to use Cbc on the command line is provided
on the course page.