Integer Programming

Wolfram Wiesemann

November 17, 2009

Contents of this Lecture

Mixed and Pure Integer Programming Problems

Definitions
Obtaining a Pure Integer Progamming Problem

Examples
Capital Budgeting
Depot Location

Further Uses of Integer Variables

Finite-Valued Variables
Logical Operations

Solving IP Problems: Cutting Plane Algorithm

Outline: Cutting Plane Algorithm
Gomory Cuts
Example
Contents of this Lecture

Mixed and Pure Integer Programming Problems

Definitions
Obtaining a Pure Integer Progamming Problem

Examples
Capital Budgeting
Depot Location

Further Uses of Integer Variables

Finite-Valued Variables
Logical Operations

Solving IP Problems: Cutting Plane Algorithm

Outline: Cutting Plane Algorithm
Gomory Cuts
Example
Definitions

◮ Mixed Integer Programming Problem.

min x0 = c T x

subject to

Ax = b
xj ≥ 0 for j ∈ N = {1, . . . , n}
xj ∈ Z for j ∈ Z ⊆ N.

Note: xj ∈ N \ Z are continuous, as before.

◮ Pure Integer Programming Problem. Z = N ∪ {x0 }, i.e.,
all variables (including slack and objective value) are integral.
Can be achieved by scaling.
Contents of this Lecture

Mixed and Pure Integer Programming Problems

Definitions
Obtaining a Pure Integer Progamming Problem

Examples
Capital Budgeting
Depot Location

Further Uses of Integer Variables

Finite-Valued Variables
Logical Operations

Solving IP Problems: Cutting Plane Algorithm

Outline: Cutting Plane Algorithm
Gomory Cuts
Example
Obtaining a Pure Integer Progamming Problem
Consider the following problem:
1 1
min x0 = − x1 − x2
3 2
subject to
2 1 4
x1 + x2 ≤
3 3 3
1 3 2
x1 − x2 ≤
2 2 3
x1 , x2 ≥ 0
x1 , x2 ∈ Z.

This is not a pure integer programming problem:

◮ x0 not integral.
◮ Slack variables not integral.
Obtaining a Pure Integer Progamming Problem

Step 1. Scale the equations of the model.

min x0′ = −2x1 − 3x2 (∗6)

subject to

2x1 + x2 ≤ 4 (∗3)
3x1 − 9x2 ≤ 4 (∗6)
x1 , x2 ≥ 0
x1 , x2 ∈ Z.
Obtaining a Pure Integer Progamming Problem

Step 2. Insert (integral) slack variables:

min x0′ = −2x1 − 3x2

subject to

2x1 + x2 + x3 = 4
3x1 − 9x2 + x4 = 4
x1 , x2 , x3 , x4 ≥ 0
x1 , x2 , x3 , x4 ∈ Z.
Contents of this Lecture

Mixed and Pure Integer Programming Problems

Definitions
Obtaining a Pure Integer Progamming Problem

Examples
Capital Budgeting
Depot Location

Further Uses of Integer Variables

Finite-Valued Variables
Logical Operations

Solving IP Problems: Cutting Plane Algorithm

Outline: Cutting Plane Algorithm
Gomory Cuts
Example
Capital Budgeting
◮ Company has resources i ∈ {1, . . . , m}. Resource i has limited
availability bi .
◮ Company can undertake projects j ∈ {1, . . . , n}. Project j
requires aij units of resource i and gives revenues cj .
◮ Which projects should be undertaken such that the resource
availabilities are observed and the revenues maximised?

n
X
max cj xj
x
j=1

subject to
n
X
aij xj ≤ bi ∀ i ∈ {1, . . . , m}
j=1

xj ∈ {0, 1} ∀ j ∈ {1, . . . , n}
Contents of this Lecture

Mixed and Pure Integer Programming Problems

Definitions
Obtaining a Pure Integer Progamming Problem

Examples
Capital Budgeting
Depot Location

Further Uses of Integer Variables

Finite-Valued Variables
Logical Operations

Solving IP Problems: Cutting Plane Algorithm

Outline: Cutting Plane Algorithm
Gomory Cuts
Example
Depot Location (1)

◮ Company has m potential distribution sites i ∈ {1, . . . , m}.

Building a distribution centre at site i costs fi .
◮ There are n customers j ∈ {1, . . . , n}, each of whose demands
need to be satisfied from one or more distribution centres.
Satisfying fraction xij of customer j’s demand from
distribution centre i costs cij , given that centre i is built.
◮ Which distribution centres should be built, and how should
the customer demand’s be satisfied, to minimise costs?
Depot Location (2)

m
X m X
X n
min f i yi + cij xij
x,y
i =1 i =1 j=1

subject to
m
X
xij = 1 ∀ j ∈ {1, . . . , n}
i =1
xij ≤ yi ∀ i ∈ {1, . . . , m} , j ∈ {1, . . . , n}
xij ≥ 0 ∀ i ∈ {1, . . . , m} , j ∈ {1, . . . , n}
yi ∈ {0, 1} ∀ i ∈ {1, . . . , m}
Contents of this Lecture

Mixed and Pure Integer Programming Problems

Definitions
Obtaining a Pure Integer Progamming Problem

Examples
Capital Budgeting
Depot Location

Further Uses of Integer Variables

Finite-Valued Variables
Logical Operations

Solving IP Problems: Cutting Plane Algorithm

Outline: Cutting Plane Algorithm
Gomory Cuts
Example
Finite-Valued Variables

Assume a variable xj can only take a finite number of values:

xj ∈ {p1 , . . . , pm } .

We can introduce variables zj1 , . . . , zjm ∈ {0, 1} and add the

constraint

zj1 + . . . + zjm = 1.

Now, we can substiute xj with

p1 zj1 + . . . + pm zjm

in the objective function and all constraints.

Finite-Valued Variables

Example. xj ∈ {1, 3, 11} can be modeled as

zj1 + zj2 + zj3 = 1

zj1 , zj2 , zj3 ∈ {0, 1} .

We then substitute xj everywhere by

1zj1 + 3zj2 + 11zj3 .

Exercise. Is it possible to save variables here?

Contents of this Lecture

Mixed and Pure Integer Programming Problems

Definitions
Obtaining a Pure Integer Progamming Problem

Examples
Capital Budgeting
Depot Location

Further Uses of Integer Variables

Finite-Valued Variables
Logical Operations

Solving IP Problems: Cutting Plane Algorithm

Outline: Cutting Plane Algorithm
Gomory Cuts
Example
Logical Operations

We can model logical operations on the constraints via integer

variables. For example, the expression

aT x ≤ b ∨ d Tx ≤ e

can be expressed by:

aT x − Mδ ≤ b
d T x − M(1 − δ) ≤ e
δ ∈ {0, 1} ,

where M is a large number.

Logical Operations
Example. We want to model the following problem:

min x

subject to
x ∈ [0, 1] ∨ x ≥ 2.
Solution. This can be expressed as:

min x

subject to

x ≤ 1 + Mδ
x ≥ 2 − M(1 − δ)
x ≥ 0.
Logical Operations

Example. We want to model the following problem:

min x1 − x2

subject to

x1 + x2 ≤ 4
x1 ≥ 1 ∨ x2 ≥ 1 but not both x1 , x2 > 1
x1 , x2 ≥ 0.
Logical Operations

Solution. This can be expressed as:

min x1 + x2

subject to

x1 + x2 ≤ 4
x1 ≥ 1 − Mδ
x2 ≥ 1 − M(1 − δ)
x1 ≤ 1 + M(1 − δ)
x2 ≤ 1 + Mδ
x1 , x2 ≥ 0.
Contents of this Lecture

Mixed and Pure Integer Programming Problems

Definitions
Obtaining a Pure Integer Progamming Problem

Examples
Capital Budgeting
Depot Location

Further Uses of Integer Variables

Finite-Valued Variables
Logical Operations

Solving IP Problems: Cutting Plane Algorithm

Outline: Cutting Plane Algorithm
Gomory Cuts
Example
The Big Picture

Want to solve a pure IP problem:

◮ Don’t know how to solve IP problems.
◮ Know how to solve continuous problems (Simplex).

Outline of a solution procedure:

◮ Solve a continuous relaxation.
◮ Contains all originally feasible solutions, plus others.
◮ If optimal solution is integral, we are done.
◮ Otherwise, tighten the relaxation and repeat.

Continuous relaxation: xj ∈ Z xj ∈ R.
Tightening: Add cutting planes (cut off current optimum).
The Big Picture

A cutting plane algorithm to solve pure integer programming

problems works as follows.
1. Solve the IP problem with continuous variables instead of
discrete ones.
2. If the resulting optimal solution x ∗ is integral, stop ⇒ optimal
solution found.
3. Generate a cut, i.e., a constraint which is satisfied by all
feasible integer solutions but not by x ∗ .
4. Add this new constraint, resolve problem, and go back to (2).
Terminates after finite number of iterations in (2). Resulting x ∗ is
integral and optimal.
Example

Consider the following problem:

max x0 = 5x1 + 8x2

subject to

x1 + x2 ≤ 6
5x1 + 9x2 ≤ 45
x1 , x2 ≥ 0
x1 , x2 ∈ Z.
Example

Step 1. Solve the IP problem with continuous variables instead of

discrete ones.
6

5o

4o o

3o o o o

2o o o o o

1o o o o o o

0o o o o o o o
0 1 2 3 4 5 6 7 8
Example

Step 2. If the resulting optimal solution x ∗ is integral, stop ⇒

optimal solution found.
6

5o

4o o
x
3o o o o

2o o o o o

1o o o o o o

0o o o o o o o
0 1 2 3 4 5 6 7 8

Resulting solution is x ∗ = (2.25, 3.75) and hence not integral.

Example

Step 3. Generate a cut, i.e., a constraint which is satisfied by all

feasible integer solutions but not by x ∗ .
6

5o

4o o
x
3o o o o

2o o o o o

1o o o o o o

0o o o o o o o
0 1 2 3 4 5 6 7 8

We generate cut 2x1 + 3x2 ≤ 15.

Example
Step 4. Add this new constraint, resolve problem, and go back to
(2).
6

5o

4o o

3o o o o

2o o o o o

1o o o o o o

0o o o o o o o
0 1 2 3 4 5 6 7 8

New optimal solution is x ∗ = (3, 3).

Note: previous x ∗ is not feasible anymore.
Example

Step 2. If the resulting optimal solution x ∗ is integral, stop ⇒

optimal solution found.
6

5o

4o o

3o o o xo

2o o o o o

1o o o o o o

0o o o o o o o
0 1 2 3 4 5 6 7 8

x ∗ = (3, 3) is integral ⇒ optimal solution found.

Example

Remark. The cut we introduced only removed non-integral

solutions. Cuts never cut off feasible solutions of the original IP
problem!
6

5o

4o o

3o o o o

2o o o o o

1o o o o o o

0o o o o o o o
0 1 2 3 4 5 6 7 8
Contents of this Lecture

Mixed and Pure Integer Programming Problems

Definitions
Obtaining a Pure Integer Progamming Problem

Examples
Capital Budgeting
Depot Location

Further Uses of Integer Variables

Finite-Valued Variables
Logical Operations

Solving IP Problems: Cutting Plane Algorithm

Outline: Cutting Plane Algorithm
Gomory Cuts
Example
Ralph E. Gomory (* 1929)

“Outline of an Algorithm for Integer Solutions to Linear Programs”

Bulletin of the American Mathematical Society, Vol. 64,
pp. 275-278, 1958.
Gomory Cut

Assume x1 , . . . , xn ≥ 0 and integral. We show how to construct a

Gomory Cut for
a1 x1 + . . . + an xn = b,
where aj , b ∈ R (not necessarily integral). Note that this can be
written as

(⌊a1 ⌋+[a1 − ⌊a1 ⌋])x1 +. . .+(⌊an ⌋+[an − ⌊an ⌋])xn = ⌊b⌋+[b − ⌊b⌋],
| {z } | {z } | {z }
f1 fn f

where ⌊β⌋ = max {α ∈ Z : α ≤ β} (largest integer smaller than or

equal to β).
Gomory Cut

We separate fractional and integral terms:

(⌊a1 ⌋ + f1 )x1 + . . . + (⌊an ⌋ + fn )xn = ⌊b⌋ + f

⇔ f1 x1 + . . . + fn xn − f = ⌊b⌋ − ⌊a1 ⌋ x1 − . . . − ⌊an ⌋ xn .

Observations.
1. As xj ∈ Z for all feasible x, right-hand side is integral.
2. Thus, for all feasible x, left-hand side must be integral, too.
3. As 0 ≤ f < 1, x ≥ 0 and left-hand side integral, left-hand side
must be non-negative.
Consequence. f1 x1 + . . . + fn xn − f ≥ 0 ⇔ f1 x1 + . . . + fn xn ≥ f
for every feasible x.
Gomory Cut

Suppose Step 1 of our cutting plane algorithm gives non-integral

x ∗ . Then there is row in Simplex tableau with
X
xi∗ + yij xj∗ = yi 0
/
j ∈I

with yi 0 ∈
/ Z.
Gomory Cut. Setting fj := yij − ⌊yij ⌋, f := yi 0 − ⌊yi 0 ⌋, we get:
X
f j xj ≥ f . (∗)
/
j ∈I
P
(∗) is fulfilled for all feasible x but not for x ∗ : ∗
/ f j xj
j ∈I = 0 < f.
Contents of this Lecture

Mixed and Pure Integer Programming Problems

Definitions
Obtaining a Pure Integer Progamming Problem

Examples
Capital Budgeting
Depot Location

Further Uses of Integer Variables

Finite-Valued Variables
Logical Operations

Solving IP Problems: Cutting Plane Algorithm

Outline: Cutting Plane Algorithm
Gomory Cuts
Example
Example

Consider the following problem:

max 3x1 + 4x2

subject to
2
x1 + x2 ≤ 3
5
2 2
x1 − x2 ≤ 1
5 5
x1 , x2 ≥ 0
x1 , x2 ∈ Z.
Example

Step 1. Convert maximisation objective into minimisation.

min x0 = −3x1 − 4x2

subject to
2
x1 + x2 ≤ 3
5
2 2
x1 − x2 ≤ 1
5 5
x1 , x2 ≥ 0
x1 , x2 ∈ Z.
Example

Step 1. Scale the equations of the problem.

min x0 = −3x1 − 4x2

subject to
2
x1 + x2 ≤ 3 (∗5)
5
2 2
x1 − x2 ≤ 1 (∗5)
5 5
x1 , x2 ≥ 0
x1 , x2 ∈ Z.
Example

Step 1. Scale the equations of the problem.

min x0 = −3x1 − 4x2

subject to

2x1 + 5x2 ≤ 15
2x1 − 2x2 ≤ 5
x1 , x2 ≥ 0
x1 , x2 ∈ Z.
Example

Step 1. Insert integral slack variables.

min x0 = −3x1 − 4x2

subject to

2x1 + 5x2 + x3 = 15
2x1 − 2x2 + x4 = 5
x1 , x2 , x3 , x4 ≥ 0
x1 , x2 , x3 , x4 ∈ Z.
Example

Step 1. Solve continuous relaxation of problem.

BV x1 x2 x3 x4 RHS
x0 3 4 0
x3 2 5 1 15
x4 2 -2 1 5
Example

Step 1. Solve continuous relaxation of problem.

BV x1 x2 x3 x4 RHS
x0 3 4 0
x3 2 5 1 15
x4 2 -2 1 5
Example

Step 1. Solve continuous relaxation of problem.

BV x1 x2 x3 x4 RHS
x0 3 4 0
x3 2 5 1 15
x4 2 -2 1 5
Example

Step 1. Solve continuous relaxation of problem.

BV x1 x2 x3 x4 RHS
x0 3 4 0
x3 2 5 1 15
x4 2 -2 1 5
7
x0 5 − 54 -12
2 1
x2 5 1 5 3
14 2
x4 5 5 1 11
Example

Step 1. Solve continuous relaxation of problem.

BV x1 x2 x3 x4 RHS
7
x0 5 − 54 -12
2 1
x2 5 1 5 3
14 2
x4 5 5 1 11
Example

Step 1. Solve continuous relaxation of problem.

BV x1 x2 x3 x4 RHS
7
x0 5 − 54 -12
2 1
x2 5 1 5 3
14 2
x4 5 5 1 11
Example

Step 1. Solve continuous relaxation of problem.

BV x1 x2 x3 x4 RHS
7
x0 5 − 45 -12
2 1
x2 5 1 5 3
14 2
x4 5 5 1 11
Example

Step 1. Solve continuous relaxation of problem.

BV x1 x2 x3 x4 RHS
7
x0 5 − 54 -12
2 1
x2 5 1 5 3
14 2
x4 5 5 1 11
x0 -1 − 12 - 35
2
1
x2 1 7 − 17 10
7
1 5 55
x1 1 7 14 14

Solution optimal; Simplex stops.

Example

Step 3. Generate cut based on x1 row.

1 5 13
x3 + x4 ≥
7 14 14
1 5 13
⇔ (15 − 2x1 − 5x2 ) + (5 − 2x1 − 2x2 ) ≥
7 14 14
⇔ x1 ≤ 3

Introduce new variable x5 with

13 1 5
x5 = − + x3 + x4 = 3 − x1 .
14 7 14
Add this cut to the problem and go back to Step (1).
Example

Step 1. Solve continuous relaxation of problem.

BV x1 x2 x3 x4 x5 RHS
x0 -1 − 21 - 35
2
1
x2 1 7 − 71 10
7
1 5 55
x1 1 7 14 14
1 5 13
ζ 7 14 -1 14
Example

Step 1. Solve continuous relaxation of problem.

BV x1 x2 x3 x4 x5 RHS
x0 -1 − 21 - 35
2
1
x2 1 7 − 71 10
7
1 5 55
x1 1 7 14 14
1 5 13
ζ 7 14 -1 14
Example
Step 1. Solve continuous relaxation of problem.
BV x1 x2 x3 x4 x5 RHS
x0 -1 − 12 - 35
2
1
x2 1 7 − 17 10
7
1 5 55
x1 1 7 14 14
1 5 13
ζ 7 14 -1 14

x0 − 45 − 57 − 81
5
1
x2 1 5 − 52 9
5
x1 1 1 3
2
x4 5 1 − 14
5
13
5

Solution optimal; Simplex stops.

Example

Step 3. Generate cut based on x2 row.

1 3 4
x3 + x5 ≥
5 5 5
1 3 4
⇔ (15 − 2x1 − 5x2 ) + (3 − x1 ) ≥
5 5 5
⇔ x1 + x2 ≤ 4

Introduce new variable x6 with

1 3 4
x6 = x3 + x5 − = 4 − x1 − x2
5 5 5
Add this cut to the problem and go back to Step (1).
Example

Step 1. Solve continuous relaxation of problem.

BV x1 x2 x3 x4 x5 x6 RHS
x0 − 54 − 75 − 81
5
1
x2 1 5 − 25 9
5
x1 1 1 3
2
x4 5 1 − 14
5
13
5
1 3 4
ζ 5 5 -1 5
Example

Step 1. Solve continuous relaxation of problem.

BV x1 x2 x3 x4 x5 x6 RHS
x0 − 54 − 75 − 81
5
1
x2 1 5 − 25 9
5
x1 1 1 3
2
x4 5 1 − 14
5
13
5
1 3 4
ζ 5 5 -1 5
Example
Step 1. Solve continuous relaxation of problem.
BV x1 x2 x3 x4 x5 x6 RHS
4 7 81
x0 5 5 5
1
x2 1 5 − 25 9
5
x1 1 1 3
2
x4 5 1 − 14
5
13
5
1 3 4
ζ 5 5 -1 5

x0 − 31 − 37 − 43
3
1
x2 1 3 − 32 7
3
x1 1 − 31 5
3
5
3
4
x4 3 1 − 14
3
19
3
1
x5 3 1 − 35 4
3
Solution optimal; Simplex stops.
Example

Step 3. Generate cut based on x2 row.

1 1 1
x3 + x6 ≥
3 3 3
1 1 1
⇔ (15 − 2x1 − 5x2 ) + (4 − x1 − x2 ) ≥
3 3 3
⇔ x1 + 2x2 ≤ 6

Introduce new variable x7 with

1 1 1
x7 = x3 + x6 − = 6 − x1 − 2x2
3 3 3
Add this cut to the problem and go back to Step (1).
Example

Step 1. Solve continuous relaxation of problem.

BV x1 x2 x3 x4 x5 x6 x7 RHS
x0 − 31 − 37 − 43
3
1
x2 1 3 − 32 7
3
x1 1 − 31 5
3
5
3
4
x4 3 1 − 14
3
19
3
1
x5 3 1 − 35 4
3
1 1 1
ζ 3 3 -1 3
Example

Step 1. Solve continuous relaxation of problem.

BV x1 x2 x3 x4 x5 x6 x7 RHS
x0 − 31 − 37 − 43
3
1
x2 1 3 − 32 7
3
x1 1 − 31 5
3
5
3
4
x4 3 1 − 14
3
19
3
1
x5 3 1 − 35 4
3
1 1 1
ζ 3 3 -1 3
Example
Step 1. Solve continuous relaxation of problem.
BV x1 x2 x3 x4 x5 x6 x7 RHS
x0 − 31 − 37 − 43
3
1
x2 1 3
− 32 7
3

x1 1 − 31 5
3
5
3
4
x4 3
1 − 14
3
19
3
1
x5 3
1 − 35 4
3
1 1 1
ζ 3 3
-1 3

x0 -2 -1 -14
x2 1 -1 1 2
x1 1 2 -1 2
x4 1 -6 4 5
x5 1 -2 1 1
x3 1 1 -3 1