Integer Programming

Fabio Furini

December 8, 2014

Integer Programming

Outline

1
2
3

General Definitions
Branch and bound
Cutting planes

Integer Programming

Optimization Problems

Optimization Problems

Integer Programming

Integer Linear Programming ILP

Notation
An integer linear program is a linear program with the additional
constraint that the variables are required to be integer.
min cT x

(1)

Ax b

(2)

x0

(3)

xN

(4)

where A Rm X n , b Rm , c Rn .
Write an example of the matrix and the vectors...

Integer Programming

Integer Linear Programming ILP

Notation
Equivalent way an writing an ILP:

min

n
X

ci xi

(5)

i=1
n
X

Aij xi bj

j = 1, . . . , m

(6)

xi 0

i = 1, . . . , n

(7)

xi N

i = 1, . . . , n

(8)

i=1

Integer Programming

Integer Linear Programming ILP

Mixed ILP
An ILP when some variables are restricted to be integer and some are not,
the problem is called: mixed integer linear program.
min cT x

(9)

Ax b

(10)

x0

(11)

xj N j = 1, . . . , p

(12)

(Mixed) Binary ILP

An ILP where the integer variables are restricted to be 0 or 1.
x {0, 1}
Integer Programming

Modeling Potential

Modeling with integer variables is useful in a wide variety of

applications/problems:
model logical requirements (boolean conditions)
model managerial decisions
select among different resources
allocate scarce resources
manage portfolios and projects
model fixed costs
many others . . .

Integer Programming

Solvers

Commercial solvers:

Open solvers:

CPLEX

SCIP

GUROBI

CBC

XPRESS

GLPK

EXCEL

LPSOLVE
Libre Office

Problems with more than a few thousand variables are often not possible
to solve unless they show a specific exploitable structure.

Integer Programming

Exact Algorithms

Branch and Bound

Cutting planes
Branch and Cut
Branch and Price
Dynamic Programming
A combination of the above methods

Integer Programming

Heuristic Algorithms

Genetic algorithm
Tabu Search
Simulated Annealing
Local Search
Ant Colony Optimization
Metaheuristic
A combination of the above methods

Integer Programming

10

Example 1

Suppose we wish to invest 19,000 e and we have four investment

opportunities:
1 investment of 6,700 e and has a net present value of 8,000 e
2 investment of 10,000 e and has a net present value of 11,000 e
3 investment of 5,500 e and has a net present value of 6,000 e
4 investment of 3,400 e and has a net present value of 4,000 e
Into which investments should we place our money so as to maximize our
total present value? Such problems are called capital budgeting problems.

Integer Programming

11

Example 1

Decision variable:

xj =

1 then we will make investment j

0 otherwise

max 8, 000x1 + 11, 000x2 + 6, 000x3 + 4, 000x4

(13)

6, 700x1 + 10, 000x2 + 5.500x3 + 3, 400x4 19, 000

(14)

xj {0, 1} i = 1, 2, 3, 4 (15)

Integer Programming

12

Example 1

The optimal linear programming solution is:

the optimal linear programming sol is: x1 =1, x2 =0.89, x3 = 0, x4 = 1
the objective function value is 21,790 e
this solution is not integral
rounding x2 down to 0 gives a feasible sol with a value of 12,000 e
the optimal solution is: x1 =0, x2 =1, x3 = 1, x4 = 1
the objective function value is 21,000 e

Integer Programming

13

Example 1

There are a number of additional constraints we might want to add:

We can only make two investments
If investment 2 is made, then investment 4 must also be made
If investment 1 is made, then investment 3 cannot be made and
vice-versa
x1 + x2 + x3 + x4 2
x2 x4 0
x1 + x3 1

Integer Programming

14

Example 2
As the project team leader, you must determine the best selection of 5 out
of 10 possible projects. Project profits p1 , p2 , . . . , p10 .

1 then we execute project j
xj =
0 otherwise

max

10
X

pj xj

(16)

xj = 5

(17)

j=1
10
X
j=1

xj {0, 1} i = 1, . . . , 10

Integer Programming

(18)

15

Example 2 additional operational constraints

If project 2 is executed, then project 3 must also be executed
x2 x3

(19)

If project 2 is executed, then project 4 cannot be executed

x4 1 x2

(20)

Executing projects 1 and 6 will allow you to execute project 7

x7 x1 ,

x7 x6

(21)

Executing projects 1 or 6 will allow you to execute project 8

x8 x1 + x6

(22)

Executing projects 2 and 3 will prevent you executing project 9

x9 2 (x2 + x3 )

(23)

Executing projects 2 or 3 will prevent you executing project 10

x10 1 x2 ,
Integer Programming

x10 1 x3

(24)
16

Branch and bound

Branch and bound

Integer Programming

17

Branch and bound

Branch and bound has been the most effective technique for solving
MILPs in the following forty years or so. It was proposed in 1960 by
Land and Dong.
It is based on dividing the problem into a number of smaller problems
(branching) and evaluating their quality based on solving the
underlying linear programs (bounding).
However, in the last ten years, cutting planes have made a resurgence
and are now efficiently combined with branch and bound into an
overall procedure called branch and cut. This term was coined by
Padberg and Rinaldi 1987.

Integer Programming

18

Branch and bound

All these approaches involve solving a series of linear programs:
(MILP)

min cT x

(25)

Ax b

(26)

x0

(27)

xj N j = 1, . . . , p

(28)

Linear Programming Relaxation:

(LP)

min cT x

(29)

Ax b

(30)

x0

(31)

The best known (and most successful) methods for solving LPs are the
interior-point and simplex methods.
Integer Programming

19

Branch and bound

Since LP is less constrained than MILP, the following are immediate:

The optimal objective value for LP is less than or equal to the optimal
objective for MILP
If LP is infeasible, then so is MILP
If the optimal solution x of LP satisfies x j integer for j = 1, ..., p,
then x is also optimal for MILP
it gives a bound on the optimal value of MILP

Integer Programming

20

Branch and bound example 1

We first explain branch and bound by solving the following pure integer
linear program:

max x1 + x2

(32)

x1 + x2 2

(33)

8x1 + 2x2 19

(34)

x1 , x2 0

(35)

x1 , x2 N

(36)

The first step is to solve the linear programming relaxation:

x1 = 1.5, x2 = 3.5 with objective value 5.

Integer Programming

21

Branch and bound example 1

How can we cut this sol while preserving the feasible integral solutions?
Integer Programming

22

Branch and bound example 1

One way is to branch, creating two linear programs:

x1 1
x1 2
Clearly, any solution to the integer program must be feasible to one or
the other of these two problems.
We will solve both of these linear programs.

Integer Programming

23

Branch and bound example 1

max x1 + x2 (37)

max x1 + x2 (42)

x1 + x2 2 (38)

x1 + x2 2 (43)

8x1 + 2x2 19 (39)

8x1 + 2x2 19 (44)

x1 1 (40)

x1 2 (45)

x1 , x2 0 (41)

x1 , x2 0 (46)

The solution is
x1 = 1, x2 = 3 with
objective value 4.

Integer Programming

The solution is
x1 = 2, x2 = 1.5 with
objective value 3.5.

24

Branch and bound example 1

These problems can be arranged in a branch-and-bound tree. Each node
of the tree corresponds to one of the problems that were solved:

Figure: B&B tree of example 1

Integer Programming

25

Branch and bound example 1

We can stop the enumeration at a node of the branch-and-bound tree for

three different reasons (when they occur, the node is said to be pruned).
Pruning by integrality occurs when the corresponding linear program
has an optimum solution that is integral.
Pruning by bounds occurs when the objective value of the linear
program at that node is worse than the value of the best feasible
solution found so far.
Pruning by infeasibility occurs when the linear program at that node
is infeasible.

Integer Programming

26

Branch and bound example 2

max 3x1 + x2

(47)

x1 + x2 2

(48)

8x1 + 2x2 19

(49)

x1 , x2 0

(50)

x1 , x2 N

(51)

Exercise: Execute the branch and bound algorithm

Point 
of intersection among two lines:
ax1 + bx2 + c = 0
rx1 + sx2 + t = 0
(, ) = ((cs - bt)/(br - as), (at - cr)/(br - as))

Integer Programming

27

Branch and bound example 2

Figure: B&B tree of example 2

Integer Programming

28

Branch and bound Algorithm

The branch-and-bound algorithm keeps a list of linear programming

problems obtained by relaxing the integrality requirements on the
variables and imposing constraints such as:
xj uj
xj lj

Each such linear program corresponds to a node of the

branch-and-bound tree.
For a node Ni , let zi denote the value of the corresponding linear
program. Let L denote the list of nodes that must still be solved. Let
zU denote an upper bound on the optimum value (initially, the bound
zU can be derived from a heuristic solution of MILP, or it can be set
to +).

Integer Programming

29

Branch and bound Algorithm

0 Initialize L = MILP, zU = +, x = .
1 Terminate? If L = , the solution x is optimal.
2 Select node Choose and delete a problem Ni from L.
3 Bound Solve Ni . If it is infeasible, go to Step 1. Else, let xi be its
solution and zi its objective value.
4 Prune
If zi zU , go to Step 1.
If xi is not feasible for MILP, go to Step 5.
If xi is feasible for MILP, let zU = zi , x = xi . Delete from L all
problems with zj zU . Go to Step 1.

5 Branch From Ni , construct linear programs Ni1 , ..., Nik with smaller
feasible regions whose union contains all the feasible solutions of
(MILP) in Ni . Add Ni1 , ..., Nik to L and go to Step 1.

Integer Programming

30

Branch-and-bound Additional Elements

Formulation (so that the gap zI - zLP is small)

Branching
Node selection
Heuristics (to find a good upper bound zU )

Integer Programming

31

Binary Branching (branching on a variable)

Problem Ni is a linear program. A way of dividing its feasible region is to

impose bounds on a variable. Let xji be one of the fractional values for
j = 1, . . . , p, in the optimal solution x i of Ni .
From problem Ni , we can construct two linear programs (satisfy the
requirements of Step 5):
Nij = Ni with the additional bound xj bxji c
Nij+ = Ni with the additional bound xj dxji e
The advantage of branching on a variable is that the number of
constraints in the linear programs does not increase, since linear
programming solvers treat bounds on variables implicitly.

Integer Programming

32

Strong Branching

Compute for all the fractional variables:

Dij+ = increase of the objective values from Ni to Nij+
Dij = increase of the objective values from Ni to Nij
branch on the variable j = 1, . . . , p such that min(Dij , Dij+ ) is the largest
(thus tightening the lower bound for pruning). Others have proposed to
choose j such that (Dij+ + Dij ) is the largest. Combining these two criteria
is even better, with more weight on the first.
But the computing time of doing it at each node Ni , for every fractional
variable xji , may be too high. Significantly more time should be spent on
these evaluations towards the top of the tree.

Integer Programming

33

Pseudocost Branching

Let fji = xji bxji c be the fractional part of xji , for j = 1, . . . , p. For an
index j such that fji > 0, define:
down pseudocost Pj =
up pseudocost Pj+ =

Dij
fj i

Dij+
1fji

It can be observed that the pseudocosts tend to remain fairly constant

throughout the branch-and-bound tree. Therefore the pseudocosts need
not be computed at each node of the tree. They are estimated instead.

Integer Programming

34

Pseudocost Branching
A good way of initializing the pseudocosts is through strong
branching at the root node or other nodes of the tree when new
variables become fractional for the first time
To update the pseudocost Pj , we average the observations over all
the nodes of the tree where xj was branched on.
The estimated pseudocosts Pj and Pj+ are used to compute
estimates of Dij and Dij+ at node Ni
Dij = Pj fji
Dij+ = Pj+ (1 fji )

Among these candidates, the branching variable xj is chosen to be the

one with largest min(Dij , Dij+ ) (or other criteria such as those
mentioned earlier).

Integer Programming

35

Node selection

How does one choose among the different problems Ni available in Step 2
of the algorithm?
Two goals need to be considered:
finding good feasible solutions (thus decreasing the upper bound zU )
proving optimality of the current best feasible solution (by increasing
the lower bound as quickly as possible)

Integer Programming

36

Node selection
Best estimate criterion
For the first goal, we estimate the value of the best feasible solution
in each node Ni :
Ei = zi +

p
X

min(Pj fji , Pj+ (1 fji ))

j=1

This corresponds to rounding the noninteger solution xi to a nearby

integer solution and using the pseudocosts to estimate the
degradation in objective value.
Depth-first search strategy
It is an effective strategy to quickly look for a feasible solution.
Breadth-first search strategy
It is an effective strategy to increase the global lower bound of the
problem.
Integer Programming

37

Node selection
Best estimate criterion
For the first goal, we estimate the value of the best feasible solution
in each node Ni :
Ei = zi +

p
X

min(Pj fji , Pj+ (1 fji ))

j=1

This corresponds to rounding the noninteger solution xi to a nearby

integer solution and using the pseudocosts to estimate the
degradation in objective value.
Depth-first search strategy
It is an effective strategy to quickly look for a feasible solution.
Breadth-first search strategy
It is an effective strategy to increase the global lower bound of the
problem.
Integer Programming

38

Heuristics

Heuristics are useful for improving the bound zU , which helps in Step
4 for pruning by bounds.
Heuristics are even more important when the branch-and-bound
algorithm is too time consuming and we need to stop the earlier the
optimization
Examples: diving heuristics, rins, solution polishing, feasibility pump
and many others . . .

Integer Programming

39

Local Branching
This heuristic is particularly suited for MILPs that are too large to solve to
optimality, but where the linear programming relaxation can be solved in
reasonable time. For simplicity, assume that all the integer variables are
0,1 valued.
Let x be a feasible solution of (MILP) (found by a diving heuristic, for
example).
The idea is to define a neighborhood of x as follows:
Pp
|xj xj | k
Pj=1
Pp
p
jI :xj =0 xj +
jI :xj =1 (1 xj ) k (linearised)

where k is an integer chosen by the user (for example k = 20)

Instead of getting lost in a huge enumeration tree, the search is
restricted to the neighborhood of x by this constraint

Integer Programming

40

Practical considerations

The most successful node selection strategy may differ depending on

the application.
For this reason, most MILP solvers have several node selection
strategies available as options.
The default strategy is usually a combination of the best estimate
criterion (or a variation) and depth-first search.

Integer Programming

41

Cutting planes

Cutting planes

Integer Programming

42

Gomory Cuts

Suppose we are given an instance of ILP and begin by solving the

LP-relaxation, producing an optimal basic feasible solution xB associated
with basis B.
If the solution is integer then stop.
Otherwise add a cutting plane such that xB :
is not feasible
all feasible integer points remain feasible

Then, repeat above processes.

Integer Programming

43

Gomory Cuts
Consider any equation in the optimized final tableau, say the i-th equation:
X
xBi +
yij xj = yi0
(1G )
j B
/

for some i, 0 i m. Due to nonnegativity of x

X
X
byij cxj
yij xj
j B
/

j B
/

Therefore we have:
xBi +

byij cxj byi0 c

(2G )

j B
/

Subtracting (2G) from (1G), we get the constraint:

X
(yij byij c)xj yi0 byi0 c
j B
/

called the Gomory cut corresponding to the source or generating row i.

Integer Programming

44

Gomory Cuts Example

max x1 + x2

(52)

x1 + x2 2

(53)

8x1 + 2x2 19

(54)

x1 , x2 0

(55)

x1 , x2 {0, 1}

(56)

We first add slack variables x3 and x4 to turn the inequality constraints

into equalities. The problem becomes:

Integer Programming

max z x1 x2 = 0

(57)

x1 + x2 + x3 = 2

(58)

8x1 + 2x2 + x4 = 19

(59)

x1 , x2 , x3 , x4 0

(60)

x1 , x2 , x3 , x4 {0, 1}

(61)
45

Gomory Cuts Example

Solving the linear programming relaxation by the simplex method, we get

the optimal tableau:

z + 0.6x3 + 0.2x4 = 5

(62)

x2 + 0.8x3 + 0.1x4 = 3.5

(63)

x1 0.2x3 + 0.1x4 = 1.5

(64)

x1 , x2 , x3 , x4 0

(65)

The corresponding basic solution is x3 = x4 = 0, x1 = 1.5, x2 = 3.5 and

z = 5.

Integer Programming

46

Gomory Cuts Example

This solution is not integer. Let us generate the Gomory mixed integer cut
corresponding to the equation:

x2 + 0.8x3 + 0.1x4 = 3.5

(66)

8x3 + x4 5.

(67)

we get the GMI cut:

We can express the above Gomory cut in the space (x1 , x2 ). This yields:
x2 3.

Integer Programming

(68)

47

[email protected]

Integer Programming

48