Computational Integer Programming

Lecture 10: Branching

Dr. Ted Ralphs

Computational MILP Lecture 10 1

Reading for This Lecture

• Wolsey Sections 7.4-7.5

• Nemhauser and Wolsey Section II.4.2
• Linderoth and Savelsburgh, (1999)
• Martin (2001)
• Achterberg, Koch, Martin (2005)
• Karamanov and Cornuejols, Branching on General Disjunctions (2007)
• Achterberg, Conflict Analysis in Mixed Integer Programming (2007)
Computational MILP Lecture 10 2

Branch and Bound Recap

• As usual, suppose S is the feasible region of an MILP and we wish to

solve maxx∈S c>x.
• To apply branch and bound, we consider a partition of S into subsets
S1, . . . Sk . Then
max c>x = max max c>x.
x∈S 1≤i≤k x∈Si

• In other words, we can optimize over each subset separately.

• Idea: If we can’t solve the original problem directly, we might be able to
solve the smaller subproblems recursively.
• Dividing the original problem into subproblems is called branching.
• Taken to the extreme, this scheme is equivalent to complete enumeration.
Computational MILP Lecture 10 3

Branching

• We have now spent several lectures discussing methods for bounding.

• Obtaining tight bounds is the most important aspect of the branch-and-
bound algorithm.
• Branching effectively is a very close second.
• Choosing an effective method of branching can make orders of magnitude
difference in the size of the search tree and the solution time.
Computational MILP Lecture 10 4

Disjunctions and Branching

• Recall that branching is generally achieved by selecting an admissible

disjunction {Xi}ki=1 and using it to partition S, e.g., Si = S ∩ Xi.
• The way this disjunction is selected is called the branching method and
is the topic we now examine.
• Generally speaking, we want x∗ 6∈ ∪1≤i≤k Xi, where x∗ is the (infeasible)
solution produced by solving the bounding problem associated with a
given subproblem.
Computational MILP Lecture 10 5

Split Disjunctions

• The most easily handled disjunctions are those based on dividing the
feasible region using a given hyperplane.
• In such cases, each term of the disjunction can be imposed by addition
of a single inequality.
• A hyperplane defined by a vector π ∈ Rn is said to be integer if πi ∈ Z
for 0 ≤ i ≤ p and πi = 0 for p + 1 ≤ i ≤ n.
• Note that if π is integer, then we have π >x ∈ Z whenever x ∈ Zp ×Rn−p.
• Then the disjunction defined by

X1 = {x ∈ Rn | π >x ≤ π0}, X2 = {x ∈ Rn | πx ≥ π0 + 1}, (1)

is valid when π0 ∈ Z.
• This is known as a split disjunction.
Computational MILP Lecture 10 6

Variable Disjunctions

• The simplest split disjunction is to take π = ei for 0 ≤ i ≤ p, where ei

is the ith unit vector.
• If we branch using such a disjunction, we simply say we are branching on
xj .
• For such a branching disjunction to be admissible, we should have
π0 < x∗i < π0 + 1.
• In the special case of a 0-1 IP, this dichotomy reduces to

xj = 0 OR xj = 1

• In general IP, branching on a variable involves imposing new bound

constraints in each one of the subproblems.
• This is easily handled implicitly in most cases.
• This is the most common method of branching.
• What are the benefits of such a scheme?
Computational MILP Lecture 10 7

The Geometry of Branching

Figure 1: Feasible region of an MILP

Computational MILP Lecture 10 8

The Geometry of Branching (Variable Disjunction)

Figure 2: Branching on disjunction x1 ≤ 2 OR x1 ≥ 3

Computational MILP Lecture 10 9

The Geometry of Branching (Variable Disjunction)

Figure 3: Branching on disjunction x2 ≤ 4 OR x2 ≥ 5 in Subproblem 2

Computational MILP Lecture 10 10

The Geometry of Branching (General Split Disjunction)

Figure 4: Branching on disjunction x1 + 2x2 ≤ 11 OR x1 + 2x2 ≥ 12

Computational MILP Lecture 10 11

The Geometry of Branching (General Split Disjunction)

Figure 5: Branching on hyperplane x1 ≤ 2 OR x1 ≥ 3 in Subproblem 1

Computational MILP Lecture 10 12

Other Disjunctions

• A generalized upper bound (GUB) is of the form:

X
xj = 1, x ∈ {0, 1}Q
j∈Q

• Suppose |Q| = 10 and we branch on the disjunction x1 ≤ 0 OR x1 ≥ 1.

• How many possible solutions to the above equation are there in each of
the branches? Is this a “good” disjunction to branch on?
P5 P10
• Consider the disjunction j=1 xj = 0 OR j=6 xj = 0.

• Is this better?
Computational MILP Lecture 10 13

Logical Disjunctions

• We can derive other types of branching based on logical considerations.

• Example #1:
– yi binary variable and yi = 0 ⇒ πx ≤ π0.
– Possible branching:

yi = 1,
yi = 0 and πx ≤ π0.

– This avoids using the big M method.

• Example #2: Solving the TSP with Lagrangian relaxation.
Computational MILP Lecture 10 14

Choosing a Branching Disjunction

• What is the real goal of branching?

• This may depend on the goal of the search
– Find the best feasible solution possible in a limited time.
– Find the provably optimal solution as quickly as possible.
• It is difficult to know how our branching decision will impact these goals,
but we may want to choose a branching that
– Decreases the upper bound,
– Increases the lower bound, or
– Result in one or more (nearly) infeasible subproblem.
• Most of the times, we focus on decreasing the upper bound.
Computational MILP Lecture 10 15

Choosing a Branching Disjunction (cont’d)

• There are many possible disjunctions to choose from.

• We generally choose the branching disjunction based on the predicted
amount of progress it will produce towards our goal.
• If the goal is to minimize time to optimality, bound improvement is often
used as a proxy.
• How do we efficiently predict the bound improvement that will result
from the imposition of a given disjunction?
Computational MILP Lecture 10 16

Strong Branching

• Strong branching provides the most accurate estimate, but is

computationally very expensive.
• The idea is to compute the actual change in bound by solving the
bounding problems resulting from imposing the disjunction.
• This can be very costly. How can we moderate this?
– Do only a limited number of dual-simplex pivots for each candidate
for each child.
– Use this as an estimate.
• Empirically, this reduces number of nodes, but this must be traded
against the computational expense.
Computational MILP Lecture 10 17

Pseudocost Branching

• An alternative to strong branching is pseudocost branching

• This is suitable primarily for branching on branching on variables.
• The pseudocost of a variable is an estimate derived by averaging observed
changes resulting from branching on each of the variables.
• For each variable, we maintain an “up pseudocost” (Pj+) and a “down
pseudocost” (Pj−).
• Then the change in bound for each child can be estimated as:

Dj+ = Pj+(1 − fj )
Dj− = Pj−fj ,

where fj = x∗j − bx∗j c.

• In other words, Dj+ and Dj− are estimates of the change in bound that
will result from imposing xj ≥ bx∗j c and xj ≥ dx∗j e, respectively.
Computational MILP Lecture 10 18

Pseudocost Initialization

• Is it reasonable to assume that effect of branching on a particular variable

is actually roughly the same in different parts of the tree?
• Empirical evidence shows that this is the case.
• Another important question is how to get initial estimates before any
branching has occurred.
• This can be done initially using strong branching.
• After initialization, we switch to pseudocost branching, updating the
pseudocost estimates after each bounding operation.
• A more systematic approach to doing this is to use what is called
reliability branching.
Computational MILP Lecture 10 19

Reliability Branching

• Strong branching is effective in reducing the number of nodes, but can

be costly.
• Using pseudocosts is inexpensive, but requires good initialization.
• Reliability branching combines both.
– Use strong branching in the early stages of the tree. Initialize/update
pseudo-costs of variables using these bounds.
– Once strong branching (or actual branching) has been carried out η
number of times on a variable, only use pseudo-costs after that.
– η is called reliability parameter.
– What does η = 0 imply? What does η = ∞ imply?
– Empirically η = 4 seems to be quite effective.
Computational MILP Lecture 10 20

Comparing Branching Candidates

• So far we have seen, how to evaluate a candidate in several ways.

• Sometimes the choice of candidate is clear after this evaluation.

100 100 100

x1 ≤ 0 x1 ≥ 1 x2 ≤ 0 x2 ≥ 1 x3 ≤ 0 x3 ≥ 1

100 100 10 30 70 80

• Are we minimizing or maximizing?

• Which candidate would you choose?
Computational MILP Lecture 10 21

Comparing Candidates

• However, choice of candidates is not always clear.

• Consider

100 100 100

x1 ≤ 0 x1 ≥ 1 x2 ≤ 0 x2 ≥ 1 x3 ≤ 0 x3 ≥ 1

100 −∞ 10 50 40 40

• Possible metrics (z̃1, z̃2, . . . z̃r are the estimates for r children of a
candidate):
– max
P z̃i
– i z̃i /r
– maxi z̃i − mini z̃i
– α1 maxi z̃i + α2 mini z̃i
Computational MILP Lecture 10 22

Comparing Candidates

• The number of fractional variables (after full strong branching) is another

possible criterion.
• For more criteria based on structure of constraints, see Active-Constraint
Variable Ordering for Faster Feasibility of MILPs, by Patel and
Chinneck, 2006.
Computational MILP Lecture 10 23

Local Branching

• Local branching is a branching scheme that emphasizes finding feasible

solutions over improving the upper bound.
• Consider the solution x∗ to an LP relaxation at a certain node in the
tree of a binary program.
• Let S be the set: {j|x∗j = 0}.
• Consider the disjunction
X X
xj ≤ k OR xj ≥ k + 1
j∈S j∈S

for small k.
• Is this a valid rule?
• Which child is easier to solve?
• For full details, see Local Branching by Fischetti and Lodi.
• We will discuss this and other methods when we talk about primal
heuristics.
Computational MILP Lecture 10 24

Valid Inequalities by Branching

• Note this one of the subproblems obtained by imposing a given binary

disjunction is infeasible, the we obtain a valid inequality!
• This is in some sense what a valid inequality is.
• For the problem in Figure 1, branching on the valid disjunction x2 −x1 ≤ 1
OR x2 − x1 ≥ 2 immediately solves the problem.
• This may make it seem easy to find valid inequalities, but we will see
later why this is not the case.