See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/313992289

Branch‐and‐Bound Algorithms

Chapter · January 2011

DOI: 10.1002/9780470400531.eorms0116

CITATIONS READS

8 17,091

1 author:

Kiavash Kianfar
Texas A&M University
27 PUBLICATIONS 223 CITATIONS

SEE PROFILE

All content following this page was uploaded by Kiavash Kianfar on 22 February 2019.

The user has requested enhancement of the downloaded file.

BRANCH-AND-BOUND BRANCH-AND-BOUND BASIC IDEAS
ALGORITHMS
In this section we explain the main concepts
KIAVASH KIANFAR of branch-and-bound as a general discrete
Department of Industrial and Systems optimization approach. Many of branch-and-
Engineering, Texas A&M University, bound concepts discussed here are based on
College Station, Texas Nemhauser and Wolsey [2] and Wolsey [3] the
reader can refer to these resources for further
For most discrete optimization problems, reading. Consider a general combinatorial
complete (or explicit) enumeration of solution optimization problem like
space to find the optimal solution is out of  
z = max f (x) : x ∈ S .
question because even for small problem
sizes the number of solution points in the
feasible region is extremely large (e.g., even if Branch-and-bound is a divide and conquer
a single point can be enumerated in 10−10 s, strategy, which decomposes the problem to
enumerating all points in a relatively small subproblems over a tree structure, which is
problem with only 75 binary variables will referred to as branch-and-bound tree. The
take about 120,000 years!). Branch-and- decomposition works based on a simple idea:
bound is a general purpose approach to If S is decomposed into S1 and S2 such
implicitly enumerate the feasible region and that S = S1 ∪ S2 , and we define subprob-
was first introduced by Land and Doig [1] lems zk = max{f (x) : x ∈ Sk } for k = 1, 2, then
for solving integer programming problems. z = maxk zk . Each subproblem represents a
It works based on a few simple principles node on the tree. Fig. 1 shows a schematic
to avoid enumerating every solution point of a branch-and-bound tree. The main prob-
explicitly. lem with feasible region S (for simplicity
Although branch-and-bound is often dis- we call it problem S) is at the root node,
cussed in the context of integer program- and is then divided into two subproblems
ming, it is actually a general approach that (with feasible regions) S1 and S2 , where we
can also be applied to solve many combi- have S1 ∪ S2 = S. The process of dividing a
natorial optimization problems even when node subproblem into smaller subproblems
they are not formulated as integer programs. is called branching and subproblems S1 and
For example, there are efficient branch-and- S2 are called branches created at node S. In
bound algorithms specifically designed for Fig. 1 subproblem S1 is further branched into
job shop scheduling or quadratic assignment smaller subproblems and so on. The branch-
problems, which are based on the combina- ing does not necessarily have to be two-way,
torial properties of these problems and do and multiway branching is also possible.
not use their integer programming formu- Observe that branching indefinitely will
lations. Nevertheless, it is true that almost only result in explicit enumeration of the fea-
all integer programming solvers use branch- sible region of S. Therefore to avoid explicit
and-bound to solve IP problems and, there- enumeration, in branch-and-bound whenever
fore, application of branch-and-bound in the possible a branch is pruned (or fathomed),
context of IP is of special importance. In meaning that its subproblem is not divided
this article we present the main concepts anymore. In other words, the feasible region
of branch-and-bound in a general setting and of the node subproblem is implicitly enumer-
then discuss some problem-specific details in ated without branching any deeper. But when
the context of integer programming. Many can we prune a branch? The main idea is to
of these details can be extended to more use bounds on the objective value of subprob-
general settings other than IP depending on lems intelligently to prune branches. That is
the problem that is being solved. why the method is called branch-and-bound.

Wiley Encyclopedia of Operations Research and Management Science, edited by James J. Cochran
Copyright © 2010 John Wiley & Sons, Inc.
1
2 BRANCH-AND-BOUND ALGORITHMS

pruned and its solution is stored as the

S incumbent if its objective value is bet-
ter than the best we know so far. The
lower bound z is also updated in this
case.
S1 S2
If a node cannot be pruned based on any of
the above conditions then new branches are
S11 S12 created to decompose the node subproblem
into smaller problems. The algorithm stops
when all nodes are pruned, that is, there is no
active node remaining. The optimal solution
will be the incumbent solution with objec-
Figure 1. Subproblems in a branch-and-bound tive value z. The finiteness of the number of
tree. steps in a branch-and-bound algorithm has
been theoretically studied and proved within
a general formulation in works such as Refs
Fig. 2 shows the flow chart of the branch- 4 and 5.
and-bound algorithm in a general context. This provides a complete high level view to
For a maximization problem, a lower bound branch-and-bound. In a more technical level
z is updated throughout the algorithm and there are many detailed questions that must
is used in pruning the nodes. The updat- be addressed in implementing the algorithm.
ing of the lower bound is normally triggered The answers to many of these questions
by finding an optimal solution with a bet- depend upon the particular problem being
ter objective value at a node subproblem. solved. Some of these questions are as fol-
Clearly, an optimal solution at a node prob- lows [3]:
lem gives a lower bound on z. Additionally,
the branch-and-bound algorithm must have • How should the upper bounds be calcu-
a mechanism to calculate an upper bound for lated?
the objective value of a node subproblem. For • What is the appropriate balance
example, in IP solving the linear program- between the time spent to find an upper
ming (LP) relaxation of the subproblem gives bound and the strength of the upper
an upper bound on the IP objective value. bound?
As observed in Fig. 2, in general, a node is
• How should we do the branching, that
pruned if one of the following cases happens:
is, how should a node subproblem be
decomposed into smaller problems?
Pruning by infeasibility. If the feasible
region of a node subproblem is empty, • How many branches should be created
the node is naturally pruned. at each branching? Two or more?
Pruning by bound. If the upper bound cal- • Should the branching rule be an a priori
culated for a node subproblem is not rule or should it adapt as the algorithm
greater than the lower bound on z, proceeds?
that is, zi ≤ z, then the node is pruned • How should we choose the next active
because there is no point in searching node to consider?
the feasible region of that node when we
know that the best objective value we As mentioned, almost all solvers for IP
can obtain is not better than a solution problems use branch-and-bound. In the next
we already know. section, we discuss the LP-based branch-and-
Pruning by optimality. When it is possi- bound algorithm for solving an IP problem
ble to find the optimal solution to a specifically and address the questions above
node subproblem Si , then the node is in the context of solving the IP problem.
 BRANCH-AND-BOUND ALGORITHMS 3

Initialization (maximization):
Put Initial Problem S at the root node;
lower bound z = –∞;
Incumbent = void:

No STOP
Is there any
Incumbent Solution is optimal;
active node?
Optimal Objective = z

Yes
Choose the next active node
on the tree (say S i )

Yes
Prune by infeasibility Is S i infeasible?

No
i i
Calculate upper bound z for problem S ;

Yes No
Prune by bound z i ≤ z?

If z ≤ z~i then Yes Is it possible to find

Prune by optimality
Incumbent = x~ ; z = z~ ; the optimal solution
i i

x~i to S i?

Branch: No
Create nodes S i k where S i = Uk S i k
i
and add them to the tree as children of S

Figure 2. General branch-and-bound algorithm.

LP-BASED BRANCH-AND-BOUND TO SOLVE Illustrative Example. Consider the IP

INTEGER PROGRAMMING PROBLEMS problem S:

Consider an integer programming problem

max z = 3x1 + 2x2
like
3x1 + 4x2 ≤ 12
  2x1 + x2 ≤ 5
z = max cx : x ∈ S , x1 , x2 ≥ 0 and integer.
 
where S = x : x ∈ Zn ∩ P and P is a polyhe-
dron. As before we refer to this problem as The branch-and-bound tree for solving this
problem S. For simplicity we talk about pure problem is shown in Fig. 4.
integer programming problems but what fol- Observe in Fig. 4 how the calculation of
lows can be easily extended to mixed integer bound in the right branch helps us to prune
programming problems too. Figure 3 shows the left branch and therefore saves us the
the flow chart of the branch-and-bound algo- effort of further enumerating the solutions in
rithm for solving the IP problem S. This flow that branch.
chart is a special case of the general flow Now let us address in more detail some
chart of Fig. 2 customized in order to solve of the questions we posed at the end of
the IP problem. the section titled ‘‘Branch and Bound Basic
4 BRANCH-AND-BOUND ALGORITHMS

Initialization (maximization):
Put Initial Problem S with formulation P at the root node;
lower bound z = –∞;
Incumbent = void:

No STOP
Is there any
Incumbent Solution is optimal;
active node?
Optimal Objective = z

Yes

Choose the next active node on the tree

i i
(say S with formulation P )

Solve LP relaxation over P i;

z i = Objective value of LP;
i
x = LP Solution

Yes
Prune by infeasibility Is P i infeasible?

No
Yes No
Prune by bound z i ≤ z?

Yes Do all variables in

Incumbent = x i; i
Prune by optimality x have integer
z = z~ ;
i
values?

Branch:
i1 i2 No
Create two subproblems S and S
with formulations P i 1 and P i 2 and add
i
them to the tree as children of S

Figure 3. Branch-and-bound flow chart for solving an IP problem.

Solution to LP Relaxation S :
x1 = 1.6; x2 = 1.8; z = 8.4;
Initialize z = – ∞.

x1 ≤ 1 x1 ≥ 2 (this branch considered first)

Solution to LP Relaxation of S 1 = S ∩ {x : x1 ≥ 2}:

Solution to LP Relaxation of S 2 = S ∩ {x : x1 ≤ 1}:
x1 = 2; x2 = 1; z 1 = 8;
x1 = 1; x2 = 2.25; z 2 = 7.5; z 2 =7.5 < 8 = z
Integer solution so update z = 8;
so pruned by bound.
Pruned by optimality.

So the optimal IP solution is x1 = 2; x2 =1; z = 8.

Figure 4. Branch-and-bound tree for the example problem.

 BRANCH-AND-BOUND ALGORITHMS 5

Ideas’’ with respect to branch-and-bound for above single variable branching is used in all
solving the IP problem. solvers.
With the single variable branching strat-
Bounding egy, an important question is which variable
should be chosen out of all variables with frac-
Solving the LP relaxation of any IP subprob- tional value? A recent comprehensive study
lem Si gives an upper bound on its objective on this issue is done in Ref. 7; also see Refs 2,
value. This is the bound that is most com- 3, 8, 9. An old and common rule is the most
monly used in practice. The LP is usually fractional variable. If C is the set of all inte-
solved using simplex-based algorithms. On ger variables with fractional LP relaxation
very large models interior point methods may value then the chosen variable is
be best for solution of the first LP [3]. A desir-
able feature of LP relaxation with simplex is  
argmaxj∈C min fj , 1 − fj ,
that an optimal or near-optimal basis of the
problem can be stored so that the LP relax- where fj = xj − xj . When branching is based
ation in subsequent nodes can be reoptimized on this rule, it is recommended that the
rapidly. xj ≤ xj  branch is considered first only if
fj ≤ 1 − fj . Otherwise the xj ≥ xj should be
Branching selected first. Example in Fig. 4 obeys this
An important question is how to branch, rule. Accordingly the most fractional vari-
meaning how to split a subproblem into able, that is, x1 , is chosen as the branching
smaller subproblems. Here we review the variable and then the right branch is selected
most common methods. after the root node because f1 = x1 − x1  =
0.6 > 0.4 = 1 − (x1 − x1 ) = 1 − f1 .
Single Variable Branching. The simplest The most fractional rule is simple to imple-
idea to split the feasible region of a sub- ment but in Ref. 7, it is shown computation-
problem Si with formulation Pi is to pick ally that this rule is not better than just any
an integer variable with fractional value in random variable selection rule. More effective
the LP relaxation optimal solution, say xj strategies have been proposed and studied
with value xj , and create branches by adding over the years, which are more sophisti-
simple linear constraints to formulation P i cated. The method of pseudocost branching
as follows: goes back to [10] and works based on cal-
culating a pseudocost by keeping a history
  of success (change in LP relaxation value)
Si1 = Si ∩ x : xj ≤ xj 
  of the left and right branching performed on
Si2 = Si ∩ x : xj ≥ xj . each variable. The branching variable with
the highest pesudocost is chosen each time.
Si1 is usually referred to as the left (down) Different functions have been proposed for
branch and Si2 is referred to as the right (up) the pseudocost such as maximum of sum
branch. This is a desirable choice because of left and right degradations (decrease in
clearly we have Si = Si1 ∪ Si2 and Si1 ∩ Si2 = upper bound) [11], maximum of the smaller
∅, and furthermore, the current LP solution of the left and right degradations [10]; also
is not feasible for any of Si1 and Si2 so in see Ref. 8.
the absence of multiple optima for LP the Strong branching [8,12] is in a sense an
upper bound would strictly decrease in each extreme effort to find the best branching vari-
of these branches. This branching scheme able. In its full version, at any node one
was first introduced in Ref. 6. tentatively branches on each candidate vari-
A generalization of the above branching able with fractional value and solves the LP
scheme is using branches such as Si1 = Si ∩ relaxation for right and left branches for
{x : dx ≤ d0 } and Si2 = Si ∩ {x : dx ≥ d0 + 1} each variable either to optimality or for a
in which d and d0 are integer. However, in specified number of dual simplex iterations.
practice, only the simplest form which is the The degradation in bound for left and right
6 BRANCH-AND-BOUND ALGORITHMS

branches are calculated and the variable is branches will be as follows:

picked on the basis of the function of these
degradations [3,8]. In addition to a number Si1 = Si ∩ {x : xji = 0, i = 1, . . . , r}
of dual simplex method iterations, the com-
Si2 = Si ∩ {x : xji = 0, i = r + 1, . . . , n},
putational effort of strong branching can also
be limited by defining a much smaller set of   
candidate branching variables. where r = min t : ti=1 xji ≥ 12 . The number
An effective variable selection strategy of nodes is usually significantly reduced with
was proposed in Ref. 7 called reliability this branching scheme compared to the single
branching. This method integrates strong variable branching.
and pseudocost branching. A threshold
is defined on the number of previous Node Selection
branchings involving a variable. Below Having a list of active (unpruned) nodes, the
this threshold the pseudocost is considered question is which node should be examined
unreliable and strong branching is used but next. There are two categories of rules for
above the threshold the pseudocost method this purpose: static rules that determine in
is used. advance the order of node selection and adap-
We also note that many solvers provide tive rules which use the information about the
the user with the option of assigning user- status of active nodes to choose a node. For
specified priorities to the integer variables. a complete review of different node selection
When branching, the solver picks the variable strategies see Ref. 8. Among static rules the
with a fractional value that has the highest depth-first and the best-bound (or best-first)
assigned priority. This is especially useful are the two well-known extremes.
when the user has some insight regarding the
importance of variables based on what they Depth-First Node Selection. Depth-first
represent in the application underlying the rule also known as last in, first out (LIFO) is
model [3]. as follows: if the current node is not pruned
the next node is one of its children; and
GUB Branching. Many IP models contain if it is pruned the next node is found by
the so-called Generalized Upper Bound backtracking which means the next node is
(GUB) or Special Ordered Set (SOS) the child of the first node on the path from
constraint of the form the current node to the root node with an
unconsidered child node. Obviously, this is a

n completely a priori rule if a rule is specified
xj = 1 to select between left and right children of a
j=1 node. This rule has known advantages:

with xj ∈ {0, 1} for j = 1, . . . , n. If the single 1. For pruning the tree a good lower
variable branching on one of the variables bound is needed. The depth-first
xj , j = 1, . . . , n, is performed then the two method descends quickly in the tree to
branches will be Si1 = Si ∩ {x : xj = 0} and find a first feasible solution which gives
Si2 = Si ∩ {x : xj = 1}. Because of the GUB a lower bound that hopefully causes
constraint, {x : xj = 0} will leave n − 1 possi- the pruning of many future nodes.
bilities {x : xi = 1}i=j while {x : xj = 1} leaves 2. The depth-first method tends to mini-
only one possibility. So Si1 is usually much mize the memory requirements for stor-
larger than Si2 and the tree is unbalanced. ing the tree at any given time during
A more balanced split is desired and GUB the algorithm.
branching proposed in Ref. 13 is a way to get 3. Passing from a node to its immediate
a balanced tree which works as follows: The child has the advantage that the LP
user provides a special order of variables relaxation can be easily resolved by
in the GUB set, say j1 , . . . , jn . Then the two adding just an additional constraint.
 BRANCH-AND-BOUND ALGORITHMS 7

However, the depth-first search can result root problem or the node subproblems to
in an extremely large search tree. This is the tighten the feasible region of the LP relax-
result of the fact that we may need to consider ation and hence obtain better bounds faster.
many nodes that would have been fathomed A branch-and-bound algorithm in which cut-
if we had a better lower bound. ting planes are used is known under the
general name of branch-and-cut [2,3,18].
Best-Bound Node Selection. In this rule In branch-and-cut at any node, after opti-
the next node is the active node with the mizing the LP relaxation, a separation prob-
best (largest) upper bound. With this rule lem is solved to find valid inequalities for
one would never branch a node whose upper feasible integer solutions, which are violated
bound is less than the optimal value. As by the LP relaxation solution. These valid
a result this rule minimizes the number inequalities are then added to the problem
of the nodes considered. However, the and the problem is reoptimized to improve
memory requirements for this method may the LP relaxation bound. Branching hap-
become prohibitive if good lower bounds pens when no further valid inequalities can
are not found early leading to relatively be found. Of course the amount of effort
little pruning. In terms of reoptimizing the spent to find valid inequalities is one of the
LP relaxations this method is also at a parameters of the algorithm that should be
disadvantage because one LP problem has decided. We refer the reader to the article
little relation to the next one. titled Branch and Cut in this encyclopedia
for further information.
Adaptive Node Selection. Adaptive meth- Another extension of branch-and-bound is
ods make intelligent use of information branch-and-price [2,3,19]. When the number
about the nodes to select the next node. of variables in an integer program is huge,
The estimate-based methods attempt to a solution method is using column gener-
select nodes that may lead to improved ation within the branch-and-bound frame-
integer feasible solutions. The best projection work. More specifically, implicit pricing of
criterion [14] and the best estimate criterion nonbasic variables is used to generate new
found in Refs 10 and 15 are among these columns or to prove LP optimality at a node of
methods. the branch-and-bound tree. Branching hap-
Many adaptive methods are two-phase pens when no columns price out to enter the
methods that essentially use a hybrid of basis and the LP solution does not satisfy
depth-first and best-bound searches. At the integrality constraints. We note that if cut-
beginning the depth-first strategy is used ting planes are also used in branch-and-price
to find a feasible solution and then the the algorithm is called branch-cut-price. We
best-bound method is used [16]. Variations refer the reader to the article titled Branch-
of two-phase methods using estimate-based Price-and-Cut Algorithms in this encyclo-
approaches are also proposed [15,17]. Some pedia for further information.
other suggested rules use an estimation of
the optimal IP solution to avoid considering
superfluous nodes, that is, the nodes in which PARALLEL BRANCH-AND-BOUND
zi < z. The tree is searched in a depth-first
fashion as long as zi is greater than the The divide and conquer nature of branch-
estimate. After that a different criterion such and-bound makes it a suitable framework
as best-bound is used [10,11]. for attacking huge problems using today’s
parallel computing capabilities. For a sur-
vey of parallel branch-and-bound algorithms
EXTENSIONS OF BRANCH-AND-BOUND refer to Ref. 20. There are three types of
parallelism that can be implemented for a
In solving integer programming problems, branch-and-bound algorithm: Type 1 is par-
pure branch-and-bound is seldom used. In allel execution of operations on generated
most cases cutting planes are added to the subproblems, for example, parallel bounding
 8 BRANCH-AND-BOUND ALGORITHMS

operations for each subproblem to accelerate 11. Gauthier JM, Ribière G. Experiments in
execution. Type 2 consists of building the tree mixed-integer linear programming using
in parallel by performing operations on sev- pseudocosts. Math Program 1977;12:26–47.
eral subproblems simultaneously. Type 3 is 12. Applegate D, Bixby RE, Chvátal V, et al. The
building several trees in parallel. In each tree traveling salesman problem: a computational
some operation such as branching, bounding study. Princeton (NJ): Princeton University
Press; 2007.
or selection is performed differently but the
trees share their information and use the 13. Beale EML, Tomlin JA. Special facilities in a
generalized mathematical programming sys-
best bounds among themselves. For further
tem for nonconvex problems using ordered
reading refer to Ref. 20. sets of variables. In: Lawrence J, editor.
Proceedings of the 5th Annual Conference
REFERENCES on Operational Research. London: Tavistock
Publications; 1970. pp. 447–457.
1. Land AH, Doig AG. An automated method 14. Mitra G. Investigation of some branch-and-
of solving discrete programming problems. bound strategies for the solution of mixed
Econometrica 1960;28(2):497–520. integer linear programs. Math Program
2. Nemhauser GL, Wolsey LA. Integer and com- 1973;4:155–170.
binatorial optimization. New York: Wiley- 15. Forrest JJH, Hirst JPH, Tomlin JA. Practical
Interscience; 1988. solution of large scale mixed integer program-
3. Wolsey LA. Integer programming. New York: ming problems with UMPIRE. Manage Sci
Wiley; 1998. 1974;20:736–773.
4. Bertier P, Roy B. Procédure de résolution pour 16. Eckstein J. Parallel branch-and-bound algo-
une classe de problèmes pouvant avoir un rithms for general mixed integer pro-
caractère combinatoire. Cah Cent Étud Rech gramming on the CM-5. SIAM J Optim
Oper 1964;6:202–208. 1994;4:794–814.
5. Balas E. A note on the Branch-and-bound 17. Beale EML. Branch-and-bound methods for
Principle. Oper Res 1968;16(2):442–445. mathematical programming systems. In:
6. Dakin RJ. A tree search algorithm fo mixed Hammer PL, Johnson EL, Korte BH, editors.
integer programming. Comput J 1965;8: Discrete optimization II. Amsterdam: North
250–255. Holland Publishing Co.; 1979.
7. Achterberg T, Koch T, Martin A. Branching 18. Caprara A, Fischetti M. Branch-and-cut algo-
rules revisited. Oper Res Lett 2005;33:42–54. rithms. In: Dell’Amico M, Maffioli F, Martello
S, editors. Annotated bibliographies in combi-
8. Linderoth JT, Savelsbergh MWP. A compu-
natorial optimization. New York: Wiley; 1997.
tational study of search strategies for mixed
pp. 45–63.
integer programming. INFORMS J Comput
1999;11:173–187. 19. Barnhart C, Johnson EL, Nemhauser GL,
et al. Branch-and-price: column generation
9. Lodi A. Mixed integer programming compu-
for solving huge integer programs. Oper Res
tation. In: Junger M, Liebling T, Naddef D,
1998;46(3):316–329.
et al., editors. 50 years of integer program-
ming 1958–2008. Berlin: Springer; 2010. pp. 20. Gendron B, Crainic TG. Parallel branch-and-
619–645. bound algorithms: survey and synthesis. Oper
Res 1994;42(6):1042–1066.
10. Benichou M, Gauthier JM, Girodet P, et al.
Experiments in mixed-integer programming.
Math Program 1971;1:76–94.

View publication stats