Branch and Bound Method

Dinesh Kumar
Branch and Bound
 Branch and Bound is a technique used for solving
Integer Programming (IP) problems.

 Branch and Bound is a Divide and Conquer strategy

in which the original problem is divided into several
sub-problems.
Branch and Bound
 Branch and Bound implicitly enumerates all possible
feasible solutions of IP.

 By solving a single sub-problem many possible

solutions may be eliminated from further
consideration.
Relaxed Problem

 In a relaxed problem, the integer constraints are relaxed.

 An appropriate relaxation should be used for pure

integer, mixed integer and binary integer programming.
Integer Programming Problem

Maximize Z = 40X1 + 90 X 2
Subject to :
9X1 + 7 X 2  56
7 X 1 + 20 X 2  70
X 1 , X 2 = non negative integers
Relaxed Problem (Pure and Mixed Integer)

Maximize Z = 40X1 + 90 X 2
Subject to :
9X1 + 7 X 2  56
Relaxed constraint
7 X1 + 20 X 2  70
X1, X 2 = 0
Relaxed Problem (in Binary IP)
 Original Problem  Relaxed Problem
 Min 3x1 + 4x2 + 9x3  Min 3x1 + 4x2 + 9x3

 S.t.  S.t.
 9x1 + 7x2 + 8x3 >= 15  9x1 + 7x2 + 8x3 >= 15

 x1, x2 and x3 are binary  0 <= x1, x2, x3 <= 1

integers

Relaxed constraint
 Branch and Bound Steps
 Initiation: Identify the appropriate relaxed problem. Solve the
LP relaxation. Stop if the optimal solution is integer.

 Branch: Divide the original problem into sub-problems, by

introducing new constraints for non-integer solution.

 Bound: Conquer the problem by deriving upper and lower

bounds for each sub-problem and fathoming the sub-
problems if possible.

 Test for optimality: If there are no unfathomed sub-

problems left then return the current best integer solution (
candidate solution) as optimal solution.
 Upper and Lower Bounds –
Maximization Objective

 Upper Bound (UB): The optimal integer solution

cannot be more than upper bound value. It is LP
relaxation solution.

 Lower Bound: The optimal integer solution cannot be

lower than the lower bound value. Lower bound is a
candidate solution.

 Candidate Solution: Feasible integer solution.

 Upper and Lower Bounds –
Minimization Objective

 Upper Bound (UB): The optimal integer solution cannot

be more than upper bound value. UB is a candidate
solution.

 Lower Bound: The optimal integer solution cannot be

lower than the LB. LB is a LP relaxation Solution.

 Candidate Solution: Feasible integer solution.

Branching
 Dividing a problem into two sub-problems is called
branching.

 Sub-problems are created by branching on an appropriately

chosen fractional variable Xi.

 Suppose the value of Xi is k + , where k is an integer and 0 <

 < 1, then we create two sub-problems as defined below:

 Sub-problem 1 = Old sub-problem + constraint Xi ≤ k

 Sub-problem 2 = Old sub-problem + constraint Xi ≥ k+1
Branching in Mixed Integer Programming

 When branching a mixed integer programming

problem, only branch on those required to be integer.
 Upper and Lower Bound for a Maximization
Problem

Upper Bound Lower Bound

Solution to the LP Relaxation Problem Feasible Integer Solution

May not be a feasible solution to Integer Candidate Solution to the integer

Programming Problem since the solution programming problem
may not be an integer solution
 Upper and Lower Bound for a Minimization
Problem

Upper Bound Lower Bound

Feasible Integer Solution LP relaxation Solution.

Candidate Solution to the integer May not be a feasible solution to Integer

programming problem Programming Problem
Bounding for Maximization
 Bounding is a fast way of finding the optimal
solution by eliminating sub-problems.

 In a maximization problem If the lower bound

(candidate solution) for a sub-problem 1 is greater
than the upper bound for any other sub-problem (say
sub problem 2), then sub-problem 2 may be safely
discarded from the search.
Bounding for Minimization
 In a minimization problem If the upper bound
(candidate solution) for a sub-problem 1 is less than
the lower bound (Optimal solution to LP Relaxation)
for any other sub-problem (say sub-problem 2), then
sub-problem 2 may be safely discarded from the
search.
Fathoming (Elimination of Sub-problems)
 If it is not necessary to branch a sub-problem, then we say
that the sub-problem is fathomed.

 For a Maximization Problem, a sub-problem can be fathomed

if:
 The sub-problem is infeasible.
 The sub-problem yields a solution in which all variables
have integer solution (candidate solution).
 The sub-problem’s optimal solution is inferior to the
current candidate solution.
Branch and Bound Example

Maximize Z = 40X1 + 90 X 2
Subject to :
9X1 + 7 X 2  56
7 X 1 + 20 X 2  70
X 1 , X 2 = non negative integers
X2

Feasible region to
Problem 1
4

Z = 355.890
X1 = 4.809, X2 = 1.817
2

2 4 6 X1 8 10
X2
Sub-problem 2
Max Z = 40 X1 + 90 X2
6
9X1 + 7X2  56
7X1 + 20 X2  70
X1  4

Z = 349.00
X1 = 4.0, X2 = 2.10
2

2 4 6 X1 8 10
X2

Sub-problem 3
Max Z = 40 X1 + 90 X2
9X1 + 7X2  56
2
Z = 341.390 7X1 + 20 X2  70

X1 = 5, X2 = 1.571 X1  5

2 4 6 X1 8 10
 B and B Tree

Problem 1
Max 40x1 + 90x2
S.t. 9x1 + 7x2 <= 56 Z = 355.890 UB = 355.890
7x1 + 20x2 <= 70
X1 = 4.809
x1, x2 >= 0
X2 = 1.817
X1  4 X1  5

Problem 2 Problem 3
This is an
UB = 349 Z = 349.000 UB since Z = 341.000 UB = 341.890
the
X1 = 4.0 X1 = 5.0
solution is
X2 = 2.100 not integer X2 = 1.571
Max 40x1 + 90x2 Max 40x1 + 90x2
S.t. 9x1 + 7x2 <= 56 S.t. 9x1 + 7x2 <= 56
7x1 + 20x2 <= 70 7x1 + 20x2 <= 70
X1 <= 4 X1 >= 5
x1, x2 >= 0 x1, x2 >= 0

Rounding down the variable values to get a lower bound

(candidate solution) may not be a good strategy
 Problem 1
UB = 355.878
Z = 355.878
X1 = 4.81
X2 = 1.82
X1  4 X1  5

Problem 2 This is an Problem 3

UB since
Z = 349.000 the Z = 341.428
X1 = 4.0 solution is X1 = 5.0
not integer
X2 = 2.100 X2 = 1.57
UB = 349.0 UB = 341.428
 Problem 1
Z = 355.890
UB = 355.890
X1 = 4.809
z dec in branches
X2 = 1.817
X1  4 X1  5

Problem 2 Problem 3
no integer so branch it
Z = 349.000 Z = 341.428 UB = 341.428
UB = 349.0 342 means
X1 = 4.0 X1 = 5.0 dont branch

X2 = 2.100 X2 = 1.571
X2  2 X2  3 X2  1 X2  2

Problem 4 Problem 5 Problem 6 Problem 7

Z = 340.000 Z = 327.1428 Z = 307.778 No feasible
integer Solution
X1 = 4.0 so cs & X1 = 1.428 327<340
so
X1 = 5.444 307<340
so fathom fathom
bl in
fathom it
X2 = 2.0 max
X2 = 3.000 X2 = 1.000
Fathomed: Inferior Fathomed: Inferior Fathomed:
Fathomed: Candidate
solution (LB) Solution (UB < LB) Solution (UB < LB) Infeasible Solution
Branching Strategies
 LIFO (Last-in-first-out) strategy: In this strategy, the most
recently created sub-problem is solved first.
 LIFO leads us down to one side of the tree and quickly
finds a candidate solution.
 LIFO is also called backtracking strategy or Depth First
Search.

 Jumptracking (Breadth First Search): When branching on a

node, jumptracking solves all the problems created by
branching. Then it branches again on the node with best Z
value. It usually creates more subproblems and requires more
computation than backtracking.
Solving Knapsack problems using B & B
Knapsack problem
 Knapsack problem will have only one constraint.

 Mathematically, 0-1 knapsack problem (Max) is given by:

n
Max  ci x i
i =1
subject to
n
 w i xi  c, x i = 0 or 1, i = 1, 2, ..., n
i =1
B and B Knapsack Example
Min Z= 4X1 + 3X 2 + 4X 3 + 9X 4
Subject to :
6X1 + 2X 2 + 5X 3 + 12X 4  10
X i = 0 or 1, i = 1, 2, 3, 4
What is the relaxation Problem?

Min Z= 4X1 + 3X 2 + 4X 3 + 9X 4
Subject to :
6X1 + 2X 2 + 5X 3 + 12X 4  10
0 = X i = 1 For i = 1, 2, 3, 4
 B and B Tree
Problem 1 Min 4x1 + 3x2 + 4x3 + 9x4
s.t. 6x1 + 2x2 + 5x3 + 12x4 >= 10
Z=7
0 <= Xi <= 1, i = 1, 2, 3 and 4
X1 = 1, X2 = 0
X3 = 0, X4 = 1/3
X4 = 0 X4 = 1

Problem 2 Problem 3
Z = 7.2 Z=9
X1 = 1.0, X2 = 0 X1 = X2 = X3 = 0.0
X3 = 0.8 X4 = 1.0

Max 4x1 + 3x2 + 4x3 + 9x4 Max 4x1 + 3x2 + 4x3 + 9x4
s.t. 6x1 + 2x2 + 5x3 + 12x4 >= 10 s.t. 6x1 + 2x2 + 5x3 + 12x4 >= 10
X4 = 0 X4 = 1
0 <= Xi <= 1 ( i = 1, 2 and 3) 0 <= Xi <= 1 ( i = 1, 2 and 3)
 B and B Tree
Problem 1
Z=7
X1 = 1
X4 = 1/3
X4 = 0 X4 = 1

Problem 2 Problem 3
LB = 7.2 UB = 9
Z = 7.2 Z=9
UB = 9.0
X1 = 1.0 X1 = 0.0 cs & ub

X3 = 0.8 X4 = 1.0

Fathomed:
Candidate
solution
 Problem 2
Z = 7.2
X1 = 1.0
X3 = 0.8
X3 = 0 X3 = 1

Problem 4 Problem 5

Fathomed Z = 7.33

Infeasible X1 = 0.833
Solution X3 = 1.00
X1 = 0 X1 = 1

Problem 6 Problem 7
Fathomed Z=8
Infeasible Fathomed
Solution
Candidate
Optimal Solution Solution
Branch and Bound for MIP
B & B for MIP
 To solve MIP using B & B, we branch only on those
variables that are required to be integers.

 For a solution to be a candidate solution, it need to

assign integer values to those variables that are
required to be integers.
MIP example

Max Z = 2x1 + x 2
Subject to
5x1 + 2x 2  8
x1 + x 2  3
x1, x2  0; x1 integer
Branch and Bound for MIP

Sub-problem 1
Z = 11/3 Level 1
X1 = 2/3
X2 = 7/3
X1 <= 0 X1 >= 1

Sub-problem 2 Sub-problem 3
Z=3 Z = 7/2
X1 = 0 X1 = 1
X2 = 3 X2 = 3/2
Candidate solution Candidate solution
Optimal solution best due to max z value
 Which sub-problem to solve first?

Possible natural choices:

➢ Option 1: LIFO, the most recently created sub-problem
➢ Option 2: The sub-problem with the best LP value.
➢ To create an incumbent (candidate solution) with
higher Z* value.
 Summary of branch-and-bound
1) Branching: Among the unfathomed sub-problems, select the
one that was created most recently.
Choose a variable xi which has a non-integer value xi* in
the LP solution of the sub-problem. Create two new sub-
problems by adding the respective constraints xi   xi*
and xi ≥  xi* .
2) Bounding: Solve the new sub-problems, record their LP
solutions. Based on the LP values, update the incumbent, and
the lower and upper bounds if necessary.
3) Fathoming: For each new sub-problem, apply the three
fathoming tests. Discard the sub-problems that are fathomed.
4) Optimality test: If there are no unfathomed sub-problems left
then return the current incumbent as optimal solution
(if there is no incumbent then IP is infeasible.)
Otherwise, perform another iteration.
 Further reading
 A H Land and A G Doig, “An automatic method of
solving discrete programming problems”,
Econometrica, Vol 28, 497-520, 1960.

 Wayne Winston, “Operations Research-Applications

and Algorithms”, Thomson Publishing.

 Hillier and Liberman, “Introduction to Operations

Research”, Tata McGraw Hill.