EMÜ 222

Deterministic Operations Research

2023-2024 Spring

Week 4
Integer Programming – Branch & Bound Method

Winston W.L., Operations Research, Chapter 9

Branch&Bound Method – Knapsack Problems

- A knapsack problem is an IP with a single constraint.

𝑐𝑖 : benefit obtained if item i is chosen

𝑎𝑖 : amount of available resource used by item i
𝑏: amount of available resource

𝑥𝑖 : 1 if item i is chosen / 0 otherwise

Branch&Bound Method – Knapsack Problems

- An easy method to solve the LP relaxation of this problem is by

inspection.

𝑐𝑖 /𝑎𝑖 can be interpreted as benefit item i earns for each unit of

resource used by item i.

Solution:
Sort the items in nonincreasing order of 𝑐𝑖 /𝑎𝑖 .
Include items starting from the first, until the resource is totally
used.
Example.

Item 𝒄𝒊 𝒂𝒊 𝒄𝒊 /𝒂𝒊

1 16 5 3.2
2 22 7 3.14

3 12 4 3
4 8 3 2.67

Order: 1-2-3-4

B&B starts with the LP relaxation of this problem.

𝑥𝑖 ∈ {0,1} is replaced with 0 ≤ 𝑥𝑖 ≤ 1.
Example.
Order: 1-2-3-4

0 ≤ 𝑥𝑗 ≤ 1

SP 1 This is an UB for the

𝑥1 = 1
original problem.
t=1 𝑥2 = 1
𝑥3 = 0.5
𝑧 = 44

𝑥3 = 0 𝑥3 = 1

SP 2 SP 3

Let’s continue with one of

the newly-created SPs, SP3.
 SP 1
Order: 1-2-3-4
𝑥1 = 1
t=1 𝑥2 = 1
𝑥3 = 0.5 This is an UB for the
𝑧 = 44 original problem.

𝑥3 = 1
𝑥3 = 0

t=2
SP 2 SP 3
𝑥1 = 1 Model for SP3
𝑥2 = 5/7 Max z = 16𝑥1 + 22𝑥2 + 8𝑥4
𝑥3 = 1
𝑧 = 43.71 5𝑥1 + 7𝑥2 + 3𝑥4 ≤ 10
0 ≤ 𝑥𝑗 ≤ 1

Let’s continue branching

on SP3.
 SP 1
t=1 𝑧 = 44 Order: 1-2-3-4

𝑥3 = 0 𝑥3 = 1

SP 2 SP 3
𝑥1 = 1
𝑥2 = 5/7 t=2
𝑥3 = 1
𝑧 = 43.71

𝑥2 = 0 𝑥2 = 1

SP 4 SP 5

Let’s continue with one

of the newly-created
SPs, SP4.
 SP 1
t=1 𝑧 = 44 Order: 1-2-3-4

𝑥3 = 0 𝑥3 = 1

SP 3
SP 2 t=2
𝑧 = 43.71

𝑥2 = 0 𝑥2 = 1

SP 4 SP 5
𝑥1 = 1
t=3 𝑥3 = 1
𝑥4 = 1
𝑧 = 36

Model for SP4

Candidate Solution
Max z = 16𝑥1 + 8𝑥4
LB=36 Let’s continue with
5𝑥1 + 3𝑥4 ≤ 10
0 ≤ 𝑥𝑗 ≤ 1 SP5.
 SP 1
t=1 𝑧 = 44 Order: 1-2-3-4

𝑥3 = 0 𝑥3 = 1

SP 3
SP 2 t=2
𝑧 = 43.71
Model for SP5
𝑥2 = 0 𝑥2 = 1 Max z = 16𝑥1 + 8𝑥4
5𝑥1 + 3𝑥4 ≤ 3
SP 4
0 ≤ 𝑥𝑗 ≤ 1
SP 5
𝑥1 = 1 𝑥1 = 3/5
t=3 𝑥3 = 1 𝑥2 = 1 t=4
𝑥4 = 1 𝑥3 = 1
𝑧 = 36 𝑧 = 43.6

Candidate Solution
LB=36

Let’s continue
branching on SP5.
 SP 1
t=1 𝑧 = 44 Order: 1-2-3-4
𝑥3 = 0 𝑥3 = 1

SP 2 SP 3
𝑧 = 43.71 t=2

𝑥2 = 0 𝑥2 = 1
t=4 Model for SP5
SP 4 SP 5 Max z = 16𝑥1 + 8𝑥4
𝑥1 = 1 𝑥1 = 3/5 5𝑥1 + 3𝑥4 ≤ 3
t=3 𝑥3 = 1 𝑥2 = 1
𝑥4 = 1 𝑥3 = 1 0 ≤ 𝑥𝑗 ≤ 1
𝑧 = 36 𝑧 = 43.6

Candidate Solution 𝑥1 = 0 𝑥1 = 1
LB=36
SP 6 SP 7

Let’s continue with

SP6.
 SP 1
t=1 𝑧 = 44 Order: 1-2-3-4
𝑥3 = 0 𝑥3 = 1

SP 2 SP 3
𝑧 = 43.71 t=2

𝑥2 = 0 𝑥2 = 1

SP 4 SP 5
𝑥1 = 1 𝑥1 = 3/5
t=3 t=4
𝑥3 = 1 𝑥2 = 1
Candidate Solution 𝑥4 = 1 𝑥3 = 1
LB=36 𝑧 = 36 𝑧 = 43.6

𝑥1 = 0 𝑥1 = 1

SP 6 SP 7
𝑥2 = 1
Model for SP6 t=5 𝑥3 = 1
𝑥4 = 1
Max z = 8𝑥4 𝑧 = 42
3𝑥4 ≤ 3
0 ≤ 𝑥𝑗 ≤ 1 Let’s
Another feasible solution, with a larger z-value continue
than the LB: Update LB as 42. This is the
candidate solution now.
with SP7.
 SP 1
t=1 𝑧 = 44 Order: 1-2-3-4
𝑥3 = 0 𝑥3 = 1

SP 2 SP 3
𝑧 = 43.71 t=2

𝑥2 = 0 𝑥2 = 1

SP 4 SP 5
𝑥1 = 1 𝑥1 = 3/5
t=3 t=4
𝑥3 = 1 𝑥2 = 1
𝑥4 = 1 𝑥3 = 1
𝑧 = 36 𝑧 = 43.6
𝑥1 = 0
𝑥1 = 1 When 𝑥1 =
𝑥2 = 𝑥3 = 1,
SP 6 t=6 SP 7 the model
𝑥2 = 1 becomes
𝑥3 = 1
t=5 𝑥4 = 1
infeasible.
𝑧 = 42

Candidate Solution
LB=42 Let’s
continue
with SP2.
 SP 1
t=1 𝑧 = 44 Order: 1-2-3-4
𝑥3 = 0 𝑥3 = 1

SP 3 t=2
SP 2 𝑧 = 43.71
𝑥1 = 1 𝑥2 = 1
t=7 𝑥2 = 1 𝑥2 = 0
𝑥4 = 2/3
𝑧 = 43.33 SP 4 SP 5 t=4
t=3 𝑧 = 36 𝑧 = 43.6

𝑥1 = 0 𝑥1 = 1
Model for SP2
Max z = 16𝑥1 + 22𝑥2 + 8𝑥4 SP 7 t=6
5𝑥1 + 7𝑥2 + 3𝑥4 ≤ 14 SP 6
0 ≤ 𝑥𝑗 ≤ 1 𝑥2 = 1
t=5 𝑥3 = 1
𝑥4 = 1
𝑧 = 42

Candidate Solution
LB=42
Let’s
continue
branching
on SP2.
 SP 1
t=1 𝑧 = 44 Order: 1-2-3-4
𝑥3 = 0 𝑥3 = 1

SP 3 t=2
SP 2 𝑧 = 43.71
𝑥1 = 1 𝑥2 = 1
t=7 𝑥2 = 1 𝑥2 = 0
𝑥4 = 2/3
𝑧 = 43.33 SP 4 SP 5 t=4
t=3 𝑧 = 36 𝑧 = 43.6

𝑥4 = 0 𝑥4 = 1 𝑥1 = 0 𝑥1 = 1

SP 8 SP 9 SP 7 t=6
SP 6
𝑥2 = 1
t=5 𝑥3 = 1
𝑥4 = 1
𝑧 = 42

Candidate Solution
LB=42
Let’s
continue
with SP9.
 SP 1
t=1 𝑧 = 44 Order: 1-2-3-4
𝑥3 = 0 𝑥3 = 1

SP 3 t=2
SP 2 𝑧 = 43.71
𝑥1 = 1 𝑥2 = 1
t=7 𝑥2 = 1 𝑥2 = 0
𝑥4 = 2/3
𝑧 = 43.33 SP 4 SP 5 t=4
t=3 𝑧 = 36 𝑧 = 43.6

𝑥4 = 0 𝑥4 = 1 𝑥1 = 0 𝑥1 = 1

SP 8 SP 7 t=6
SP 9 SP 6
𝑥1 = 1 𝑥2 = 1
t=8 𝑥2 = 6/7 t=5 𝑥3 = 1
𝑥4 = 1 𝑥4 = 1
𝑧 = 42.86 𝑧 = 42

Candidate Solution
Model for SP9 LB=42
Max z = 16𝑥1 + 22𝑥2
5𝑥1 + 7𝑥2 ≤ 11
0 ≤ 𝑥𝑗 ≤ 1
 SP 1
t=1 𝑧 = 44 Order: 1-2-3-4
𝑥3 = 0 𝑥3 = 1

SP 3 t=2
SP 2 𝑧 = 43.71
𝑥1 = 1 𝑥2 = 1
t=7 𝑥2 = 1 𝑥2 = 0
𝑥4 = 2/3
𝑧 = 43.33 SP 4 SP 5 t=4
t=3 𝑧 = 36 𝑧 = 43.6

𝑥4 = 0 𝑥4 = 1 𝑥1 = 0 𝑥1 = 1

SP 8 SP 7 t=6
SP 9 SP 6
𝑥1 = 1 𝑥2 = 1
t=8 𝑥2 = 6/7 t=5 𝑥3 = 1
𝑥4 = 1 𝑥4 = 1
𝑧 = 42.86 𝑧 = 42

Candidate Solution
If we continue with SP9, the best solution LB=42
we can find would have a z-value of 42 the Let’s
most. continue
No need to find alternative solutions. with SP8.
Fathom this node, and do not continue.
 SP 1
t=1 𝑧 = 44 Order: 1-2-3-4
𝑥3 = 0 𝑥3 = 1

SP 3 t=2
SP 2 𝑧 = 43.71
𝑥1 = 1 𝑥2 = 1
t=7 𝑥2 = 1 𝑥2 = 0
𝑥4 = 2/3
𝑧 = 43.33 SP 4 SP 5 t=4
t=3 𝑧 = 36 𝑧 = 43.6

𝑥4 = 0 𝑥4 = 1 𝑥1 = 0 𝑥1 = 1
t=9
SP 9 SP 7 t=6
SP 8 t=8 SP 6
𝑥1 = 1 𝑧 = 42.86
𝑥2 = 1
𝑥2 = 1 t=5 𝑥3 = 1
𝑧 = 38
𝑥4 = 1
𝑧 = 42

Candidate Solution
Model for SP8 LB=42
Max z = 16𝑥1 + 22𝑥2 Let’s
5𝑥1 + 7𝑥2 ≤ 14 continue
0 ≤ 𝑥𝑗 ≤ 1 with SP8.
 SP 1
t=1 𝑧 = 44 Order: 1-2-3-4
𝑥3 = 0 𝑥3 = 1

SP 3 t=2
SP 2 𝑧 = 43.71
𝑥1 = 1 𝑥2 = 1
t=7 𝑥2 = 1 𝑥2 = 0
𝑥4 = 2/3
𝑧 = 43.33 SP 4 SP 5 t=4
t=3 𝑧 = 36 𝑧 = 43.6

𝑥4 = 0 𝑥4 = 1 𝑥1 = 0 𝑥1 = 1
t=9
SP 9 SP 7 t=6
SP 8 t=8 SP 6
𝑥1 = 1 𝑧 = 42.86
𝑥2 = 1
𝑥2 = 1 t=5 𝑥3 = 1
𝑧 = 38
𝑥4 = 1
𝑧 = 42

Candidate Solution
z-value of SP8 (38) < LB (42) LB=42
Fathom SP8.
 SP 1
t=1 𝑧 = 44 Order: 1-2-3-4
𝑥3 = 0 𝑥3 = 1

SP 3 t=2
SP 2 𝑧 = 43.71
𝑥1 = 1 𝑥2 = 1
t=7 𝑥2 = 1 𝑥2 = 0
𝑥4 = 2/3
𝑧 = 43.33 SP 4 SP 5 t=4
t=3 𝑧 = 36 𝑧 = 43.6

𝑥4 = 0 𝑥4 = 1 𝑥1 = 0 𝑥1 = 1
t=9
SP 9 SP 7 t=6
SP 8 t=8 SP 6
𝑥1 = 1 𝑧 = 42.86
𝑥2 = 1
𝑥2 = 1 t=5 𝑥3 = 1
𝑧 = 38
𝑥4 = 1
𝑧 = 42

Optimal Solution

SP6 gives the optimal solution.

𝑥2 = 1, 𝑥3 = 1, 𝑥4 = 1, 𝑧 = 42
Branch&Bound Method – Combinatorial Opt. Problems

- A combinatorial optimization problem is any problem that has a

finite number of feasible solutions.

- Examples:
Machine scheduling (10 jobs to be completed by due dates, how
to schedule those jobs)

Traveling salesperson problem (A salesperson must visit 10

cities before returning home, in which sequence to visit the cities to
minimize total traveling distance)
 Branch&Bound Method – Machine Scheduling

4 jobs to be processed on a single machine. In what order should

the jobs be processed to minimize total delay?

Job Processing Time Due Date (di)

(pi) (days) (end of day)
1 6 8
2 4 4
3 5 12
4 8 16

Example. Consider order: 1-3-4-2

Job 1 3 4 2
Completion time 6 11 19 23
Due Date 8 12 16 4
Delay
 Branch&Bound Method – Machine Scheduling

4 jobs to be processed on a single machine. In what order should

the jobs be processed to minimize total delay?

Job Processing Time Due Date (di)

(pi) (days) (end of day)
1 6 8
2 4 4
3 5 12
4 8 16

Example. Consider order: 4-2-3-1

Job 4 2 3 1
Completion time 8 12 17 23
Due Date 16 4 12 8
Delay
Branch&Bound Method – Machine Scheduling Job pi di
1 6 8
2 4 4
3 5 12
1, 𝑖𝑓 𝑗𝑜𝑏 𝑖 𝑖𝑠 𝑡ℎ𝑒 𝑗𝑡ℎ 𝑗𝑜𝑏 𝑡𝑜 𝑏𝑒 𝑝𝑟𝑜𝑐𝑒𝑠𝑠𝑒𝑑
Let 𝑥𝑖𝑗 = ቊ 4 8 16
0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
toplam 23

We will start B&B with deciding on the job that is last processed.
D denotes the total delay.

23

23-8 23-16
Branch&Bound Method – Machine Scheduling Job pi di
1 6 8
önceki delayler + işin yapılmasından kalan zaman - o işin delayi
2 4 4
3 5 12
4 8 16

15 23

? 4

7 + (15-8)

Let’s continue 7 + (15-12)

with the best
Delay of Jobs 1&4 = 7+7=14
Node, Node 7.
Delay of Jobs 2&4 = 7+11=18
7 + (15-4)
Branch&Bound Method – Machine Scheduling Job pi di
1 6 8
önceki delayler + işin yapılmasından kalan zaman - o işin delayi 2 4 4
3 5 12
4 8 16

10 15 23

? 3 4

Delay of Jobs 2&1&3&4

= 0+2+3+7=12
This is an Upper Bound.
Fathom all nodes 10 + (10-4)
10 + (10 - 8)
performing worse than
0 4 10 15 23 0 6 10 15 23
this UB.
2 1 3 4 1 2 3 4
Branch&Bound Method – Machine Scheduling Job pi di
1 6 8
2 4 4
3 5 12
4 8 16

10 15 23

? 3 4

UB =12

0 4 10 15 23 0 6 10 15 23

2 1 3 4 1 2 3 4
Branch&Bound Method – Machine Scheduling Job pi di
1 6 8
2 4 4
3 5 12
4 8 16

18 23

? 3

All are fathomed, since their D > LB.

Branch&Bound Method – Machine Scheduling Job pi di
1 6 8
2 4 4
3 5 12
4 8 16

18 23

? 3

Note that we
employ Optimal solution.
Jumptracking Total delay:12
method. Order: 2-1-3-4
 Branch&Bound Method – Traveling Salesperson Problem

We are at depot (node 0). We need to visit 5 cities and come back to
the depot. Find a tour that crosses each node once and minimizes
the total cost?

2
1
5
3
0
4

Let 𝑐𝑖𝑗 be the cost of going from node i to node j.

Total cost of this tour: 𝑐01 + 𝑐12 + 𝑐25 + 𝑐54 + 𝑐43 + 𝑐30
Traveling Salesperson Problem
Mathematical Model

Arrive to each node

Leave each node

1 2 This is a possible
5
solution for this
3
problem. There are
0
4 subtours!
 Traveling Salesperson Problem
Mathematical Model

Assignment
Problem

2 Eliminates subtours
E.g. 𝑢2 − 𝑢3 + 6𝑥23 ≤ 5 𝑢3 − 𝑢2 + 6𝑥32 ≤ 5 Both 𝑥23 and 𝑥32
If 𝑥23 = 1, 𝑢2 + 1 ≤ 𝑢3 . If 𝑥32 = 1, 𝑢3 + 1 ≤ 𝑢2 . cannot be 1.
3
 Branch&Bound - Traveling Salesperson Problem

𝑐𝑖𝑗 1 2 3 4 5 𝑐𝑖𝑗 1 2 3 4 5
1 - 132 217 164 58 1 M 132 217 164 58
2 132 - 290 201 79 2 132 M 290 201 79
3 217 290 - 113 303 3 217 290 M 113 303
4 164 201 113 - 196 4 164 201 113 M 196
5 58 79 303 196 - 5 58 79 303 196 M

Solve the Assignment

Problem (Use constraints (1)
and (2)). See the Excel
Solution.

1 3

5
4
2
 1 2

4
5
3
 1 2

3
5
4
7
 7
z=668
𝑥13 = 𝑥34 = Alternative
𝑥42 = 𝑥25 = Optimal.
𝑥51 = 1
UB=668
 Heuristics for TSP

Heuristics lead to a good (but not necessarily optimal) solution to a

problem.

We will discuss two heuristics for TSP: nearest-neighbor heuristic

and cheapest-insertion heuristic.

Nearest-Neighbor Heuristic (NNH)

- Begin at any city.
- Then visit the nearest ‘unvisited’ city.

Cheapest-Insertion Heuristic (CIH)

- Begin at any city and find the closest neighbor. Create a subtour
with these cities.
- Break an arc & insert a city in between, which results in the least
cost.
 Nearest-Neighbor Heuristic
𝑐𝑖𝑗 1 2 3 4 5
1 - 132 217 164 58 Nearest-Neighbor Heuristic
2 132 - 290 201 79 - Begin at any city.
-Then visit the nearest ‘unvisited’ city.
3 217 290 - 113 303
4 164 201 113 - 196
5 58 79 303 196 -

Start with node 1:

1 – 5 – 2 – 4 – 3 – 1 : z=668 (Optimal solution is found by coincidence)

Start with node 3:

3 – 4 – 1 – 5 – 2 – 3: z=704

You can start from each city and find a tour, then use the best tour.
 Cheapest-Insertion Heuristic
𝑐𝑖𝑗 1 2 3 4 5 Cheapest-Insertion Heuristic
1 - 132 217 164 58 - Begin at any city and find the closest
neighbor. Create a subtour with these
2 132 - 290 201 79
cities.
3 217 290 - 113 303 -Break an arc & insert a city in between,
4 164 201 113 - 196 which results in the least cost.
5 58 79 303 196 -

Start with node 1. Obtain subtour 1-5. Length: 116

1 5

1 5

Continue with either

the 1st or the 4th
solution.
Cheapest-Insertion Heuristic 2

1 5

3 2

1 5

4 2
Continue with the 2nd solution.
1 5

4
3 2

1 5

Length= Initial
length+added lengths
=116+153+233+166 = 668