Optimization

Techniques
for AI
Lecture
CS/AI 15
2101
Branch-and-bound strategy
 It is a tree search.
 It gradually (as it traverses the tree) assigns
values to variables
 Root node – no variable is assigned

 Leaf node- all variables are assigned

 Intermediate node- some variables are assigned

values and some are not.

 A mechanism to generate branches when
searching the solution space
 A mechanism to generate a bound so that many
branches can be terminated

2
Branch & Bound
Technique for
Travelling Salesman
Problem
ILP formulation of TSP
(complete)
Minimize z =
Subject to

,
for all subsets
A Simple Relaxation Scheme
for TSP

Travelling Salesman Problem Relaxation

Relaxing Subtour Prevention

Assignment Problem
Hungarian
method
Assignment Problem
Hungarian Method
LP formulation of
Assignment Problem
Linear Programming Formulation

Min
s.t. for each resource i
for each job j
= 0 or 1 for all i and j.
Hungarian Method
Step 1: For each row, subtract the minimum
number in that row from all numbers in that row.
Step 2: For each column, subtract the minimum
number in that column from all numbers in that
column.
Step 3: Draw the minimum number of lines to
cover all zeroes.
If this number = n, STOP — an assignment
can be made.
Step 4: Else, determine the minimum uncovered
number (call it d).
Subtract d from uncovered numbers.
Add d to numbers covered by two lines.
Numbers covered by one line remain the
same.
GO TO STEP 3.
Drawing Minimum No of
lines
1. Mark all rows with no assignments (with a “‧”).
2. For each row just marked, mark each column that has
a 0 in that row (with a “‧”).
3. For each column just marked, mark each row that has
an assignment in that column (with a “‧”).
4. Repeat steps 2 and 3 until no more marks can be
made.
5. Draw lines through unmarked rows and marked
columns.
Making Assignment

1. For each row, if only one 0 remains in the row, make

that assignment and eliminate the row and column from
consideration in the steps below.
2. For each column, if only one 0 remains, make that
assignment and eliminate that row and column from
consideration.
3. Repeat steps 1 and 2 until no more assignments can be
made. (If 0’s remain, this means that there are at least
two 0’s in each remaining row and column. Make an
arbitrary assignment to one of these 0’s and repeat steps
1 and 2.)
Example- Hungarian method
Step1-2
Step 1: For each row, subtract the minimum number in that
row from all numbers in that row.
Step 2: For each column, subtract the minimum number in
that column from all numbers in that column.

5 8 10 7 5 0 3 5 2 0 3 4 2
3 5 7 3 3 0 2 4 0 0 2 3 0
Subtract Subtract
4 5 3 2 2 2 3 1 0 2 3 0 0
9 2 6 8 2 7 0 4 6 7 0 3 6
0 0 1 0 0 3 4 2
0 2 3 0
2 3 0 0
7 0 3 6
Example- Example 2
Step 1: For each row, subtract the minimum number in that
row from all numbers in that row.
Step 2: For each column, subtract the minimum number in
that column from all numbers in that column.

1 4 6 3 1 0 3 5 2 0 3 2 2
9 7 10 9 7 2 0 3 2 2 0 0 2
Subtract Subtract
4 5 11 7 4 0 1 7 3 0 1 4 3
8 7 8 5 5 3 2 3 0 3 2 0 0
0 0 3 0
Hungarian Method Example
2A feasible assignment is not possible at this


moment.
 To draw a minimum number of lines through
some of the rows and columns, Such that all
Theare
zero values nextcrossed
step is to select the smallest
out.
uncrossed out element which is
subtracted from every uncrossed
out element and added to every
0 3 2 2 element
0 2 1 at1 the0 intersection
2 1 1 1 of4two
6 3

2 0 0 2
lines
3 0 0 2 3 0 0 2 9 7 10 9

0 1 4 3 0 0 3 2 0 0 3 2 4 5 11 7

3 2 0 0 3 2 0 0 3 2 0 0 8 7 8 5
Example 3
Example 4
Job 1 Job 2 Job 3 Job 4
Machine 14 5 8 7
1
Machine 2 12 6 5
2
Machine 7 8 3 9
3
Job 1 Job 2 Job 3 Job 4
Machine 2 4 6 10
Machine
4 9 0 3 2
1
Machine 0 10 4 3
2
Machine 4 5 0 6
3 Job 1 Job 2 Job 3 Job 4
Machine
Machine 09 20 43 82
41
Machine 0 10 4 3
2
Machine 4 5 0 6
3
Machine 0 2 4 8
Example 4: We need 3<n lines, so
continue to Step 3
Job 1 Job 2 Job 3 Job 4
Machine 9 0 3 0
1
Machine 0 10 4 1
2
Machine 4 5 0 4
3
Machine 0 2 4 6 Job 1 Job 2 Job 3 Job 4
d= 1: 4Subtract d from Machine 10 0 3 0
uncovered element, add d 1

to intersection elements Machine 0 9 3 0

2
Machine 5 5 0 4
3
Machine 0 1 3 5
4
 Example 4
Job 1 Job 2 Job 3 Job 4
Machine 9 0 3 0
1
Machine 0 10 4 1
2 Job 1 Job 2 Job 3 Job 4
Machine 4 5 0 4 10 0 3 0
3 0 9 3 0
Machine 0 2 4 6 5 5 0 4
4 Job 1 Job 2 Job 3 Job 4
0 1 3 5
Machine 10 0 3 0
1
Machine 0 9 3 0
2 Need n lines, so we have
Machine 5 5 0 4
the optimal assignment
3 and we stop
Machine 0 1 3 5
4
Example 4: Final Solution

Job 1 Job 2 Job 3 Job 4

10 0 3 0
0 9 3 0
5 5 0 4
0 1 3 5

Optimal assignment
x12 1, x33 1, x41 1, x24 1
 TSP is a Minimization
 Problem
Maintain UB: the cost of the best TSP solution so
far identified;
 : the value of the objective function of the
modified AP at node h of the search tree;
 : the set of included arcs (variables fixed at 1) at
node h of the search tree;
 : the set of excluded arcs ( variables fixed at 0)
at node h of the search tree.
 Priority Queue is preferred
 Take a candidate If PAP has no feasible
problem PTSP from L solution, this node not
with smallest . PLP is to be expanded.
obtained by AP-
relaxation. It is If has no subtours,
modified AP, with all update UB and
diagonal are blocked, incumbent. This node
and some assignments not to be expanded.
are done. If contains subtours
and is not < UB, this
node is fathomed.
This node not to be
Push P1TSP , P2(0, 1), , …, expanded.
Pr(0, 1), If contains subtours
and is < UB then
Get estimate of the generate two
relaxed problems subproblems. P1TSP ,
P1TSP , P2TSP , …, PrTSP
P2TSP , …, PrTSP
An Example- 6 citiesTSP
Cost Matrix
 27 43 16 30 26
7  16 1 30 25
20 13  35 5 0
21 16 25  18 18
12 46 27 48  5
23 5 5 9 5 
 TSP- relaxation Row Min
 27 43 16 30 26 16  11 27 0 14 10
7  16 1 30 25 1
6  15 0 29 24
20 13  35 5 0 0
20 13  35 5 0
21 16 25  18 18 16
5 0 9  2 2
12 46 27 48  5 5
7 39 22 43  0
23 5 5 9 5  5
18 0 0 4 0 

5 0 0 0 0 0

 11 27 0 14 10  10 26 0 13 10  6 22 0 9 10
1  15 0 29 24 0  14 0 28 24 0  10 1 24 24
15 13  35 5 0 14 12  35 4 0 13 8  35 0 0
0 0 9  2 2 0 0 9  2 3 4 0 9  2 7
2 39 22 43  0 1 38 21 43  0 0 34 17 43  0
13 0 0 4 0  13 0 0 5 0  17 0 0 9 0 

: {1 to 4, 2 to 1, 3 to 5, 4 to 2, 5 to 6, 6 to 3 }
=16+7+5+16+5+5= 54
 Is a feasible solution of

: {1 to 4, 2 to 1, 3 to 5, 4 to 2, 5 to 6, 6 to 3 }
=16+7+5+16+5+5= 54

2 3

1 6

4 5

elaxed Assignment Problem’s optimal solution contains subt

 Illustration of branching  27 43 16 30 26
7  16 1 30 25
20 13  35 5 0
 6 22  9 10 
21 16 25 18 18
0  10 1 24 24 
12 46 27 48 5
13 8  35 0 0 
23 5 5 9 5
P1 4 0 9  2 7
64
0 34 17 43  0
𝟎
=

17 0 0 9 0 
𝟏𝟒
𝒙

z at this node 54+10= 64

P
54
 6 22 0 9 10
0  10 1 24 24
,

13 8  35 0 0
P2
55  0 9  2 7
0 34 17 43  0
17 0 0 9 0 

Z at this node 54+1= 55