FUNDAMENTALS OF OPTIMIZATION

Heuristic methods

3
CONTENT

• Greedy algorithm
• Local search

4
 Overview
• Exact methods (Constraint Programming, Branch and Cut, ..) cannot handle large-scale combinatorial
optimization problems
• In practice, high quality solutions found in a reasonable computation time are required

5
Overview of greedy methods

• S: a solution is represented by a set of components Greedy() {

S = {};
• C: set of candidates of components to be added to the
solution while C   and
not solution(S){
• select(C): select the most promising component among
candidates x = select(C);
C = C \ {x};
• solution(S): return true if S is a solution to the original
if feasible(S  {x}) {
problem
S = S  {x};
• feasible(S): return true if S does not violate any }
constraints
}
return S;
}

6
 TSP
• Given n points 1, 2, …, n in which d(i,j) is the distance from point i to point j. Find the shortest closed
tour starting from 1 visiting other points and terminating at 1 such that the total travel distance is
minimal

7
 TSP
• Greedy idea
greedyTSP( distance matrix d(1..n,1..n)){
• The tour is initialized by point 1
S = [1]; last = 1;
• At each step
C = {2,3,…, n};
• Select the nearest point to the last point of the
tour under construction and add this point to the while(C not empty){
end of the tour j = argMiniC{ d(last, i)};
S = S ::j;
last = j;
C = C \ {j};
}
S = S::1
return S;
}

8
 Multi Knapsack Problem

• Given unlimited number of bins having capacity Q and n items 1, 2, …, n in which the weight of item i
is w(i). How to put these n items into bins such that the total weight of items put into each bin
cannot exceed Q and the number of bins used is minimal
• Possible strategies
• Multi-Stage selection
• Sort items in some order
• Consider each item in the sorted list
• For each item, select an appropriate bin for the current item
• Single-Stage selection: consider all pairs of (bin, item) for the selection
• Each step, consider all pairs of bin, item for the selection

9
 Multi Knapsack Problem (Multi-Stage selection)
greedyMNS(){ selectFirstFitBin(bins,load,w){
bins = []; // list of bins being used for b in bins do {
items = sort items in some order; if load[b] + w <= Q then
for i in items do { return b;
b = select(bins,load,w(i)); }
if b = NULL then { return NULL;
b = length(bins) + 1; }
bins = bins::b;
selectBestFitBin(bins,load,w){
load[b] = 0;
R = ; sel_b = NULL
}
for b in bins do {
binOfItem[i] = b;
if load[b] + w <= Q and (Q-load[b]-w) then
load[b] = load[b] + w[i];
R = Q – load[b] - w; sel_b = b;
}
}
return binOfItem;
return sel_b;
}
}

10
Multi Knapsack Problem (Single-Stage selection)

selectBinAndItem(bins, cand){ greedyMNS(){

R = ; sel_bin = NULL; sel_item = 0; bins = []; cand = {1, 2, . . ., n};
for i in cand do{ while (cand not empty){
for b in bins do{ (sel_bin, sel_item) = selectBinAndItem(bins, cand);
if load[b] + w(i) ≤ Q if (sel_bin = NULL) then {
and (Q – load[b] – w(i) < R then { b = bins.length; bins = bins::<b>;
R = Q – load[b] – w(i); sel_bin = b; sel_item = argmaxicand(w(i));
sel_bin = b; sel_item = i; }
} cand.remove(sel_item);
} binOfItem[sel_item] = sel_bin;
} load[sel_bin] = load[sel_bin] + w(sel_item);
return (sel_bin, sel_item); }
} return binOfItem;
}

11
Exercise - BCA

• At the beginning of the semester, the head of a computer science department have to assign
courses to teachers in a balanced way.
• The department has m teachers T={1, 2, ..., m} and n courses C={1, 2,..., n}.
• Each course c  C has a duration hc.
• Each teacher t  T has a preference list which is a list of courses he/she can teach depending on
his/her specialization.
• We know a list of pairs of conflicting two courses that cannot be assigned to the same teacher as
these courses have been already scheduled in the same slot of the timetable. This conflict
information is represented by a conflict matrix A in which A(i,j)=1 indicates that course i and j are
conflict.
• The load of a teacher is the total duration of courses assigned to her/him.
• How to assign n courses to m teachers such that each course assigned to a teacher is in his/her
preference list, no two conflicting courses are assigned to the same teacher, and the maximal load
for all teachers is minimal

12
CONTENT

• Greedy algorithm
• Local search

13
 Local search
• Method for finding high-quality solutions to optimization function LocalSearch(N, f){
problems
// N: neighborhood
• Key idea // f: quality function
• Generate an initial solution s = Generate an initial solution;
• Iteratively move from a current solution to one of its s* = s;
neighbors
while termination not reach do {
s = Select(N(s), s); //neighbor selection
if s = NULL then break;
if s is better than s* then
s* = s;
}
... return s*;
}

14
 Local search
• Example of n-queen: How to place n queens on a chess board
such that no two queens attack each other

15
 Local search
• Example of n-queen: How to place n queens on a chess board
such that no two queens attack each other

16
 Local search
• Quality function f(s)
• Measure the quality of a solution s
• Depend on specific problems
• For pure optimization problems
• Quality function can be the objective function of the problem
• For satisfaction problems (n-queen, sudoku, etc.)
• Quality function can be the number of constraint violations
• For constrained optimization problems
• Quality function can be the combination of objective function (optimality) and the constraint
violations (satisfiability)
• Without loss of generality, suppose that f(s) is need to be maximized

17
“Landscape” of search

Hill Climbing gets stuck in local minima depending on?

18
Neighbor selections
function LocalSearch(N, f){ function S-Improvement(N, f, s){
// N: neighborhood s’ = argMinxNf(x);
// f: quality function if f(s’) > f(s) then return s’;
s = Generate an initial solution; else return NULL;
s* = s; }
function S-RandomImprovement(N, f, s) {
while termination not reach do {
select n N with probability 1/|N|;
s = Select(N(s), f, s); //neighbor selection
if f(n) > f(s) then return n;
if s = NULL then break;
else return s;
if f(s) > f(s*) then
}
s* = s;
function S-Metropolis(N, f, s, t){
}
select n N with probability 1/|N|;
return s*;
if f(n) > f(s) then return n;
} 𝑓 𝑠 −𝑓(𝑛)
else with probability 𝑒 𝑡 ;
else return s;
}

19
THANK YOU !

20