COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Federico Della Croce
Federico Della Croce Heuristics 1 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Solving Combinatorial Optimization problems (COPs)
Solution approaches for COPs, can be split into different groups but an
immediate, though rough, classification for these methods, divides them
into two main categories:
• exact algorithms
• heuristic algorithms.
Federico Della Croce Heuristics 2 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Exact Methods - Pros and Cons
Pros.
• a proven optimal (or no feasible) solution can be obtained for every
finite size instance
• information on upper/lower bounds on the optimal solution can be
drawn at any moment
• general modeling framework and general purpose open-source and
commercial solvers
Cons.
• the size of the instances that are practically solvable is limited
• the best exact algorithms are problem specific and often require an
important software development time
Federico Della Croce Heuristics 3 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristic Methods - Pros and Cons
Pros.
• heuristic algorithm of reasonable performance can be typically
developed rather quickly
• can examine a huge number of possible solutions in short
computational time
• many solution frameworks methods
Cons.
• cannot prove optimality
• typically do not provide theoretical bounds on the quality of the
solutions they return
• no general problem modeling framework nor general solvers
Federico Della Croce Heuristics 4 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
•Constructive polynomial time Heuristics
– Myopic (Greedy) algorithms;
– Simplifying the steps of an exact solution procedure
(ex. Beam Search vs Branch and Bound).
•Local Search
– Classic Local Search
• Steepest Descent
• First Improvement
Federico Della Croce Heuristics 5 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
•Improved heuristics
– Metaheuristics
• Iterated Local Search
• Variable neighborhood search
• Tabu Search, Grasp, Simulated Annealing
• Genetic algorithms
• …
– Matheuristics
Federico Della Croce Heuristics 6 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Myopic rules (greedy algorithms)
•The solution is generated in a constructive way
•At each iteration a partial solution is available
and this solution is extended by selecting one
of the possible choices available at that point.
Federico Della Croce Heuristics 7 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Myopic rules (greedy algorithms)
•Example for TSP
NN (Nearest Neighbor) rule:
select the next town to be visited as the
nearest to the current town.
Federico Della Croce Heuristics 8 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Ex. for the uncapacitated p-median problem
•The problem: locate p facilities (medians) on a
network so as to minimize the sum of all distances
from each vertex to each nearest facility.
•Hakimi theorem: the medians belong to the
vertices set.
•Can be seen as a pure mathematical problem:
given a nxm matrix D, select p columns of D in
order that the sum of minimum coefficients in each
line within these columns be the smallest possible.
•Computational complexity: NP-hard for general
values of p.
Federico Della Croce Heuristics 9 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
ILP model for the p-median problem
xij = 0/1 variable = 1 if vertex i
is allocated to vertex j.
dij = cost for allocating i to j.
V = Vertex set.
Federico Della Croce Heuristics 10 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Example for the p-median problem
P=2
Medians = {2,3} Total cost = 8
1 2 3 4 5
1 0 2 5 7 9
2 2 0 4 7 3
3 5 4 0 3 6
4 7 7 3 0 4
5 9 3 6 4 0
Federico Della Croce Heuristics 11 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Myopic rules (greedy algorithms)
•Ex. for the p-median problem
– (1-median) select as first median the nearest node
on average to all the other nodes;
– (adding medians) add a new median to the current
solution in such a way that the solution is improved
as much as possible.At each iteration a partial
solution is available and this solution is extended at
that point.
Federico Della Croce Heuristics 12 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Improving myopic rules
•“Randomization”:
– make randomly one of the best “k” choices;
– make a random choice among the choices that are at
most p% worse than the best choice (according to the
myopic rule).
•Whenever a random choice is foreseen:
Multi-start
Federico Della Croce Heuristics 13 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
BEAM SEARCH
(Truncated Branch and Bound)
•Search tree (as for Branch and Bound)
•Breadth First strategy
•At each level of the search tree keep only the
best k nodes (k = Beam width).
•How to evaluate the nodes:
– Greedy algorithms;
– Optimal solution of a relaxed problem;
– Weighted sum of 1) and 2).
Federico Della Croce Heuristics 14 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
COMPLEXITY OF BEAM SEARCH
•D = # search tree levels
•E = # children of a generic node
•k = Beam width
Hence
•kE = # nodes per level
•J = O(DkE) = total number of opened nodes
(to be multiplied by the complexity for
evaluating a single node)
Federico Della Croce Heuristics 15 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
LOCAL (NEIGHBORHOOD) SEARCH
•In general, a local search consists in searching for
a solution xj belonging to the neighborhood N(xi) of
xi, according to some predefined rules.
•Neighborhood: to each x& is associated N(x)&.
•Example: & = set of binary vectors
xk = current solution with xk &.
•N(xk) = set of solutions x& that can be obtained
from xk by swapping a single component from 0 to 1
or viceversa.
Federico Della Croce Heuristics 16 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
LOCAL (NEIGHBORHOOD) SEARCH
•We assume that xk N(xk).
•Given an initial arbitrary solution x1X, at each
step k we search for a new solution xk+1N(xk).
•The local search proceeds by iterations
searching at each iteration to find a new
solution belonging to the neighborhood of the
current solution xk and that is better than xk.
Federico Della Croce Heuristics 17 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Main steps of a local search
1.Initialization: selection of an initial (feasible)
solution x1 as current solution and computation
of its objective function value.
2.Neighborhood generation: construction of a
(totally feasible) neighborhood N(xi) of the
current solution xi and selection of a new
solution xi+1 N(xi).
Federico Della Croce Heuristics 18 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Main steps of a local search
3. Acceptance test: check whether solution xi+1
can be accepted; in the negative case the
current solution remains xi; in the positive
case the current solution must be updated: xi+1
becomes the new current solution.
4. Stopping test: if the test is positive, STOP, the
best solution found represents the output of
the procedure, else GO TO step 2.
Federico Della Croce Heuristics 19 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
LOCAL SEARCH: INITIALIZATION
•The initial solution is typically computed by
means of a polynomial time (greedy) heuristic
procedure.
If the problem is so “hard” that no simple
procedures are available to get a feasible
solution
– THEN
•Local search applied to the feasibility phase:
– the objective function to be minimized becomes the
sum of infeasibilities.
Federico Della Croce Heuristics 20 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
LOCAL SEARCH:
NEIGHBORHOOD GENERATION
•Problem representation.
•Definition of the neighborhood and
computation of its size.
•Complexity of computing a neighbor solution.
Federico Della Croce Heuristics 21 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
LOCAL SEARCH:
NEIGHBORHOOD GENERATION
(SYMMETRIC TSP)
•Solution representation: string
2 3
1 4 1 2 3 4 5 6
6 5
Federico Della Croce Heuristics 22 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
LOCAL SEARCH:
NEIGHBORHOOD GENERATION
(SYMMETRIC TSP)
•Neighborhood: swap two arcs (the solution
remains feasible) 2 3
1 4
6 5
Federico Della Croce Heuristics 23 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
LOCAL SEARCH:
NEIGHBORHOOD GENERATION
(SYMMETRIC TSP)
•The arcs swap corresponds to
|1|2|3|4|5|6| Æ |1|2|5|4|3|6|
Federico Della Croce Heuristics 24 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
LOCAL SEARCH:
NEIGHBORHOOD GENERATION
(SYMMETRIC TSP)
•The arcs swap corresponds to
|1|2|3|4|5|6| Æ |1|2|5|4|3|6|
•a pairwise interchange between two elements of the
string while applying a twist on the subsequence which
is between the two elements.
Federico Della Croce Heuristics 25 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Complexity of computing a neighbor
solution
•Constant # of sums for any size of the problem
•f(xi+1) = f(xi) + c3,6 + c2,5 – c2,3 – c5,6
where ci,j = distance between cities i and j
Federico Della Croce Heuristics 26 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
LOCAL SEARCH:
NEIGHBORHOOD GENERATION
(0/1 KNAPSACK)
n
•Solution representation:
min ¦ c j x j
j 1
¦w
j 1
j xj dW
xij^0,1`
x1 x2 x3 x4 x5 x6 x7 x8
0 0 1 0 1 1 0 1
Federico Della Croce Heuristics 27 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
LOCAL SEARCH:
NEIGHBORHOOD GENERATION
(0/1 KNAPSACK)
•Neighborhood:
– swap the value of a variable;
– Swap xi with xj if xi + xj = 1;
– Non feasible solutions may be generated.
– Neighborhood size: O(n2).
– Complexity of computing a neighbor solution :
constant.
Federico Della Croce Heuristics 28 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
LOCAL SEARCH:
NEIGHBORHOOD GENERATION
(0/1 KNAPSACK)
•Another representation for the 0/1 KNAPSACK
Given any sequence of the xi variables, the variables are
set to 1 whenever it is possible.
Federico Della Croce Heuristics 29 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
LOCAL SEARCH:
NEIGHBORHOOD GENERATION
(0/1 KNAPSACK)
•For the sequence | x1 | x2 | x3 | x4 | x5 | x6 | x7 | x8 |
we get:
x1 = 1 IF w1 d W ELSE x1 = 0
x2 = 1 IF w1 x1 + w2 d W ELSE x2 = 0
x3 = 1 IF w1 x1 + ... + wi-1 xi-1 + wi d W ELSE x3 = 0
etc.
Federico Della Croce Heuristics 30 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
LOCAL SEARCH:
NEIGHBORHOOD GENERATION
(0/1 KNAPSACK)
•New neighborhood:
– Swap a pair of variables, e.g. x2 and x7, inducing the
new sequence | x1 | x7 | x3 | x4 | x5 | x6 | x2 | x8 |.
We get:
x1 = 1 IF w1 d W ELSE x1 = 0
x7 = 1 IF w1 x1 + w7 d W ELSE x7 = 0
x3 = 1 IF w1 x1 + w7 x7 + w3 d W ELSE x3 = 0
etc.
Federico Della Croce Heuristics 31 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
LOCAL SEARCH:
NEIGHBORHOOD GENERATION
(0/1 KNAPSACK)
•New neighborhood size = O(n2).
•Complexity of computing a neighbor solution: O(n).
•Why the solutions space moves from 2n to n! ?
Federico Della Croce Heuristics 32 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
LOCAL SEARCH:
NEIGHBORHOOD GENERATION
(K-TSP)
• K salesmen partition the cities in K subtours in
order to minimize the distance.
• First representation: two-levels decomposition.
– Assign the cities to the salesmen;
– Solve K TSP problems.
Federico Della Croce Heuristics 33 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
LOCAL SEARCH:
NEIGHBORHOOD GENERATION
(K-TSP – first representation)
•Two-level local search:
– Level 1: assignment of the cities to the salesmen.
– Level 2: solution of K TSP deduced from the assignment
at level 1.
Federico Della Croce Heuristics 34 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
LOCAL SEARCH:
NEIGHBORHOOD GENERATION
(K-TSP – second representation)
•Another representation:
– introduce k fictituous cities all with the same location (the
departure city of the salesmen) and solve a unique aggregated
TSP
Federico Della Croce Heuristics 35 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
LOCAL SEARCH:
NEIGHBORHOOD GENERATION
(K-TSP – second representation)
•Example with 3 salesmen and 8 cities.
•Introduce 3 fictituous cities A, B, e C.
•The sequence | A|1|2|3|B|4|5|6|C|7|8| indicates that
salesman A will visit cities 1,2,3 in that order and then get
back to his home city; same occurs to salesman B with cities
4,5,6 and salesman C with cities 7,8.
Federico Della Croce Heuristics 36 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
LOCAL SEARCH:
NEIGHBORHOOD GENERATION
(Graph Coloring)
•A non-oriented graph G(N,E) is said r-cromatic if it is
possible to color its vertices with r colors in such a way
that no adjacent vertices have the same color.
•The smallest value of r such that G(N,E) is r-cromatic is
defined as the cromatic number J(G).
•The Graph Coloring problem consists in searching for
J(G).
Federico Della Croce Heuristics 37 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
LOCAL SEARCH:
NEIGHBORHOOD GENERATION
(Graph Coloring)
•Solution representation:
– The Graph Coloring problem can be tackled by
iterating on the number J of available colors and
searching for a feasible solution for each value of J.
– For any given value of J
• search for a J-partition of G minimizing the total
number of crossing edges (edges belonging to
the same partition);
• Every J-partition with no crossing edges is a J-
coloring of G;
Federico Della Croce Heuristics 38 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
LOCAL SEARCH:
NEIGHBORHOOD GENERATION
(Graph Coloring)
•The neighborhood N(x) of this J-partitioning problem is
given by all J-partitions reachable from the J-partition x
moving a bad vertex (a vertex which is adjacent to at least
another vertex of the same partition) from a partition to
another.
Federico Della Croce Heuristics 39 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
CLASSIC LOCAL SEARCH:
STEEPEST DESCENT STRATEGY
1.Initialization: selection of an initial solution x1 &
– x’ = current solution = x1
2.Generation of the Neighborhood N(x’) of the current
solution x’ and evaluation of each neighbor. Let x* N(x’)
the best neighbor solution.
3.Acceptance Test:
– IF x* improves upon the current solution x’ THEN
– set x’= x* and GO TO 2, ELSE GO TO 4.
4.Stop.
Federico Della Croce Heuristics 40 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
CLASSIC LOCAL SEARCH:
FIRST IMPROVEMENT STRATEGY
1.Initialization: selection of an initial solution x1 &
– x’ = current solution = x1
2.Generation of the Neighborhood N(x’) of the current
solution: N(x’) = ^ x2, x3, ..., xn`.
3.Neighborhood evaluation:
– FOR i = 2, ..., n DO:
– IF xi improves upon the current solution x’ THEN :
– set x’= xi and GO TO 2.
4.Stopping test: IF no improving solution exists in the
neighborhood THEN STOP.
Federico Della Croce Heuristics 41 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Metaheuristics
Iterated Local Search
• Iteratively applies neighborhood search steps in order to reach
different local minima.
Each time a new local minimum is reached, a perturbation is applied
to the corresponding solution in order to get a new solution to
which re-apply the neighborhood search.
• The perturbation is typically obtained by randomly generating a
neighbor of the current local minimum within a much wider
neighborhood than the one considered in the local search.
• An acceptance test determines whether to accept the new local
minimum or get back to the old one.
• A stopping test determines the end of the procedure.
Federico Della Croce Heuristics 42 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Main steps of the Iterated Local Search
1. Initialization:
– Select an initial solution x1 &;
– Apply a local search procedure to the initial solution x1
generating a local minimum that represents the current
solution.
2. Repeat:
– Perturbation of the current solution and corresponding
generation of a new solution x’;
– Application to solution x’ of a local search procedure yelding
a new local minimum x’’;
– Acceptance test: if the answer is positive, x’’ becomes the
new current solution;
Until Stopping test is satisfied.
Federico Della Croce Heuristics 43 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Metaheuristics
Variable Neighborhood Search
• Can be seen as a special kind of Iterated Local
Search.
• Foresees the use a series of neighborhood
structures handled in a hierarchical way in the
perturbation phase plus a (typically) unique
neighborhood structure for the local search phase.
• The neighborhood structures of the perturbation
phase are typically sorted by increasing
neighborhood size.
Federico Della Croce Heuristics 44 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Metaheuristics
Variable Neighborhood Search
• The local search step is typically of a first
improvement type.
• A new local minimum is accepted only if it improves
upon the current solution.
• Each time there is no improvement, the
neighborhood considered in the perturbation phase
is modified (getting larger).
• The procedure stops once all neighborhoods of the
perturbation phase have been considered.
Federico Della Croce Heuristics 45 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Main steps of the
Variable Neighborhood Search
1. Initialization:
– Select an initial solution x1 &;
– Select a set of neighborhood structures Nk with
k=1,…, kmax to be used in the procedure;
– k = 1; x* = current solution = x1.
Federico Della Croce Heuristics 46 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Main steps of the
Variable Neighborhood Search
2. Repeat:
– Generate randomly a solution x’ belonging to the
neighborhood Nk of the current solution x*;
– Apply a local search procedure to solution x’
yelding a new local minimum x’’;
– Acceptance test: IF x’’ improves upon x*, THEN
x*=x’’; k = 1; LOOP;
– ELSE k = k +1; LOOP;
Until k = kmax OR stopping test is satisfied.
Federico Della Croce Heuristics 47 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Metaheuristics
Variable Neighborhood Descent
• Represents an extension/generalization of Variable
Neighborhood Search.
• Foresees the use a series of neighborhood
structures handled in a hierarchical way in the
perturbation phase: these structures are however
used directly in the local search phase.
Federico Della Croce Heuristics 48 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Main steps of the
Variable Neighborhood Descent
1. Initialization:
– Select an initial solution x1 ∈Χ;
– Select a set of neighborhood structures Nk with k=1,…,kmax to be used
in the procedure;
– k = 1; x* = current solution = x1.
2. Repeat:
– Generate a random solution x’ belonging to the neighborhood Nk of x*.
– Apply a local search procedure to solution x’ yelding a new local
minimum x’’;
– IF x’’ improves upon x*, THEN x* = x’’; k = 1; LOOP;
ELSE k = k+1; LOOP;
Until k = kmax OR stopping test satisfied.
Federico Della Croce Heuristics 49 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Metaheuristics
Greedy Randomized Adaptive Search Procedure
• It represents an evolution of the classic local search with multistart;
• At each iteration, the local search is preceded by a greedy
constructive procedure where the greedy choices are executed with
some randomness (greedy randomized type procedure).
• “Greedy randomized”: at each step of the greedy procedure the best k
choices (according to the greedy procedure) are selected and one of
them is randomly applied.
• Only two parameters are required:
– The value of k (how many choices at each step?)
– # multistart iterations
Federico Della Croce Heuristics 50 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Main steps of the
Greedy Randomized Adaptive Search Procedure
1. x* = current solution = +infty;
2. FOR k = 1,…,kmax DO
• Generate a solution xk by means of a greedy randomized
constructive procedure;
• Apply a local search procedure to the initial solution xk
yelding solution xk’;
• If xk’ is better than x*, then x* = xk’.
Federico Della Croce Heuristics 51 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Metaheuristics
Simulated Annealing
• First Improvement like Local Search: tries to avoid local minima.
• Analogy with the physical process of Annealing, namely the slow
cooling of a metal to reach a solid state with minimum energy.
• Same principle in Simulated Annealing:
• Initially both better and worse solutions (w.r.t. the current solution)
are accepted (so-called phase of “high temperature”) and this allows a
good exploration of the solutions space.
• Then the “temperature” is gradually reduced and the selection of new
moves becomes more and more selective.
• At the end just improving moves are accepted.
Federico Della Croce Heuristics 52 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Main steps of
Simulated Annealing
1) Initialization:
Select an initial solution x1∈Χ
Set the current solution: x’ = x1
Set to 0 the iterations counter: I = 0.
2) Neighborhood generation N(x’) of the current solution.
3) Select randomly a solution x’’ from N(x’);
Update the iterations counter: I = I + 1;
IF x’’ is better than or equal to x’ THEN x’ = x’’,
ELSE: −
F ( x '') − F ( x ')
Compute probability p = e T (I )
Select a random real value p∈[0,1]
IF p< p
THEN x’ = x’’,
ELSE keep the current solution unchanged
4) Stopping test: IF the stopping test is positive STOP,
ELSE GO TO 2.
Federico Della Croce Heuristics 53 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Values of the temperature T(I)
The temperature has a typical behaviour as the one depicted in the figure
below.
T (I)
T0
T1
T
T2(I)
T3
L 2L 3L 4L I
• T (k⋅L) = Tk = αk ⋅ T0
• T0 = initial temperature: usually the value is computed in such a way to
have an acceptance probability w.r.t. an average neighbor solution.
• α ∈[0,1] = cooling coefficient (e.g. α = 0.98)
• L = length of the “plateau” (typically proportional to the size of N()).
Federico Della Croce Heuristics 54 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Stopping test
Typically the following alternatives are considered:
– The solution did not improve more than ε1% for a given
value of consecutive iterations (e.g. k⋅L).
– The total number of accepted moves is below ε2% in the
last k2⋅L iterations.
– A given threshold S* of has been reached.
– An overall CPU time limit τ* has been reached.
The calibration of such parameters
must be derived based on experimental testing.
Federico Della Croce Heuristics 55 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Metaheuristics
Tabu Search
• Steepest Descent like local search: tries to avoid
local minima.
• At each iteration all neighbor solutions (or a proper
subset of the considered neighborhood) are
evacuate.
• All decisions are deterministic.
• Main features
– “Tabu moves”
– Tabu List
– Aspiration Criteria (e.g.: an overall improving solution is always
accepted)
– Intensification / diversification in the solutions space
Federico Della Croce Heuristics 56 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Main steps of Tabu Search
1) Initialization:
Select an initial solution x1∈Χ; Set the current solution: x’ = x1;
Empty the Tabu List. TL = {}; Set to 0 the iterations counter: I = 0.
2) Select the neighborhood N(x’) of the current solution and evaluate N(x’) or a subset
N'(x’).
Let x* be the best found solution in N(x’) such that:
x* improves upon the best currently available feasible solution
(aspiration criterion)
or
m(x’,x*) does not belong to the Tabu List TL, that is the move from x’ to x* is not
a Tabu move
THEN
set x’ = x*;
insert the move m(x*,x’) in the Tabu list, that is TL=TL ∪ m(x*,x’);
IF the Tabu List size has reached a given threshold,
THEN eliminate from TL the “oldest” move;
Update the iterations counter: I = I + 1.
3) Stopping test: IF the stopping test is positive STOP, ELSE GO TO 2.
Federico Della Croce Heuristics 57 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Implementation of a
Tabu Search procedure
To implement a Tabu Search procedure the following
items must be considered:
– Definition of a neighborhood or of a subset of such
neighborhood and evaluation of all its components.
– Selection of the moves to be inserted in the Tabu List and
representation of such moves.
– Length of the Tabu List.
– Selection of the aspiration criterion (criteria).
– Selection of some stopping test.
– Intensification / diversification techniques.
Federico Della Croce Heuristics 58 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Genetic algorithms
• Developed: USA in the 1970’s
• Early names: J. Holland, K. DeJong, D. Goldberg
• Now part of a more complex framework: the
Evolutionary Algorithms
• Attributed features:
• not too fast
• good heuristics for combinatorial problems but…
usually not the best
• Easy to implement, “applicable to every problem”
Federico Della Croce Heuristics 59 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Genetic algorithms
• Imitation of natural evolutionary processes.
• Heuristic combinatorial optimization approach that is based on
the following evolutionary operators
– selection
– reproduction
– mutation
Federico Della Croce Heuristics 60 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Genetic algorithms
Genetic algorithms possess the following characteristics:
• The search is not entrapped in local minima.
• They are naturally suitable to be applied on parallel
architectures parallele.
• Easy hybridization with other heuristics.
Federico Della Croce Heuristics 61 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Genetic algorithms for C.O.
1. The search space of all possible solutions is mapped onto a set of finite
strings; the alphabet of symbols is of finite dimensions, in a typical case
it includes two symbols (0 and 1). Each string (chromosome) has a
corresponding point in the search space.
2. An iteration threshold is chosen, typically a number of generations.
3. A set of solutions is selected (usually at random) and this constitutes the
initial population, i.e. the first generation.
4. A fitness is computed for each of the individuals; fitness is a parameter
which reflects the quality of the solution represented by the individual.
5. A set of individuals is selected to reproduce itself, the selection takes
place in a random way but with a probability proportional to fitness.
Federico Della Croce Heuristics 62 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Genetic algorithms for C.O.
6. From the selected set some progeny is generated by applying different
genetic operators.
7. Some of the old individuals are eliminated from the population to allow
the entrance of the new ones.
8. The new individuals' fitness is computed and they are included in the
population. This step marks the end of a generation.
9. IF the iteration threshold is reached, THEN the process is finished and
the best chromosome is returned, ELSE GO TO step 5.
Federico Della Croce Heuristics 63 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Constructing a Genetic Algorithm
The main phases in the construction of a genetic algorithm are the following:
• Coding
• Initial population
• Fitness function definition
• Genetic operators
– Crossover
– Mutation
– Inversion
• Genetic operators selection
• Reproduction
• Parameters calibration
Federico Della Croce Heuristics 64 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Coding
• Consists in defining a representation of the feasible solutions in terms of
chromosomes.
• strictly depends from the problem. For a given problem many options of
coding are possible (traditionally a binary alphabet is used for the
chromosomes), but the genetic operators definition is linked to the used
coding.
Federico Della Croce Heuristics 65 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Initial population
• random generation of several of chromosomes
• initial population elements generated according to
simple heuristics applied to the considered problem
Federico Della Croce Heuristics 66 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Fitness
• The fitness is a parameter that measures the quality of the solution of
any given chromosome;
• it is used to compute chromosome reproduction probability (the larger
the fitness function, the larger the probability to select that chromosome
for reproduction);
• example:
– fitness = f(x) in a maximization problem
– fitness = 1/f(x) in a minimization problem;
Federico Della Croce Heuristics 67 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Classic genetic operators
• STANDARD CROSSOVER
Operates on a couple of chromosomes: the “children" are originated from the
“parents” swapping the genes between two cutting sites ramdomly
chosen.
parent 1 1 0 1 | 0 1 1 1 | 1 0
parent 2 0 0 0 | 1 1 0 1 | 0 1
^ ^ cutting sites
child 1 1 0 1 | 1 1 0 1 | 1 0
child 2 0 0 0 | 0 1 1 1 | 0 1
Federico Della Croce Heuristics 68 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Classic genetic operators
ORDER CROSSOVER
• Two cutting sites are ramdomly chosen.
• Choose a a subsequence of a list from one parent and preserve
the relative order of the items from the other.
• Each child is obtained by selecting the crossing section of a
parent and filling the remaining elements with the missing
symbols according to the relative order of the other parent.
Federico Della Croce Heuristics 69 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Classic genetic operators
ORDER CROSSOVER (example)
Parent 1 = (1 2 3 | 5 4 6 7 | 8 9) Parent 2 = (4 5 2 | 1 8 7 6 | 9 3)
Child 1 = (x x x | 5 4 6 7 | x x) Child 2 = (x x x | 1 8 7 6 | x x)
The loading in parent 1, starting from its second cut point, is
9 o 3 o 4 o 5 o 2 o 1 o 8 o 7 o 6.
Remove the items already in child 1, obtaining the partial tour
9 o 3 o 2 o 1 o 8.
Insert this partial list after the second cut point of child 1, obtaining
Child 1 = (2 1 8 | 5 4 6 7 | 9 3 ). Similarly, child 2 = (3 4 5 | 1 8 7 6 | 9 2).
Federico Della Croce Heuristics 70 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Classic genetic operators
MUTATION
Operates on a single chromosome: a gene is ramdomly substituted by
another symbol of the alphabet. It generally is applied to a minimal part of
the chromosomes (ramdomly chosen).
1 0 1 0 1 1 1 1 0
^
1 0 0 0 1 1 1 1 0
Federico Della Croce Heuristics 71 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Classic genetic operators
INVERSION
Operates on a single chromosome: the order between the cutting sites is
inverted
1 0 1 | 0 1 1 1 | 1 0
^ ^
1 0 1 | 1 1 1 0 | 1 0
Federico Della Croce Heuristics 72 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Genetic operators selection
• If the algorithm uses more than one operator, also the genetic operators
selection can be executed in a probabilistic way.
• The operators selection probabilities may be constant or evolve suring
the process.
• Mutation and inversion typically have low probability to be used.
However they may become very useful whenever the population is quite
uniform.
Federico Della Croce Heuristics 73 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Reproduction
• Generational replacement (the new generation fully substitutes the old
one).
• Elitist technique (the best element is always kept in the new generation;
as a variant several “best elements” are kept).
• Steady state technique (most of the element are kept and just a few
elements are substituted; typically, the worst element in probability are
eliminated).
Federico Della Croce Heuristics 74 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Heuristics
Parameters calibration
• Various parameters determine the behavior of a genetic algorithm. The
main ones are the following:
• Population size;
• Percentage of elements of a population substituted in the following
population;
• Number of generations;
• Probability in applying the different genetic operators.
Federico Della Croce Heuristics 75 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Hybridization
• Combination (hybridization) of techniques is a popular and well
established practice in the metaheuristic field.
• The popularity, success and importance of that specific line of
research is well documented by the huge number of successful
reported applications.
• So, combining techniques with complementary advantages, such as
exact and metaheuristic approaches appears to be a winning chance.
• Recently, the hybridization of exact methods and metaheuristic has
originated the so-called Matheuristics1 .
• Hereafter two slides stolen from M. Fischetti one of the inventors of
matheuristics.
1
Matheuristics 2006, 1st International Workshop on Mathematical Contributions to
Metaheuristics, 27-30 August 2006, Bertinoro (FC)
Federico Della Croce Heuristics 76 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Matheuristics
MIP solvers for hard optimization problems
• Mixed-integer linear programming (MIP) plays a central role in modelling difficult-to-solve
(NP-hard) combinatorial problems
• General-purpose (exact) MIP solvers are very sophisticated tools, but in some hard cases they
are not adequate even after clever tuning
• One is therefore tempted to quit the MIP framework and to design ad-hoc heuristics for the
specific problem at hand, thus loosing the advantage of working in a generic MIP framework
• As a matter of fact, too often a MIP model is developed only “to better describe the problem” or,
in the best case, to compute bounds for benchmarking the proposed ad-hoc heuristics
Can we devise an alternative use of a general-purpose MIP solver, e.g.,
to address important steps in the solution process?
2
Federico Della Croce Heuristics 77 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Matheuristics
MIPping critical sub-tasks in the design of specific algorithms
We teach engineers to use MIP models for solving their difficult problems
(telecom, network design, scheduling, etc.)
Be smart as an engineer!
Model the most critical steps in the design of your own algorithm
through MIP models, and solve them (even heuristically) through a
general-purpose MIP solver…
6
Federico Della Croce Heuristics 78 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Framework Description
• The over guiding framework of our approach is the Local Search
approach.
• Problem tackled are all formulated as MIPs.
• We rely on (partial) starting solutions that can be generated by a
MIP solver or by any problem specific heuristic.
• We generate exponential size neighborhoods and we handle them by
means of a MIP solver2 .
• Neighborhoods are defined in such way that they are “linear
constraints” of the corresponding MIP formulation of the problem.
2
Notice that also other exact methods can be applied
Federico Della Croce Heuristics 79 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Matheuristic applications
• Nurse rostering problem.
• Multi-dimensional knapsack problem.
Federico Della Croce Heuristics 80 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Nurse Rostering Problem – Personnel Shifts
MARCH 2016 1 2 3 4 5 6 7 8 9 10 11 12 13 14
Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo
HEAD NURSE H H O O O O
Nurse 1 N N O O M A N N O O A A A N
Nurse 2 A O M M O M M N N O O M M A
Nurse 3 H A H N H H M A N N O O O O
Nurse 4 O A A A N O O M M N N O O M
Nurse 5 O M N O O M A O A A O N O O
Nurse 6 N N O O M A A A O M M A N O
Nurse 7 M A O A A N N O O M M A A O
Nurse 8 M O N N O O A A A O M N N O
Nurse 9 O M A O A A O M M A A O O M
Nurse 10 (50%) M O O O N N O O O O O M M N
Nurse 11 A O M M O M M N N O O M M A
Nurse 12 N O O A A N N O O A A O A A
Nurse 13 O M A N N O O M A N N O O M
Nurse 14 (Mat.) O O M M M O O H H H H H N N
Nurse 15 H H H H H O O H H H H H O O
Nurse 16 A N N O H O O H M M N N O O
LEGENDA:
M = Morning; A = Afternoon; N = Night
H = Holiday; O = Off/Rest;
Federico Della Croce Heuristics 81 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Nurse Rostering Problem(s) (NRP)
• The problem considered is based on a real situation encountered in a
ward of a private hospital.
• At the end of each month, the nurses’ shifts for the following month
must be scheduled.
• The ward is made up of a given number of hired nurses, but the
timetable of these nurses is not always sufficient to cover the legal
minimum of personnel presence in every shift.
• The hospital management can make use of freelance nursing staff to
cope with the minimum demand of personnel.
• Freelance staff has a cost.
• The problem consists in assigning a working shift or a day off to each
nurse, on each day, in order to reduce the outsourced work.
Federico Della Croce Heuristics 82 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
NRP What is needed?
1 Problem description (head nurse).
• Problem specs.
• Legal regulations.
• Special regulations (commons, etc.).
• ...
2 Problem formulation.
• Constraints.
• Objective Function.
3 Solution Approach.
• Solution algorithm.
Federico Della Croce Heuristics 83 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
NRP Problem Description
• Labor Contracts rules:
• no more than 2 consecutive Night shifts
• no sequence Night–Morning shift.
• no complete freelance assignment for shifts.
• ...
• Operational constraints
• minimum personnel (ex. Morning 3, Afternoon 3, Night 2)
• shifts number and type balance among nurses
• holidays requests.
• ...
• Special constraints
• maternity
• part–time
• ...
Federico Della Croce Heuristics 84 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
NRP Problem formulation – Variables and Objective
Function
• The considered problem can be formulated as an IP model.
• With n nurses and m days in a month, it is sufficient to introduce a
set of 0/1 variables denoted xi,j,k (i = 1, ...n, j = 1, ...m, k = 1, ...t)
indicating if nurse i is assigned to shift k on day j.
• Correspondingly, a positive integer variable yj,k is introduced
indicating the number of freelance nurses needed in working shift k
on day j.
• Objective Function: min m
P Pt
j=1 k=1 yj,k
Federico Della Croce Heuristics 85 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
NRP Problem formulation – Constraints
• no sequence Night–Morning shift
xi,j,N + xi,j+1,M ≤ 1 ∀i = 1, . . . , n, j = 1, . . . , m − 1
• no more than 2 consecutive Night shifts
P2
s=0 xi,j+s,N ≤2 ∀i = 1, . . . , n, j = 1, . . . , m − 2
• minimum personnel
Pn
i=1 xi,j,k ≥ min persj,k ∀j = 1, . . . , m, k = 1, . . . , 3
• holiday requests
xi,j,H = 1 ∀i, j ∈ holiday requestsi,j
• at least 4 and at most 8 Afternoon shifts for each nurse
Pm
4≤ j=1 xi,j,A ≤8 ∀i = 1, . . . , n
Federico Della Croce Heuristics 86 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
NRP Solution Approach
• The first attempt is done to verify the problem model against the
real–life situation so as to check coherence and correctness.
• We can model and solve the problem by means of a MIP solver.
Solving....
Federico Della Croce Heuristics 87 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
NRP Solution Approach
• The first attempt is done to verify the problem model against the
real–life situation so as to check coherence and correctness.
• We can model and solve the problem by means of a MIP solver.
Solving....
NO FEASIBLE SOLUTION EXISTS!!!
Federico Della Croce Heuristics 87 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
NRP Solution Approach – Possible mistakes 1
1 Bugs into the model.
2 Constraints violations: given the constraints it is impossible to
generate a feasible roster.
Federico Della Croce Heuristics 88 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
NRP Solution Approach – Possible mistakes 2
• The head nurse has generated rosters since many years.
• How it is possible?
• Check of a head nurse’s solution
Federico Della Croce Heuristics 89 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
NRP Solution Approach – Possible mistakes 2
• The head nurse has generated rosters since many years.
• How it is possible?
• Check of a head nurse’s solution
• UNFEASIBLE!!!
Federico Della Croce Heuristics 89 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
NRP Solution Approach – Possible mistakes 2
• The head nurse has generated rosters since many years.
• How it is possible?
• Check of a head nurse’s solution
• UNFEASIBLE!!!
• This means that the head nurse violates some given constraints while
generating rosters.
Federico Della Croce Heuristics 89 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
NRP Solution Approach – how to go on?
• Identification of the subset of constraints usually (smoothly) violated
by the head nurse.
• Those constraints are then relaxed so as to find feasible solutions.
Relax means:
• accept (small) violations
ex. no more than 4 consecutive working days but a fifth day is accepted once a
month.
• enlarge feasible solution space
ex. minimum number of Afternoons shifts from 4 to 3.
Federico Della Croce Heuristics 90 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
NRP Solution Approach – Recap
Having relaxed constraints we now have:
• new problem description,
• new constraint set,
• guarantee of feasible solutions.
Federico Della Croce Heuristics 91 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
NRP Solution Algorithm – 1
• It is known that solvers are not particularly efficient on large size
nurse rostering problems.
• One hour of CPU time is not sufficient to find the optimal solution or
even good solutions.
• We want to improve solutions quality given the same CPU time (1 h.)
• Matheuristic algorithm. This means that:
• we can reuse the MIP model
• we can take advantage of solvers
• we have to design neighborhoods and implement a local search phase.
Federico Della Croce Heuristics 92 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
NRP Solution Algorithm – 2
Given a feasible solution X̄ we define the following neighborhoods:
1 Hamming distance
N1 (X̄ , h1 ) sets the number of variables xi,j,k that can chance value w.r.t the
current solution is less or equal than h1
2 Days slot
N2 (X̄ , h2 ) defines all subset of size h2 of consecutive days that can change
value.
3 Nurse slot
N3 (X̄ , h3 ) defines all subset of size h3 of consecutive nurses that can change
value..
Federico Della Croce Heuristics 93 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
NRP Solution Algorithm – 3
• All described constraints can be modeled as linear constraints thus
any MIP solver is able to solve them.
• Neighborhoods define portions of solutions that remain constant while
trying to optimize portions were we let variables free.
• We have generated subproblems very similar to the global problem
but of a smaller (controlled) size.
Federico Della Croce Heuristics 94 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
NRP Solution Algorithm – An Example
Federico Della Croce Heuristics 95 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
NRP Solution Algorithm – Pseudocode
Begin
(1) choose an initial feasible solution z
(2) initialize parameters h1, h2 and h3
(3) iter := 1
(4) while iter <=3 do
(5) add neighborhood constraint to the model -- N_iter
(6) set the variables not belonging to the considered neighborhood
to their corresponding values in z
(7) solve subproblems within local time limit
(8) if f(x) < f(z) then do
(9) update solution
(10) iter:=1
(11) else do
(17) iter:=iter+1
(19) end if
(20) if the overall CPU time is greater than 3600 s. then
(21) EXIT
(22) end if
(24) end-do
End
Federico Della Croce Heuristics 96 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
NRP Solution Algorithm – Results 1
Federico Della Croce Heuristics 97 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
NRP Solution Algorithm – Results 2
Federico Della Croce Heuristics 98 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
NRP Conclusions
1 We have built a MIP model for creating rosters.
2 We had to refine the model so as to guarantee feasibility.
3 We have obtained starting feasible solutions by means of a MIP
solver .
4 We have developed an ad–hoc algorithm to improve solution quality.
5 We have heuristically applied and exact method.
Federico Della Croce Heuristics 99 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Matheuristics for MKP
• MKP: well known tough NP-Hard model that represents several real
problems.
• There exist plenty of heuristics and meta-heuristics as much as exact
approaches for this problem.
• OR-Library benchmark problems with 100 - 250 - 500 variables and 5
- 10 - 30 constraints.
• Typically 0/1 multi-dimensional knapsack problems with 30 - 35
variables can be solved (by XPRESS-CPLEX etc.) to optimality
within few seconds.
• Most of the benchmark problems with 500 variables and 30
constraints are currently open with respect to optimality
Federico Della Croce Heuristics 100 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Matheuristics for MKP: ILP model and notation
n
X
max pj xj
j=1
n
X
wij xj ≤ ci i = 1, ...m
j=1
xj ∈ {0, 1} j = 1, ...n
that is
• a set of n items with profits pj > 0
• a set of m resources with capacities ci > 0 ;
• each item j consumes an amount wij ≥ 0 from each resource i ;
• the selected items must not exceed the resource capacities ci ;
• the 0 − 1 decision variables xj indicate which items are selected ;
• the goal is to choose a subset of items with maximum total profit.
Federico Della Croce Heuristics 101 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Matheuristics for MKP: notation
• Z ∗ = the value of the continuous relaxation solution X ∗
• Z = the value of the optimum X
• rj = the reduced cost of the non basic variable xj∗
• uj = the upper pseudo-cost of the basic variable xj∗
• lj = the lower pseudo-cost of the basic variable xj∗
• parameters rj , uj , lj are available from the continuous relaxation
solution.
Federico Della Croce Heuristics 102 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
MKP: properties of the continuous relaxation solution X ∗
• For all non-basic 0-1 xj variables such that xj∗ = 0,
IF xj = 1 then Z ≥ Z ∗ + |rj |.
• For all non-basic 0-1 xj variables such that xj∗ = 1,
IF xj = 0 then Z ≥ Z ∗ + |rj |.
• In other words, the larger |rj |, the higher the probability that xj∗ = xj .
• For all basic 0-1 xj variables,
IF xj = 1 then Z ≥ Z ∗ + |uj |(1 − xj∗ );
IF xj = 0 then Z ≥ Z ∗ + |lj |xj∗ .
• In other words, if |uj |(1 − xj∗ ) < |lj |xj , it is more likely to have xj∗ = 1
and viceversa.
• IF Z ∗ is reliable, the number of variables where xj∗ = 1 − xj is limited.
Federico Della Croce Heuristics 103 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
MKP: A basic core problem approach
• Compute Z ∗ .
• Fix the n − k non-basic xj variables with largest |rj | to their xj∗ value.
• Solve to optimality (or with a time limit) the remaining core problem
with k variables where no more than γ variables are such that
xj∗ = 1 − xj .
• Fair values of k and γ are 50 and 10.
Such a simple approach dominates the best performing metaheuristics
(among others, a genetic algorithm)
Federico Della Croce Heuristics 104 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Matheuristics: discussion (1)
• The most promising way to apply matheuristics appears to be based
on a neighborhood search slave procedure realized by a MIP solver (or
an exact method).
• It seems reasonable to “mip3 ” all metaheuristics that rely in some
part of the algorithm of features based on local search; for example a
Math - Iterated Local Search or a Math - Genetic algorithm.
• It seems that the typical intensification phase of a method when
tackled by means of an exact method is well performing because:
1 The solutions achieved by pure metaheuristics are often not local
minima for combinatorial neighborhoods.
2 MIP local minima are more robust than the metaheuristic ones. They
are, in fact, also local minima for a corresponding metaheuristic
procedure.
3
transform a local search procedure with polynomial size neighborhood, into a very
large scale neighborhood procedure where an exponential size neighborhood is handled
by means of an exact method
Federico Della Croce Heuristics 105 / 106
COMBINATORIAL OPTIMIZATION: heuristic approaches
Matheuristics: discussion (2)
• The definition of a neighborhoods structure in an exact procedure
exploiting the MIP formulation of the problem is often
straightforward.
• Many kind of different neighborhoods can be defined for solving a
combinatorial optimization problem.
• Not only general purpose methods (e.g. branch and bound) but also
dedicated exact methods can be embedded into a matheuristic in
order to cope with problem-specific features.
• Also the diversification phase can be nested in the matheuristic
framework in order to escape from local minima.
• Last but not least, the use of generic solvers, commercial or open
source will guarantee, in our opinion, a continuous development of
matheuristic approaches.
Federico Della Croce Heuristics 106 / 106