Al-Nahrain University,

College of Engineering
Electronics and Communications Dep.

12/8/2017 Optimization Technique

Tabu Search

Prepared by:
ADNAN
ZAID
SHAHAD

Supervised by:
Assistant Prof. Dr.Dhirgham K. Naji
 Agenda

Introduction to Tabu Search

- Tabu criterion
 Tabu Tenure
- Aspiration
- Examples of use
 Knapsack
 TSP
- Intensification
- Diversification
 Terminology:Neighborhoods
• To make sure we understand this
correctly:
• A neighborhood is a set of solutions
that are close to one given solution
 A set of solutions that can be reached from
another by making only one move
 N(x) usually denotes the neighborhood of x

• A neighborhood operator is a function

that takes a solution and returns its
neighborhood
 N: S →2S
 So, while a neighborhood is one specific
collection of neighbors, a neighborhood
operator is the blue-print for how to make
those neighbors if we are given an initial
solution around which to build the
neighborhood
• Let us assume we have a knapsack
problem, with n binary variables
• Furthermore,let us use flip-
neighborhoods
• That is, N(x) = {all y in S, such that the
Hamming distance between y and x is 1}
 N is the neighborhood operator
 N(x) is the neighborhood of solution x

Tabu
• The word tabu (or taboo) comes
from Tongan
– a language of Polynesia
– used by the aborigines of Tonga island
to indicate things that cannot be
touched because they are sacred
 Tabu Search

• Tabu Search:
 Cut off the search from parts of the search
space (temporarily)
 Guide the search towards other parts of
the search by using penalties and bonuses

• Uses principles for intelligent problem

solving
• Uses structures that are exploring the
search history, without remembering
everything
 Branch&Bound, A*: have complete
memory
 Simulated Annealing: have no memory
 Main Ideas of Tabu Search
• Based on Local Search – LS
• Allows non-improving moves
 can exit local optima

• Uses extra memory to avoid looping,

and to diversify the search
• General strategy for controlling a LS, or
other “inner” heuristic
• Meta-Heuristic (Glover)

General Formulation
 Some Critical Choices

• Choice of neighborhood, N
• Definition of the tabu memory
• How to select the candidate list
• The definition of the evaluation function
– Improvement in solution values
– Tabu criteria
– Aspiration criteria
– Long term strategies
 Basic Tabu Search

• Local Search with “Best Improvement”

strategy
 Always select the best move

• But: Some neighbors are tabu, and

cannot be selected
 Defined by the tabu criterion
 Tabu neighbors might be selected anyway if
they are deemed to be good enough
- Aspiration criterion
• Memory – tabu list
 Flowchart of a Standard Tabu
Search Algorithm

Initial solution Create a candidate

Evaluate solutions
(i in S) list of solutions

Update Tabu & No Stopping Choose the best

Aspiration conditions admissible solution
Conditions satisfied ?

Yes

Final solution
 Tabu Attribute Selection

• Attribute
 A property of a solution or a move

• Can be based on any aspect of the

solution that are changed by a move
• Attributes are the basis for tabu
restrictions
 We use them to represent the solutions
visited recently
• A move can change more than one
attribute
 e.g. a 2-opt move in TSP involves 4 cities
and 4 edges

• Similar to the “features” in GLS, but we

don’t require the attributes to have
costs
 Example – Attributes in TSP

• Attributes based on the edges

– A1: Edges added to the tour
– A2: Edges removed from the tour

• Move of type exchange

– Exchanges two cities
– 4 edges removed
– 4 edges added
– Exchange(5,6)
• A1:(2,5),(5,7),(4,6),(6,1)
• A2:(2,6),(6,7),(4,5),(5,1)

5 5

2 3 2 3
7 7

1 1

4 6 4
6
 Example 2

• Minimum spanning tree problem with

constraints.
 Objective: Connects all nodes with minimum
costs

Costs

B B
20 30 20 30

10 5 10 5
A C E A C E

25 25
15 40 15 40
D D

An optimal solution without

considering constraints

Constraints 1: Link AD can be included only if link DE also is included. (Penalty: 100)
Constraints 2: At most one of the three links – AD, CD, and AB – can be included.
(Penalty of 100 if selected two of the three, 200 if all three are selected.)
 Iteration 1

 Cost=50+200 (constraint
penalties)

Add Delete Cost

BE CE 75+200=275
B
20 30 BE AC 70+200=270

10 5 BE AB 60+100=160
A C E
CD AD 60+100=160
25
CD AC 65+300=365
Delete 15 D
40 Add
DE CE 85+100=185
DE AC 80+100=180
DE AD 75+0=75

New cost = 75 (iteration 2)

(Local optimum)

Constraints 1: Link AD can be included only if link DE also is included. (Penalty: 100)
Constraints 2: At most one of the three links – AD, CD, and AB – can be included.
(Penalty of 100 if selected two of the three, 200 if all three are selected.)
 Tabu list: DE
Iteration 2

 Cost=75

Add Delete Cost

Delete
B 30 Add AD DE* Tabu move
20
AD CE 85+100=185
10 5 AD AC 80+100=180
A C E

BE CE 100+0=100
25
15 40 Tabu BE AC 95+0=95
D
BE AB 85+0=85

CD DE* 60+100=160
CD CE 95+100=195

* A tabu move will be considered only if it would result

in a better solution than the best trial solution found
previously (Aspiration Condition)
Iteration 3 new cost = 85 Escape local optimum

Constraints 1: Link AD can be included only if link DE also is included. (Penalty: 100)
Constraints 2: At most one of the three links – AD, CD, and AB – can be included.
(Penalty of 100 if selected two of the three, 200 if all three are selected.)
 Tabu list: DE & BE
Iteration 3

 Cost=85

Add Delete Cost

Tabu AB BE* Tabu move

20 B
30
AB CE 100+0=100

10 5 AB AC 95+0=95
A C E
AD DE* 60+100=160
25 Add
15 40 Tabu AD CE 95+0=95
D
AD AC 90+0=90
Delete
CD DE* 70+0=70
CD CE 105+0=105

* A tabu move will be considered only if it would result

in a better solution than the best trial solution found
previously (Aspiration Condition)
Iteration 4 new cost = 70 Override tabu status

Constraints 1: Link AD can be included only if link DE also is included. (Penalty: 100)
Constraints 2: At most one of the three links – AD, CD, and AB – can be included.
(Penalty of 100 if selected two of the three, 200 if all three are selected.)
 B Optimal Solution
20 30
Cost = 70
10 5
A C E Additional iterations only find
25 Inferior solutions
15 40
D

Tabu Tenure (1)

• The tabu criterion will disallow moves
that change back the value of some
attribute(s)
• For how long do we need to enforce
this rule?
 For ever: the search stops because no
changes are allowed
  For too long: the search might become too
limited (too much of the search space is cut
off due to the tabu criterion)
 For too short: the search will still cycle, but
the length of the cycle can be more than 2
• The number of iterations for which the value of
the attribute remains tabu is called the Tabu
Tenure

Tabu Tenure (2)

• Dependent on the tabu attributes
• Example: TSP – n cities – 2-opt
 Use edges-added and edges-dropped as
tabu attributes
 |n2| edges in the problem instance
 |n| edges in the solution
Many more edges outside the solution than
in the solution
 Using the same TT would be unbalanced
 Example: 0/1 Knapsack
• Flip-Neighborhood
• If the move is selecting an item to
include in the solution, then any
move trying to remove the same item
is tabu for the duration of the tabu
tenure
• Similarly, an item thrown out is not
allowed in for the duration of the
tabu tenure iterations
• Here the attribute is the same as the
whole move
 Flip Neighborhood

Current Solution

0000 0100 1000 1100

0 11 5 16

0001 0101 1001 1101

7 18 12 23

0010 0110 1110

1010
9 20 25
14

0011 0111 1011 1111

16 27 21 32
 Flip Neighborhood

Neighbor

Infeas.
Neighbor
Variables flipped so far: none

0100 1000 1100

0000
11 5 16
0

1001 1101
0001 0101
12 23
7 18

0010 0110 1010 1110

9 20 14 25

0011 0111 1011 1111

16 27 21 32
 Flip Neighborhood
Neighbor

Infeas.
Neighbor
Variables flipped so far: 3

0000 0100 1000 1100

0 11 5 16

0001 0101 1001 1101

7 18 12 23

0010 0110 1010 1110

9 20 14 25

0011 0111 1011 1111

16 27 21 32
 Flip Neighborhood

Neighbor

Infeas.
Neighbor
Variables flipped so far: 3

0000 0100 1000 1100

Tabu!
0 11 5 16

0001 0101 1001 1101

7 18 12 23

0010 0110 1110

1010
9 20 25
14

0011 0111 1011 1111

16 27 21 32
 Flip Neighborhood

Neighbor

Infeas.
Neighbor

Variables flipped so far: 3, 2

0000 0100 1000 1100

0 11 5 16

0001 0101 1001 1101

7 18 12 23

0010 0110 1010 1110

9 20 14 25

0011 0111 1011 1111

16 27 21 32
 Flip Neighborhood

Neighbor

Infeas.
Neighbor
Variables flipped so far: 3, 2

0000 0100 1000

0 1100
11 5 16

0001 0101 1001 1101

7 18 12 23
Tabu!

0010 0110 1010 1110

9 20 14 25

0011 0111 1011 1111

16 27 21 32
 Local and Global optima

Solution value

Solution space
 Aspiration Criterion

• The tabu criterion is usually not exact

 Some solutions that are not visited are
nevertheless tabu for some time

• Possible problem: one of the

neighbors is very good, but we
cannot go there because some
attribute is tabu
• Solution: if we somehow know that
the solution is not visited before, we
can allow ourselves to move there
anyway
 i.e., the solution is a new best solution:
obviously we have not visited it before!
 TS - Diversification

• Basic Tabu Search often gets stuck

in one area of the search space
• Diversification is trying to get to
somewhere else
• Historically random restarts have
been very popular
• Frequency-based diversification tries
to be more clever
 penalize elements of the solution that have
appeared in many other solutions visited
 Intensification and Diversification
- simple mechanisms

• Use of frequency-based memory

• Based on a subset Sf of all the
solutions visited (or moves executed)
• Diversification:
 Choose Sf to contain a large part of the
generated solutions (e.g. all the local
optima)

• Intensification:
 Choose Sf to be a small subset of elite
solutions
• E.g., that have overlapping attributes
 Can have several such subset
• Partitioning, clustering-analysis
 Whips and Carrots

• Used in the move evaluation

function, in addition to the change in
the objective function value and tabu
status
• A carrot for intensification will be a
whip for diversification
• Diversification:
 Moves containing attributes with a high
frequency count are penalized
 TSP-example: g(x)=f(x)+w1 ij

• Intensification:
 Moves to solutions containing attributes with a
high frequency among the elite solutions are
encouraged
 TSP-example: g(x)=f(x)-w2ij
 TS Example: TSP

• Representation: permutation vector

• Move: pairwise exchange
5

 i, j  i j i, j  1, n
2 3

1 2 3 4 5 6 7
1

4
6
Move: Exchange 5

in permutation 2
7
3

2 vector
6 7 3 4 5 1 1

4
6

Move: Exchange(5,6) 5

2 3

7
2 5 7 3 4 6 1
1

4
6
 TSP Example

 n
 
• Number of neighbors:  2 
• For every neighbor: Move value
 k 1  f (ik 1 )  f (ik ), ik 1  N (ik )

• Choice of tabu criterion

 Attribute: cities involved in a move
 Moves involving the same cities are tabu
 Tabu tenure = 3 (fixed)

• Aspiration criterion
 new best solution
TSP Example: Data structure

• Data structure: triangular table,

storing the number of iterations until
moves are legal
• Updated for every move

2 3 4 5 6 7
1 0 2 0 0 0 0
2 0 3 0 0 0
3 0 0 0 0
4 1 0 0
5 0 0
6 0
TSP Example: Tabu Criteria/Attributes

• Illegal to operate on given cities

• Illegal to change the city in position k
in the vector
• Criteria on edges
– Links often present in good solutions
– Length of links w.r.t. the average

• For permutation problems 5

– Attributes related to previous/next

2 3
often work well
7

4
6

1 2 3 4 5 6 7
2 4 7 3 5 6 1
 TSP Example: Iteration 0

Starting solution: Value = 234

1 2 3 4 5 6 7
2 5 7 3 4 6 1

Tabu list:
2 3 4 5 6 7
1 0 0 0 0 0 0
2 0 0 0 0 0
3 0 0 0 0
4 0 0 0
5 0 0
6 0
 TSP Example: Iteration 1

Current solution: Value = 234 Candidate list:

1 2 3 4 5 6 7
Exchange Value
2 5 7 3 4 6 1
5.4 -34
7.4 -4
1 2 3 4 5 6 7 3.6 -2
2 4 7 3 5 6 1 2.3 0
4.1 4
After move: Value = 200

Tabu list:
2 3 4 5 6 7
1 0 0 0 0 0 0
2 0 0 0 0 0
3 0 0 0 0
4 3 0 0
5 0 0
6 0
 TSP Example: Iteration 2

Current solution: Value = 200

1 2 3 4 5 6 7
2 4 7 3 5 6 1
Exchange Value
Candidate list: 3.1 -2
2.3 -1
Choose move (3,1)
3.6 1
7.1 2
6.1 4
Tabu list:
2 3 4 5 6 7
1 0 0 0 0 0 0
2 0 0 0 0 0
3 0 0 0 0
4 3 0 0
5 0 0
6 0
 TSP Example: Iteration 2

Current solution: Value = 200

1 2 3 4 5 6 7
2 4 7 3 5 6 1

Exchange Value
Candidate list: 3.1 -2
2.3 -1 Choose move (3,1)
3.6 1
7.1 2 Update tabu list
6.1 4
Tabu list:
2 3 4 5 6 7
1 0 3 0 0 0 0
2 0 0 0 0 0
3 0 0 0 0
4 2 0 0
5 0 0
6 0
 TSP Example: Iteration 3

Current solution: Value = 198

1 2 3 4 5 6 7
2 4 7 1 5 6 3

Candidate list: Exchange Value Tabu!

1.3 2
2.4 4 Choose move (2,4)
7.6 6
4.5 7 NB: Worsening move!
5.3 9
Tabu list:
2 3 4 5 6 7
1 0 3 0 0 0 0
2 0 0 0 0 0
3 0 0 0 0
4 2 0 0
5 0 0
6 0
 TSP Example: Iteration 3

Current solution: Value = 198

1 2 3 4 5 6 7
2 4 7 1 5 6 3

Candidate list: Exchange Value Tabu!

1.3 2
2.4
7.6
4
6
Choose move (2,4)
4.5 7 NB: Worsening move!
5.3 9
Tabu list:
2 3 4 5 6 7 Update
1 0 2 0 0 0 0 tabu list
2 0 3 0 0 0
3 0 0 0 0
4 1 0 0
5 0 0
6 0
 TSP Example: Iteration 4

Current solution: Value = 202

1 2 3 4 5 6 7
4 2 7 1 5 6 3

Candidate list: Exchange

4.5
Value
-6 Tabu!
5.3
7.1
-2
0
Choose move (4,5)
1.3 3 Aspiration!
2.6 6
Tabu list:
2 3 4 5 6 7
1 0 2 0 0 0 0
2 0 3 0 0 0
3 0 0 0 0
4 1 0 0
5 0 0
6 0
 Observations

• In the example 3 out of 21 moves are

prohibited
• More restrictive tabu effect can be
achieved by
 Increasing the tabu tenure
 Using stronger tabu-restrictions
• Using OR instead of AND for the 2 cities in a
move
 TSP Example: Frequency
Based Long Term Memory

• Typically used to diversify the search

• Can be activated after a period with no
improvement
• Often penalize attributes of moves that have
been selected often

Tabu-status (closeness in time)

1 2 3 4 5 6 7
1 2
2 3
3 3
4 1 5 1
5 4 4
6 1 2
7 4 3

Frequency of moves