MIC’2001 - 4th Metaheuristics International Conference 399

HC-ACO: The Hyper-Cube Framework for Ant

Colony Optimization
Christian Blum∗ Andrea Roli∗ Marco Dorigo∗
∗
IRIDIA - Université Libre de Bruxelles
Av. Roosvelt 50 - Bruxelles (Belgium)
Email: {cblum, aroli, mdorigo}@ulb.ac.be

1 Introduction

Ant Colony Optimization (ACO) [2] is a recently proposed metaheuristic approach for solving hard
combinatorial optimization problems. The inspiring source of ACO is the foraging behavior of real
ants. In most ACO implementations the hyperspace for the pheromone values used by the ants to
build solutions is only implicitly limited. In this paper we propose a new way of implementing ACO
algorithms, which explicitly defines the hyperspace for the pheromone values as the convex hull of
the set of 0-1 coded feasible solutions of the combinatorial optimization problem under consideration.
We call this new implementation the hyper-cube framework for ACO algorithms. The organization
of this extended abstract is as follows. In section 2 we briefly present the original Ant System [3] for
static combinatorial optimization problems. In section 3 we propose the hyper-cube framework for
ACO algorithms and we present pheromone updating rules for Ant System (AS) and MAX -MIN
Ant System (MMAS). In section 4 we discuss some of the advantages involved with the hyper-cube
framework for ACO algorithms, while Section 5 outlines future work.

2 ACO algorithms

Let O = {o1 , ..., on } be a finite set of objects. We consider the problem of choosing an optimal subset
(ordered or unordered; according to the constraints of the problem) of cardinality 0 ≤ l ≤ n from the
finite set O. This cardinality can be fixed or variable. All subsets s fulfilling the constraints of the
problem are called feasible solutions of the problem. The set of all feasible solutions is denoted by S.
The quality of a solution s is evaluated by an objective function f : S → IR and is denoted by f (s). We
will deal with minimization problems. Therefore the set of optimal solutions Ŝ is a subset of S with
f (si ) < f (sj ) ∀si ∈ Ŝ, sj ∈ S \ Ŝ.

In ACO algorithms artificial ants construct a solution by building a path on a construction graph
G = (C, L) where each element from the set C of solution components corresponds to an object in the
set O and the elements of L (called connections) fully connect the elements of C. To each solution
component oi we have associated a desirability value ηi (which is called heuristic information) and a
pheromone value τi . In each iteration of the algorithm, each ant (from a set of k ants) generates a
feasible solution according to probabilities p(or |s[ol ]) where s[ol ] is a partial solution with element ol
as last added element. In a particular implementation of ACO algorithms, known as Ant System (AS),
these probabilities are determined by the following state transition rule:

p(or |s[ol ]) =
 [ηr ]α [τr ]β
α
ou ∈J(s[ol ]) [ηu ] [τu ]
β if or ∈ J(s[ol ])
(1)
0 otherwise

Porto, Portugal, July 16-20, 2001

400 MIC’2001 - 4th Metaheuristics International Conference

In this formula α and β are parameters to adjust the relative importance of heuristic information and
pheromone values and J(s[ol ]) denotes the set of solution components which are allowed to be added
to the partial solution s[ol ]. Once all ants have constructed a solution the online delayed pheromone
update rule is applied. This pheromone update rule for ant Ai (i = 1, ..., k) with generated solution si
(i = 1, ..., k) consisting of solution components ol ∈ O is as follows:

k  1
if oj ∈ si
τj ← ρ · τj + ∆τji where ∆τji = f (si ) (2)
0 otherwise
i=1

where f (si ) is the quality of solution si and 0 < ρ < 1 is a pheromone decay parameter. This pheromone
update rule leads to an increase of pheromone on solution components which have been found in better
quality solutions than other solution components (where the pheromone value will decrease). Although
AS is important, because it was the first ACO algorithm proposed, in the last few years some changes
and extensions of AS have been proposed, e.g. MAX -MIN Ant System (MMAS) [6]. In general,
ACO algorithms are proven to be a very effective – for some problems like the QAP even the state-of-
the-art – metaheuristic method for combinatorial optimization problem solving.

3 HC-ACO

In the AS framework described in the previous section, the pheromone values associated to the solution
components are used to probabilistically construct solutions to the optimization problem under consid-
eration. If we regard the set of pheromone values as a vector τ = (τ1 , ..., τn ), this vector is moving in a
hyperspace with different limits for different pheromone updating rules. We will denote this hyperspace
in the following with T . For AS, the pheromone values τi are limited by
1 k
lim τi (t) ≤ · (3)
t→∞ 1 − ρ f (sopt )

where sopt ∈ Ŝ. It is clear that the limits of T can be very different depending on the pheromone
updating rule itself (e.g., AS versus AS using elitist strategies [3]) and the amount of pheromone added
in each step, which is a function of the solution quality. The hyper-cube framework described in the
following will give a well-defined description of the hyperspace T .

3.1 The hyper-cube framework

For most combinatorial optimization problems a mathematical programming formulation exists where
solutions are modeled as binary vectors (0-1 Integer Programming). In this formulation we have a set
of decision variables, corresponding to the solution components, which can assume values {0, 1}. A 1
is indicating that the corresponding solution component is a member of the solution defined by the
vector. In this kind of problem modeling the set of feasible solutions S is a subset of the set of corners
of the n-dimensional hyper-cube (in case of n solution components). If we relax the {0, 1}-constraints,
the extended set of feasible solutions S̃ is the set of all vectors v ∈ IRn which are expressible as convex
combinations of binary vectors v i ∈ S (see figure 1a for an example):

v ∈ S̃ ⇔ v = αi v i , αi ∈ [0, 1] (4)
vi ∈S

As in the 0-1 Integer Programming (IP) formulation for combinatorial optimization problems, in the
hyper-cube framework for ACO algorithms we regard the solutions s = (s1 , ..., sn ) for our problem as
binary vectors of length n. Again each position is indicating the presence or absence of the corresponding
solution component in the solution described by this vector. Additionally the vector of pheromone values
τ = (τ1 , ..., τn ) will be moving in S̃ and can therefore be seen as a probability distribution over the
solution components (in τ , τi is associated to solution component oi , 1 ≤ i ≤ n).

Porto, Portugal, July 16-20, 2001

MIC’2001 - 4th Metaheuristics International Conference 401

Solution of ant 2
(0,1,1) (0,1,1)
(1,1,1) (1,1,1)

(0,0,1) (0,0,1)
(1,0,1) (1,0,1)

τ
(1,1,0) d (1,1,0)

Solution of ant 1
(0,0,0) (1,0,0) (0,0,0) (1,0,0)
a) b)

Figure 1: The set S of feasible solutions consists of the three vectors (0, 0, 0), (1, 1, 0) and (0, 1, 1). The
gray shaded area is the set S̃. In b), two solutions have been created by two ants. d is the weighted
average of these two solutions ((0, 0, 0) is of higher quality) and τ will be shifted towards d

3.2 Pheromone updating rules for the hyper-cube framework

In the following we rewrite the pheromone updating rules for Ant System (respectively MMAS) for
the hyper-cube framework and we will give a new interpretation. Ant System’s new updating rule is
obtained via a normalization of equation (2):

 
1

k f (si )
if oj ∈ si
τj ← ρ · τj + (1 − ρ) · ∆τji where ∆τji = k 1
l=1 f (sl ) (5)
 0 otherwise
i=1

In MMAS in every iteration just one solution (called supd in the following) updates the pheromone
values so that the left part of equation (5) becomes τj ← ρ · τj + (1 − ρ) · ∆τjupd . Solution supd can be
the iteration best solution generated by ants or the global best solution generated by the ants since the
start of the algorithm. In MMAS the right part of equation (5) therefore is reduced to ∆τjupd = 1 if
oj ∈ supd and 0 otherwise. The updating rule given in equation (5) can be re-interpreted as a shift of
the probability distribution within the hyperspace T towards the probability distribution given by the
weighted average of solutions. The following theorem gives a formal description of that:

Theorem 1 The pheromone updating rule in the hyper-cube framework given by equation (5) is equiv-
alent to

τ ←− τ + µ · (d − τ ) (6)

k
f (si ) · sj
1 i
d = (d1 , ..., dn ) where dj = k 1
, j = 1, ..., n (7)
i=1 l=1 f (sl )

and si ∈ S upd , the set of solutions used for updating. 0 < µ < 1 is a parameter called learning rate.

Vector d in Theorem 1 can be interpreted as a weighted average of the set of solutions used for updating.
So, the pheromone updating rule for Ant System in the hyper-cube framework computes a probability
distribution of the solution components present in the set of solutions generated by the ants. Then
the old probability distribution τ which was used by the ants to generate the solutions gets shifted by
a factor 0 < µ < 1 in the direction of the new probability distribution d. This fact – as we will see
later – opens the door to comparisons with genetic algorithm techniques. An example for Ant System’s
updating rule is given in figure 1b. It is also important to mention that, if the starting values for the
pheromone values describe a vector τ ∈ T (the convex hull of all feasible solutions), then by applying
these pheromone updating rules τ will always stay in T .

Porto, Portugal, July 16-20, 2001

402 MIC’2001 - 4th Metaheuristics International Conference

4 Benefits of the hyper-cube framework

The most important aspect involved with the introduction of the hyper-cube framework for ACO
algorithms is the fact that this framework favours a mathematical examination of ACO algorithms. This
aspect is essential for detailed comparisons with Evolutionary Computation algorithms. Also, the fact
that τ ∈ T for the duration of the algorithm can be used for different techniques (i.e., diversification).

A diversification scheme for ACO algorithms: In the following we define the concepts of global
desirability and global frequency for solution components oj . We denote the set of all solutions generated
by the ants since the start of the algorithm by Sants . The global desirability of solution components oj
(j = 1, ..., n) is given by equation (8a) and the global frequency by equation (8b):

1 
(a) vjdes ←− max{ : s ∈ Sants , sj = 1} (b) vjf r ←− sj (8)
f (s)
s∈Sants

and v f r , to re-position the vector of pheromone values

In the following we will use these vectors, v des
τ in T . We use b1 ants to construct solutions according to the global desirability values and b2 ants to
construct solutions according to the global frequencies using the transition rule:


 pr
if or ∈ J(s[ol ])
p(or |s[ol ]) = ou ∈J(s[ol ]) pu
(9)
0 otherwise

1
where pi = vif r
(respectively pi = vudes ) for i = 1, ..., n, and b1 and b2 are parameters. Then we produce
the new vector τ of pheromone values using these b1 + b2 solutions in the same way as we compute
vector d in the pheromone updating rule of Theorem 1. We compute it as a weighted average of
these solutions. This mechanism gives us the possibility to restart ACO algorithms in an intelligently
biased way whenever it reaches stagnation. The usage of the global desirability ensures a quite high
pheromone value for solution elements which have been found in good quality solutions in the past (→
exploitation), whereas the usage of global frequencies guides the search to areas in the search space
which have not been explored so far (→ exploration). Preliminary results for the QAP suggest the
usefulness of the above diversification scheme.

Comparisons with Evolutionary Computation: The new hyper-cube framework simplifies the
examination of similarities and differences between ACO algorithms and Evolutionary Computation.
In the field of Evolutionary Computation in the last decade some interesting developments quite similar
to ACO algorithms have been proposed. They have in common that they use a probabilistic mech-
anism for recombination of individuals. This leads to algorithms where the population statistics are
kept in probability vectors instead of the population itself as done in usual Genetic Algorithms. In
each iteration of the algorithm these probabilities are used to generate new solutions. The new so-
lutions are then used to adapt the probability vector (or even to replace it by the vector of solution
component probabilities given by the new set of solutions). The first approach of such a kind was
given by the work of Syswerda [7], who replaced the usual two parent recombination (TPR) operator
by an operator called Bit-Simulated Crossover (BSC). Another approach called Population-Based In-
cremental Learning (PBIL) has been proposed by Baluja et al. [1]. Harik et al. [4] extended the idea
of PBIL and proposed an algorithm called Compact Genetic Algorithm (cGA). Mühlenbein and Voigt
[5] finally presented a new form of recombination called gene pool recombination (GPR). Later they
generalized their idea of GPR in proposing a class of algorithms called Univariate Marginal Distribution
Algorithms (UMDA). The new hyper-cube framework clarifies the relation of ACO algorithms to these
algorithms and highlights the great advantage of ACO algorithms as the way of using the distribution of
solution component probabilities for solution construction. This provides a structured way of handling
constrained problems which the other approaches are lacking.

Porto, Portugal, July 16-20, 2001

MIC’2001 - 4th Metaheuristics International Conference 403

5 Conclusions

We have presented the hyper-cube framework for ACO algorithms, which provides a well defined hy-
perspace T for the pheromone values. Its potential usefulness is indicated by the discussion in section
4. The subject of our current work involves testing the diversification scheme presented in section 4
and doing comparative studies of ACO algorithms and Evolutionary Computation techniques in greater
detail. We will experimentally investigate the influence of the new implementation for MMAS, as the
hyper-cube framework changes the way of setting values τmin and τmax . In usual MMAS implementa-
tions, τmin and τmax are dependent on the upper limit for pheromone values, which gets approximated
during the run of the algorithm. This leads to a repeated resetting of these values, which could poten-
tially disturb the dynamics of the system. In the hyper-cube framework this is not the case as we know
the limits for pheromone values beforehand. We also intend to investigate the potential usefulness of
the hyper-cube framework for combinatorial optimization problems with linear objective functions. In
this case the derivation of the objective function could be a basis for a decision about the usefulness of
a solution for updating the pheromones. This could also be the start of merging ACO algorithms with
techniques for cutting off certain areas of the search space (reducing the search space).

Acknowledgements: This work was supported by the “Metaheuristics Network”, a Research Training Network
funded by the Improving Human Potential programme of the CEC, grant HPRN-CT-1999-00106. Andrea
Roli acknowledges support from the CEC through a “Marie Curie Training Site” (contract HPMT-CT-2000-
00032) fellowship. The information provided is the sole responsibility of the authors and does not reflect the
Community’s opinion. The Community is not responsible for any use that might be made of data appearing in
this publication. Marco Dorigo acknowledges support from the Belgian FNRS, of which he is a Senior Research
Associate.

References
[1] S. Baluja and R. Caruana. Removing the Genetics from the Standard Genetic Algorithm. In A. Prieditis
and S. Russel, editors, The International Conference on Machine Learning 1995, pages 38–46, San Mateo,
California, 1995. Morgan Kaufmann Publishers.
[2] M. Dorigo and G. Di Caro. The Ant Colony Optimization meta-heuristic. In D. Corne, M. Dorigo, and
F. Glover, editors, New Ideas in Optimization, pages 11–32. McGraw-Hill, 1999.
[3] M. Dorigo, V. Maniezzo, and A. Colorni. Ant System: Optimization by a colony of cooperating agents.
IEEE Transactions on Systems, Man and Cybernetics - Part B, 26(1):29–41, 1996.
[4] G. R. Harik, F. G. Lobo, and D. E. Goldberg. The Compact Genetic Algorithm. IEEE Transactions on
Evolutionary Computation, 3(4):287–297, 1999.
[5] H. Mühlenbein and H.-M. Voigt. Gene Pool Recombination in Genetic Algorithms. In I.H. Osman and J.P.
Kelly, editors, Proc. of the Metaheuristics Conference, Norwell, USA, 1995. Kluwer Academic Publishers.
[6] T. Stützle and H. H. Hoos. MAX -MIN Ant System. Future Generation Computer Systems, 16(8):889–914,
2000.
[7] G. Syswerda. Simulated Crossover in Genetic Algorithms. In L.D. Whitley, editor, Proc. of the second
workshop on Foundations of Genetic Algorithms, pages 239–255, San Mateo, California, 1993. Morgan
Kaufmann Publishers.

Porto, Portugal, July 16-20, 2001