Markov Chain-based Optimization Time Analysis

of Bivalent Ant Colony Optimization for Sorting

and LeadingOnes ∗
arXiv:2405.03353v1 [cs.NE] 6 May 2024

Matthias Kergaßner† Oliver Keszocze

[email protected] [email protected]
Rolf Wanka
[email protected]
Department of Computer Science, Friedrich-Alexander-Universität
Erlangen-Nürnberg, Germany

Abstract
So far, only few bounds on the runtime behavior of Ant Colony Opti-
mization (ACO) have been reported. To alleviate this situation, we inves-
tigate the ACO variant we call Bivalent ACO (BACO) that uses exactly
two pheromone values. We provide and successfully apply a new Markov
chain-based approach to calculate the expected optimization time, i. e.,
the expected number of iterations until the algorithm terminates. This
approach allows to derive exact formulæ for the expected optimization
time for the problems Sorting and LeadingOnes. It turns out that the
ratio of the two pheromone values significantly governs the runtime be-
havior of BACO. To the best of our knowledge, for the first time, we can
present tight bounds for Sorting (Θ(n3 )) with a specifically chosen objec-
tive function and prove the missing lower bound Ω(n2 ) for LeadingOnes
which, thus, is tightly bounded by Θ(n2 ). We show that despite we have
a drastically simplified ant algorithm with respect to the influence of the
pheromones on the solving process, known bounds on the expected opti-
mization time for the problems OneMax (O(n log n)) and LeadingOnes
(O(n2 )) can be re-produced as a by-product of our approach. Experi-
ments validate our theoretical findings.

Keywords: Ant colony optimization, optimization time analysis, Markov

chains
∗ This is the preprint of a full-paper accepted for presentation at the Genetic and Evolu-

tionary Computation Conference (GECCO), 2024.

† Corresponding author

1
1 Introduction
Ant algorithms are nature-inspired meta-heuristics that imitate colonies of ants
building up an efficient path between their nest and a source of food. They
quickly gained a lot of attention in the scientific community due to their ability
to be adapted to a large variety of problems in particular combinatorial opti-
mization problems. Initially the algorithms Ant System, Ant Colony System
and Ant Colony Optimization (ACO) were introduced [6, 9, 11] and further de-
veloped to the Min-Max-Ant System [20, 21]. Ant algorithms have been applied
to a variety of problems, e. g., their intuitive vocation of Shortest Paths [22] but
also the Traveling Salesperson Problem [8, 20], Minimum Spanning Tree [17],
Minimum Cut [13], Minimum Weight Vertex Cover [19] and Examination Time
Tabling [1]. Besides those combinatorial optimization problems, ACO has been
studied for the class of pseudo-boolean function, e. g., OneMax, LeadingOnes
and BinaryValue [12, 15, 4]. Although these problems are not considered real-
world applications, their analysis yields many insights into and understanding
of ACO. Furthermore they are typical benchmarks that allow comparisons of
the optimization time to other meta-heuristics such as Evolutionary Algorithms
(EA) and Particle Swarm Optimization (PSO). Optimization time, i. e., the
number of iterations of an algorithm, is a fundamental matter in the context
of algorithmic fields. Scharnow et al. [18] analyzed the expected optimization
time of an evolutionary algorithm for Sorting and single-source-shortest-paths
(SSSP). Doerr et al. [2] improved that result for SSSP further and achieved tight
bounds. Mühlenthaler et al. [14] proved bounds on the expected optimization
time of PSO applied to Sorting and OneMax. Neumann and Witt [16] proved
that the expected optimization time of ACO for any linear pseudo-boolean func-
tion is bounded by O(n log n). A wide and up-to-date overview on the topic of
evolutionary computation and recent developments is given in [3].
The aim of this work is to contribute to the successful analyses of ACO by
providing an approach that yields novel tight bounds on the expected optimiza-
tion time for Sorting (Θ(n3 )). Furthermore we demonstrate that our approach
produces tight bounds on the expected optimization time for LeadingOnes
(Θ(n2 )). So far, to the best of our knowledge, the lower bound for Leading-
Ones was an open question [5]. Overall we provide a powerful tool that will be
deployed to investigate further optimization time bounds of ACO.

2 Bivalent Ant Colony Optimization

Ant Colony Optimization is an algorithmic approach to solve black-box opti-
mization problems. The input of an ACO algorithm is a construction graph G
and an objective function f . The latter maps any path the ant1 walks on G to
a real number, representing the quality of the solution derived from this path.
Depending on the pheromone strategy, pheromones are spread along the edges
of that path to influence its probability to be incorporated into the next path.
1 we use a single ant; see discussion below

2
 Algorithm 1: BACO
Input : Construction graph G, objective function f
Parameter: Pheromone levels τmin and τmax
random walk policy P
Output : Best path π ∗ found
1 Initialize all pheromones with τmin
∗
2 The ant constructs an initial best path π by applying P
3 while termination criterion not fulfilled do
4 The ant constructs a path π on G by applying P
5 if f (π) > f (π ∗ ) then
6 π∗ ← π
7 Update pheromones
∗
8 return π

A high pheromone value increases the probability of an edge to be in the path

of the subsequent iteration. The counterpart is the evaporation of pheromones
over time. Overall, pheromones serve as memory where good solutions are en-
forced and weak ones decay over time. Thus finding the right balance between
the spreading and evaporation of pheromones is essential.
We present the Bivalent Ant Colony Optimization algorithm (BACO) in
the following (see Algorithm 1). The pheromones in BACO are bivalent and
can only take the values τmin and τmax . Let G be the construction graph and
f the objective function as described above. Initially, every egde of G has the
pheromone level τmin . BACO uses a single ant that walks across G according
to the random walk policy P. This policy prescribes on which node the ant
starts, when the ant should stop and can, e. g., prevent the ant from visiting a
node more than once (cf. tabu list). Such random walk rules highly depend on
the problem and the adaptation of ACO to it. The first random walk of the ant
yields the initial best path π ∗ . During every iteration the ant builds up a path π
on G that is evaluated via the objective function f . If π is better than π ∗ , then
π ∗ is replaced by π and pheromones are updated. Therefore every edge that is
used in path π ∗ obtains the pheromone value τmax while all the remaining edges
are assigned the τmin pheromone value. In this work, the termination criterion
is chosen such that BACO iterates until an optimum solution has been found.
As all problems investigated have a known optimal solution, this is a reasonable
criterion.
It is common in the analysis of ant algorithms to use a single ant only [16, 17].
Also in this work, having more than one ant would not yield any advantage due
to the following reasoning. If k ants walk across G they produce k paths per
iteration. In case there is a new best path among these, the ants will not
benefit from the corresponding spread of pheromones until the next iteration.
This results in an increased number of objective function evaluations. If actual
multi-threading is used, the runtime of the program will improve when k scales
with the number of CPU cores. However, this does not affect the computational

3
complexity that is calculated in this work since we investigate the number of
objective function evaluations. Thus in the following the number of evaluations
equals the number of iterations.
Another characteristic of BACO is the bivalent pheromones. This serves
two purposes at the same time. Bounding pheromones limits the evaporation
which fundamentally improves the runtime of the algorithm [20, 21, 7, 17].
If pheromones were unbounded, the pheromone levels on many edges decay
quickly such that they will probably never be chosen by the ant. Vice versa,
we precisely calculate such bounds to ensure that the ant can pick any path
with a reasonable probability even if numerous iterations have already passed.
In fact, the crucial parameter is the pheromone ratio t = τmin/τmax and not the
actual values of τmin and τmax . Doerr et al. [5] show that the right balance
of pheromone bounds is not only a tool to fine-tune ACO algorithms, but that
ACO is not robust with respect to the ratio t. It also turns out in this paper that
the parameter t is crucial for the runtime behavior of BACO. Therefore, t needs
to be determined carefully. The second purpose of bivalent pheromones is the
significant simplification of evaporation. As described in the pheromone update
mechanism above, pheromones evaporate from τmax to τmin immediately, if the
corresponding edge is not enforced in the update.
In the subsequent analysis, we analyze BACO precisely and demonstrate
that, despite its simplifications, it is competitive. We will present the decisive
role of the pheromone ratio t on the expected optimization time and use this
insight to identify optimal pheromone ratios.

3 Markov Chain-based Optimization Time Anal-

ysis of BACO
We quantify the progress of BACO using the objective function f . Let m
be the number of elements of the codomain of f and let the Markov states 0
through m − 1 be identified with the smallest up to the largest function value
respectively. Then the final state m − 1 corresponds to the optimum of f . The
current Markov state depends on the objective value of the current best solution.
Let p be the probability distribution vector of the first Markov state, i. e., p(i)
is the probability that the initial Markov state is i. Denote the probability to
reach state j from i within one iteration of the algorithm by Pr[i → j]. Then
the (m × m)-Markov matrix M is defined elementwise by M (i, j) = Pr[i → j].
Set Pr[m − 1 → j] = 0 for any value j since the algorithm terminates as soon as
state m−1 is reached. This facilitates the subsequent calculations. Furthermore
M is an upper triangular matrix, since BACO discards candidate solutions that
are worse than the current best, i. e., Pr[i → j] = 0 for j < i.
The following theorem yields a formula to calculate the expected optimiza-
tion time of BACO exactly. To make this work the Markov process has to have
another property, namely for every state i the probability to leave it has to be
positive. In other words, the probability Pr[i → i] to stay in a non-optimal state

4
i must not equal 1. Otherwise the process could stagnate prematurely in such
a sub-optimal state.
Theorem 3.1. Let p be the probability distribution vector of the initial Markov
state and M the Markov matrix. The expected optimization time T (n, t) of
BACO with the pheromone ratio t is given by

T (n, t) = p · (1 − M )−1 · (1 − M )−1 · M · (0, . . . , 0, 1)⊤ , (1)

where 1 denotes the unit matrix and the vector (0, . . . , 0, 1)⊤ consists of zeros
and a one as the last entry, both of fitting dimension.
Proof. The probability distribution vector p of the initial Markov state multi-
plied with the Markov matrix M raised to the power i, i ≥ 0 yields a vector
where the jth entry equals the probability to be in state j after i iterations. So
p · M i · (0, . . . , 0, 1)⊤ equals the chance to be in the final state m − 1 after exactly
i iterations. With that the expected optimization time is calculated as follows:
∞
X
i · p · M i · (0, . . . , 0, 1)⊤


T (n, t) =
i=0
∞
!
X
i
· (0, . . . , 0, 1)⊤


=p· i·M
i=0
= p · (1 − M )−1 · (1 − M )−1 · M · (0, . . . , 0, 1)⊤

Note that 1 − M is invertible since its eigenvalues are non-zero due to the
following reasoning. M is an upper triangular matrix and its diagonal entries
are in the right-open interval [0, 1[ as discussed above. Thus 1 − M is also
upper triangular and its diagonal entries are strictly positive and in the left-
open interval ]0, 1]. Consequently the eigenvalues of 1 − M are non-zero, so the
inverse exists. A detailed reasoning to the limit of the infinite sum of powers of
M can be found in Appendix A.

The computation of the inverse in Theorem 3.1 should be avoided as it

quickly becomes time consuming and numerically unstable. Therefore Eq. (1) is
substituted by an explicit formula in the following. For this, the Markov matrix
M needs to fulfill two properties:
• Each row of M sums up to 1 except for the last row. The sum of the last
row does not play a role in the subsequent proof.
• ∀i, j, 0 ≤ i < j ≤ m − 1, i + 1 < j:

M (i, j)
= ϕi (2)
M (i + 1, j)

This means that these ratios of values of M only depend on the row index
i and not on the column.

5
Let A = 1 − M for the remainder of this work. Note that the same values ϕi
apply to A by definition. Consider the following Theorem.
Theorem 3.2. The inverse matrix of A = 1 − M takes the form

1
 A(i,i) if i = j

A−1 (i, j) = δj if i < j ,

0 else


where δj , 1 ≤ j ≤ m − 1 is defined as

A(j − 1, j)
δj = − . (3)
A(j − 1, j − 1) · A(j, j)
Proof. Since M is an upper triangular matrix, A = 1−M has the same property.
By prerequisite the diagonal entries of M are not equal to 1. Thus the diagonal
of A is non-zero and the diagonal of A−1 consists of the corresponding inverse
values.
What is left to prove is A−1 (i, j) = δj for all i < j. First prove by mathe-
matical induction that A−1 (i, m − 1) equals 1 for all i. A−1 (m − 1, m − 1) = 1
follows directly from the definition of M . Now assume A−1 (k, m − 1) = 1 holds
true for all k > i and conclude the claim for A−1 (i, m − 1).
m−1
!
X
0 = A · A−1 (i, m − 1) = A(i, k) · A−1 (k, m − 1)

k=i
m−1
X
= A(i, i) · A−1 (i, m − 1) + A(i, k) · A−1 (k, m − 1)
| {z }
k=i+1 =1

The sum of each row of matrix M except the last equals 1. Thus all rows of
A except for the last one sum up to 0. With that the former calculation is
continued.
every row sum is 0
= −A(i,i)
z }| {
m−1
!
X
0 = A(i, i) · A−1 (i, m − 1) + A(i, k)
k=i+1

= A(i, i) · A−1 (i, m − 1) − A(i, i)

This implies that A−1 (i, m − 1) = 1 holds. By induction the last column of A−1
consists of ones only. Next, prove the off-diagonal entries.
!
0 = A · A−1 (j − 1, j)

= A(j − 1, j − 1) · A−1 (j − 1, j) + A(j − 1, j) · A−1 (j, j)

A(j − 1, j)
= A(j − 1, j − 1) · A−1 (j − 1, j) +
A(j, j)

6
Now solve this equation for A−1 (j − 1, j) to obtain the claim

A(j − 1, j)
A−1 (j − 1, j) = − = δj .
A(j − 1, j − 1)A(j, j)

Finally, apply Eq. (2) to prove the rest. Let i, j, 0 ≤ i, j ≤ m − 1 and i + 1 < j.
j
!
X
0 = A · A−1 (i, j) = A(i, k) · A−1 (k, j) = A(i, i) · A−1 (i, j)

k=i
j
X
+ A(i, i + 1) · A−1 (i + 1, j) + A(i, k) · A−1 (k, j)
| {z }
k=i+2 (2)
= ϕi ·A(i+1,k)
−1 −1
= A(i, i) · A (i, j) + A(i, i + 1) · A (i + 1, j)
j
X
+ ϕi · A(i + 1, k) · A−1 (k, j)
k=i+2
| {z }
(A·A−1 )(i+1,j)=0
= −A(i+1,i+1)·A−1 (i+1,j)
−1
= A(i, i) · A (i, j)
−1
+A (i + 1, j) · (A(i, i + 1) − ϕi · A(i + 1, i + 1))

Rearrange the equation to the following form:

A−1 (i, j) 1
= · (ϕi · A(i + 1, i + 1) − A(i, i + 1))
A−1 (i + 1, j) A(i, i)

The fraction on the left side of the equation is independent of the column index
j since j does not appear on the right side. Since the last column of A−1 consists
of ones only, the fraction itself must be equal to 1. Thus the fraction equals 1
for all the other columns respectively. Overall this implies that in column j of
A−1 all entries above the diagonal take the same value δj .
Now the exact values of A−1 are known and Eq. (1) can be simplified as
follows.

Theorem 3.3. Let p and M be defined as in Theorem 3.1. If each row of M

but the last sums up to 1 and the property in Eq. (2) is fulfilled, then the expected
optimization time T (n, t) of BACO can be calculated explicitly.
  
m−2 m−2
X
p(i) ·  1 X
T (n, t) = + δj  (4)
i=0
A(i, i) j=i+1

Proof. Start with Eq. (1) and use the identity M = 1 − A. Then exploit the

7
fact that each value in the last column of A−1 equals 1.

T (n, t) = p · A−1 · A−1 · (1 − A) · (0, . . . , 0, 1)⊤

= p · A−1 · (A−1 − 1) · (0, . . . , 0, 1)⊤ = p · A−1 · (1, . . . , 1, 0)⊤
| {z }
= last column of A−1 −1

Write out the multiplications to obtain the claim.

In the next sections, two problems are investigated where this formula ap-
plies, namely LeadingOnes and Sorting. The results are discussed in Sec-
tion 3.3.

3.1 LeadingOnes
The LeadingOnes function counts the number of consecutive ones from the
beginning of a bit string of length n. Thus the objective value is a number from
0 through n.
For pseudo-boolean objective functions such as LeadingOnes, a directed
construction graph G called chain is used [12, 5]. The starting node is connected
to two nodes which denote the first bit of x to be 0 or 1 respectively. These two
nodes are connected to a merge-node. From here the same structure repeats for
the second bit of x etc. This way an ant walking across G builds up a path that
is identified with a bit string.
We will see that the LeadingOnes problem fulfills the requirements for
Theorem 3.3 allowing us to calculate the expected optimization time of BACO.
We first calculate the probability distribution of the initial Markov state and
the Markov matrix for LeadingOnes and then derive the exact formula for the
expected optimization time as well as bounds on it.

Observation 3.4. Let m = n+1 be the number of Markov states. The first path
across G constructed by the ant yields a uniformly at random chosen candidate
solution for LeadingOnes. The probability distribution p for the initial Markov
state takes the following values:


 1 i+1 0 ≤ i ≤ m − 2
2
p(i) =
 1
n i=m−1
2

The probability to have the first i bits set to 1 initially equals 2−i . In case i
is less than n = m − 1, the probability 1/2 to have a subsequent 0 is multiplied
to this term.

Observation 3.5. Let m = n + 1 be the number of Markov states and t =

τmin/τmaxthe pheromone ratio. The Markov matrix M of dimension (m × m) is
defined elementwise by M (i, j) = Pr[i → j]. The exact values are listed using

8
the following partitioning of M :
 
(1) (2) ··· (2) (3)
.. .. .. .. 
0
 . . . . .
 . .. .. .. 
. .

 . (2) . 
 .. .. 
 . . (1) (3)
0 ··· ··· 0 (4)

1. 0 ≤ i ≤ m − 2:
 i  i+1
τmax τmin 1
M (i, i) = 1 − · =1−t·
τmax + τmin τmax + τmin 1+t

2. 0 ≤ i < j ≤ m − 2:
 i  j−i
τmax τmin 1
M (i, j) = · ·
τmax + τmin τmax + τmin 2
 i+1  j−i
1 1
=t· ·
1+t 2

3. 0 ≤ i ≤ m − 2:
 i  n−i−1
τmax τmin 1
M (i, m − 1) = · ·
τmax + τmin τmax + τmin 2
 i+1  n−i−1
1 1
=t· ·
1+t 2

4. M (m − 1, m − 1) = 0
In BACO, new candidate solutions that are worse than the current best solu-
tion are dismissed. Thus the lower triangle of M is 0. To calculate the remaining
entries the following recurring structure is used. An ant walking across G has
two options per bit. One of them is weighted with τmax pheromones, the other
one with τmin . Thus the probability to choose either of them is τmax/(τmax +τmin )
and τmin/(τmax +τmin ) respectively. Diagonal entry M (i, i), 0 ≤ i ≤ m − 2, equals
the probability to stay in state i. This value is calculated via the counter-
event where the current prefix of ones is extended by at least one bit. Entry
M (m − 1, m − 1) is set to 0 to represent the termination of the algorithm. For
an entry M (i, j) of the upper triangle we differentiate between two cases. If
j < m − 1 holds, then the ant needs to reproduce the path for the leading i bits
and afterwards pick the τmin options repeatedly until j bits in total are set to
1. The subsequent bit needs to be set to 0. However, if j = m − 1 holds, then
that final 0 does not exist.
Now the requirements for Theorem 3.3 are fulfilled (straightforward calcu-
lation omitted) and it can be applied.

9
Lemma 3.6. Let p and M as in Observations 3.4 and 3.5, respectively, and
let t > 0 be the pheromone ratio. The expected optimization time of BACO
applied to LeadingOnes is the following:
1+t
T (n, t) = · ((1 + t)n − 1) (5)
2t2
To obtain this equation first calculate δj = (1/2t) · (1 + t)j+1 , j ≤ m − 2, from
Eq. (3) using the identity A = 1−M and cases (1) and (2) from Observation 3.5.
Then use it for Theorem 3.3 and simplify the term using the exact formula for
partial geometric series. Explicit bounds on the expected optimization time of
BACO are deduced in the following theorem.
Theorem 3.7. Let t > 0 be the pheronome ratio of BACO applied to Leading-
Ones. Let furthermore c be a positive constant. The expected optimization time
T (n, t) is bounded as listed in Table 1.
Proof. The pheromone ratio t in Claim c1 is a positive constant c. Thus the
constant prefactor 1+c 2c2 in Eq. (5) is omitted. Consequently, the bound on T (n, t)
is Θ((1 + c)n ), i. e., T (n, t) is exponentially growing. Start with Eq. (5) to prove
Claim c2. Let t = ncs , c > 0 and s > 0.
c
(5) 1 + ns c n
 
T (n, t) = 2 · 1+ s −1
2 · ncs n


  c  n 
∈ Θ n2s · 1 + s −1
n
ns  nns
 !!
c
= Θ n2s · 1+ s −1
n
  n
   1−s

= Θ n2s · (ec ) ns − 1 = Θ n2s · ec·n −1

This proves Claim c2. For c > 0 and x ≥ 0 the claim

c
nx · e nx − 1 ∈ Θ(1)

can be proven using L’Hospital’s rule as follows:

c
c e nx − 1
lim nx · e nx − 1 = lim


n→∞ n→∞ n−x
c
−x−1
L’Ho. −c · x · n ·e x
n c
= lim −x−1
= lim c · e nx ∈ Θ(1)
n→∞ −x · n n→∞

Apply this to the last term of the preceding calculation to prove Claim c3 with
s ≥ 1:
  c   
T (n, t) ∈ Θ n2s · e ns−1 − 1 = Θ n2s−(s−1) = Θ ns+1

To prove Claim c4, let t ∈ O(1) the pheromone ratio. Then the prefactor 1 + t
in Eq. (5) is bounded by Θ(1). Thus the remainder of Eq. (5) yields the stated
bound.

10
Table 1: Expected optimization time of BACO applied to LeadingOnes de-
pending on the pheromone ratio t (Theorem 3.7, Claims c1 through c4).
t = t(n) T (n, t)
c1. c>0 Θ ((1 + c)n )
  1−s

c
c2. ns , 0 <s<1 Θ n2s · ec·n −1
c


c3. ns , s ≥1 Θ ns+1
Θ t12 · ((1 + t)n − 1)


c4. O(1), t > 0

We also investigated BACO applied to the pseudo-boolean OneMax func-

tion. From that we can confirm the known upper bound O(n · log n) in [12].
However, since this proof does not contribute new insights to the topic we omit
it in this work.

3.2 Sorting
In this section, we analyze the runtime behavior of BACO for sorting a list of
unique keys. In contrast to the previous problem, Sorting is not a direct opti-
mization problem and does not have a particular, obvious objective function. In
fact, there are many ways to measure the quality (“sortedness”) of a candidate
solution such as the length of a longest ascending subsequence or the number
of transpositions needed to sort the list of keys. Here, the Final Position Prefix
(FPP) objective is introduced and used. The FPP objective maps a list of n
keys to its number of leading keys that are in the same position as in the sorted
list. This yields a number from 0 to n, excluding n − 1 since this case is impos-
sible. The evaluation of FPP (inside the black box, not generating additional
cost for the optimization time) has linear complexity regarding the number of
comparisons of keys. In fact, it takes roughly up to n + log2 n comparisons when
the following approach is used. First identify the sorted prefix, i. e., the leading
keys which are sorted in increasing order. Then search for the minimum value
in the suffix. This minimum value determines which keys of the sorted prefix
are actually in their final position, namely every key that is smaller. This last
step is realized via binary search.
The construction graph G used for Sorting is a directed graph with n + 1
vertices: a starting node vstart and nodes v0 through vn−1 . There is an edge from
vstart to every other node. Nodes v0 through vn−1 induce a complete digraph.
Each node vi , 0 ≤ i ≤ n − 1, corresponds to one key. The ant starts at vstart
and builds up a Hamiltonian path visiting every node exactly once. The order
of nodes yields an order of the keys that is evaluated with respect to FPP.
Let the Markov states be the numbers from 0 through n−1. States 0 through
n − 2 are identified with the corresponding objective value of the current best
solution. The final state n − 1 denotes the optimal FPP value n.

11
Observation 3.8. Let m = n be the number of Markov states. The first path
across G constructed by the ant yields a uniformly at random chosen candi-
date solution for Sorting. The probability distribution vector p for the initial
Markov state takes the following values:
(
(n − i − 1) · (n−i−1)!
n! 0≤i≤m−2
p(i) = 1
n! i=m−1
The probability to initially have the first i keys in their final position is
1/n · 1/(n−1) · · · 1/(n−i+1).
If i is smaller than n, then the probability (n−i−1)/(n−i)
to have a subsequent key that is not in its final position is multiplied to this
term.
Observation 3.9. Let m = n be the number of Markov states. The Markov
matrix M takes the following values analogously to Observation 3.5:
i+1
Y 1
1. 0 ≤ i ≤ m − 2: M (i, i) = 1 − t ·
1 + (n − k)t
k=1

2. 0 ≤ i < j ≤ m − 2:
j
i+1
! !
Y 1 Y 1
M (i, j) = (n − j − 1) · t · ·
1 + (n − k)t n−k
k=1 k=i+1

3. 0 ≤ i ≤ m − 2:
i+1
! m−2
!
Y 1 Y 1
M (i, m − 1) = t · ·
1 + (n − k)t n−k
k=1 k=i+1

4. M (m − 1, m − 1) = 0
As mentioned in the Section 3.1, matrix M is of upper triangular shape
since BACO discards solution candidates that are worse than the current best.
Entries M (i, i) for 0 ≤ i ≤ m−2 are calculated using the counter-event in which
the objective value is increased by at least 1. Starting at node vstart , the ant
needs to take the τmax -marked path for i steps and then deviate to visit the
correct successor that increases the FPP objective value by one. Assuming the
ant has already visited k nodes including vstart and only τmax -edges have been
chosen, then the probability to again pick a τmax -edge is τmax/(τmax +(n−k)·τmin ) =
1/(1+(n−k)·t). The formulæ for the upper triangle start similarly, but the prefix

of the current best solution is extended from i to j keys. The pheromones do

not appear in this part of the formulæ since the suffix of a solution that is not
part of the final position prefix has no influence on the FPP objective and thus
all permutations of that suffix are equally likely. Analogously to LeadingOnes,
if j < n holds there needs to be a key after the final position prefix that is not
in its final position and if j equals n no such key exists.
Now the requirements for Theorem 3.3 are satisfied (straightforward calcu-
lation omitted) and it can be applied.

12
Lemma 3.10. Let p and M as in Observations 3.8 and 3.9, respectively, and
let t > 0 be the pheromone ratio. The expected optimization time of BACO
applied to Sorting is the following:
n−1 n−1
!
1 X Y
T (n, t) = · i · i! · (1 + r · t)
t · n! i=1 r=i
i−1 n−1
!!!
X k Y
+ · (1 + r · t) (6)
k+1
k=1 r=k

To obtain this equation first calculate

j+1
n−j−1 1 Y
δj = · · (1 + (n − k) · t)
n−j t
k=1

from Eq. (3) in the same way as for LeadingOnes. Then use it for Theorem 3.3
and simplify the term using index shifts.
We provide lower and upper bounds on the expected optimization time in
order to enable further classification of the underlying complexity.
Lemma 3.11. For any pheromone ratio t > 0 the expected optimization time
of BACO applied to Sorting is bounded as follows:
n−2 n
≤ T (n, t) ≤ · (1 + n · t)n (7)
2t t
Proof. The following identity can easily be proven for all n ∈ N by mathematical
induction:
n−1
X n−1
X
i2 · i! = (n − 1) · n! − i! (8)
i=1 i=1

To prove the lower bound on T (n, t) start with its definition:

n−1 n−1
!
(6) 1 X Y
T (n, t) = · i · i! · (1 + r · t)
t · n! i=1
| {z }
r=i
≥1
i−1 n−1
!!!
X k Y
+ · (1 + r · t)
k+1 | {z }
k=1 | {z } r=k ≥1
≥ 12
n−1 i−1
!! n−1
1 X 1 X 1 X
i2 · i!


≥ · i · i! · 1+ · 1 ≥ ·
t · n! i=1 2 2t · n! i=1
k=1
n−1
! n−1
!
(8) 1 X 1 X i!
= · (n − 1) · n! − i! = · n−1−
2t · n! i=1
2t i=1
n!

13
 n−2
≥
2t
The upper bound is derived as follows:
n−1 n−1
!
(6) 1 X Y
T (n, t) = · i · i! · (1 + r · t)
t · n! i=1 r=i
i−1 n−1
!!!
X k Y
+ · (1 + r · t)
k+1
k=1 r=k
n−1 i−1
!!
1 X
n
X
n
≤ · i · i! · (1 + n · t) + (1 + n · t)
t · n! i=1 k=1
n−1
!
1 X
= · i2 · i! · (1 + n · t)n
t · n! i=1
n−1
!
(8) (1 + n · t)n X n
= · (n − 1) · n! − i! ≤ · (1 + n · t)n
t · n! i=1
t

Using these bounds on the expected optimization time we further clas-

sify the underlying complexity of BACO for Sorting with respect to various
pheromone ratios t.
Theorem 3.12. Let t be the pheronome ratio of BACO applied to Sorting.
The expected optimization time T (n, t) is bounded as listed in Table 2.
Proof. Use Lemma 3.10 to prove Claim c5:
n−1 n−1
!
(6) 1 X Y
T (n, t) = · i · i! · (1 + r · t)
t · n! i=1 r=i
i−1 n−1
!!!
X k Y
+ · (1 + r · t)
k+1
k=1 r=k
n−1 n−1
! i−1 n−1
!!
1 X X X X
≥ · i · i! · r·t + r·t
2 · t · n! i=1 r=i k=1 r=k
n−1 i n−1
!
1 X X X
= · i · i! · r
2 · n! i=1
k=1 r=k

Calculate

the double sum explicitly and apply Eq. (8) to prove that T (n, t) ∈
Ω n3
holds. Next prove Claim c6. Based on Lemma 3.11, the lower bound
Ω nt is obvious while the upper bound is derived as follows. Let t ∈ O n12 .

Then n2 · t ∈ O(1) and exp(n2 · t) ∈ O(1) hold. Use this in the following
calculation.
n
n2 · t
(7) n

n n 2
n
T (n, t) ≤ · (1 + n · t)n = · 1 + ≤ · en ·t ∈ O
t t n t t

14
Claim c7 is a special case of c6. To prove Claim c8, use the upper bound in
Lemma 3.11 and set t = c/ns , c > 0, s < 2 for the following calculation:
n
ns+1 c · n2−s
(7) n

 c n
T (n, t) ≤ c · 1 + n · s = · 1+
ns n c n
s+1
n 2−s
 2−s

≤ · ec·n ∈ O ns+1 · ec·n
c
In Claim c9, t is set to 1 which results in a blind search. For the proof, the
following identity is utilized:
n−1
X
∀n ∈ N : (i · i!) = n! − 1 (9)
i=1

With that the lower bound is calculated.

n−1 n−1
!
(6)1 X Y
T (n, t) = · i · i! · (1 + r)
n! i=1 r=i
i−1 n−1
!!!
X k Y
+ · (1 + r)
k+1
k=1 r=k
n−1 i−1
!!!
1 X n! X k n!
= · i · i! · + ·
n! i=1
i! k + 1 k!
k=1
n−1 i
! n−1 i
!
1 X 1 X n! 1 X X 1
≥ · i · i! · · = · i · i! ·
n! i=1 2 k! 2 i=1 k!
k=1 k=1
n−1
1 X (9) 1
≥ · (i · i!) = · (n! − 1) ∈ Ω(n!)
2 i=1
2

The proof of the upper bound starts analogously (left out, cf. above):
∞
n−1 i
! n−1 !
(6) 1 X X n! X X 1
T (n, t) = · · · ≤ · i · i! · ≤ i · i! ·
n! i=1 k! i=1
k!
k=1 k=1
(9)
= (n! − 1) · (e − 1) ∈ O(n!)

3.3 Discussion of the Results

In this section the results of the preceding analyses of BACO are discussed.
Table 3 presents the expected optimization time bounds for BACO, the (1+1)-
EA from [10] and OnePSO from [14] applied to LeadingOnes and Sorting
(no results are available for OnePSO applied to LeadingOnes).
In Section 3.1 we provide an exact formula for the expected optimization
time of BACO for LeadingOnes (Eq. (5)). For the pheromone ratio t = c/ns

15
Table 2: Expected optimization time of BACO applied to Sorting depending
on the pheromone ratio t (Theorem 3.12, Claims c5 through c9).
t = t(n) T (n, t)


c5. t(n) >
0 Ω n3

c6. O n12 Θ nt
c


c7. ns , s ≥ 2 Θ ns+1 
2−s
c
c8. ns , s <2 O ns+1 · ec·n
c9. 1 Θ (n!)

Table 3: Expected optimization time bounds of BACO, (1+1)-EA [10] and

OnePSO [14] applied to LeadingOnes and Sorting.
LeadingOnes Sorting



BACO Θ n2 Θ n3



(1+1)-EA Θ n2 Θ n2 · log n



OnePSO – Ω n2 , O n2 · log n

with s ≥ 1 we prove that this formula is tightly bounded by Θ(ns+1 ). If s is

in the interval [0, 1[ then this bound becomes superpolynomial. Thus, when s
is chosen smaller than 1 then there is a phase transition from polynomial to
superpolynomial complexity. Overall t = c/n is optimal with respect to the ex-
pected optimization time Θ(n2 ). Note that the upper bound O(n2 ) has already
been shown in [4] but that so far no lower bound was known. Additionally, we
prove that for any t ∈ O(1) the expected optimization time is tightly bounded
by Θ t12 · ((1 + t)n − 1) .

Next we utilize our approach to obtain an exact formula for the expected
optimization time T (n, t) of BACO
for Sorting (Eq. (6)). We prove that
1
for any pheromone ratio t ∈ O n2 the expected optimization time is tightly
bounded by Θ nt . In particular, if t = c/ns and s ≥ 2, then T (n, t) is tightly

bounded by Θ(ns+1 ). Together with the lower bound Ω(n3 ) for any (positive)
pheromone ratio t = t(n) it follows that t = c/n2 is optimal with respect to
the expected optimization time which, consequently, is bounded by Θ(n3 ). For
2−s

t = c/ns and s < 2 we prove the upper bound O ns+1 · ec·n . We assume that
for s = 2 the analog phase transition as for LeadingOnes from polynomial to
superpolynomial complexity occurs. However, this remains an open question.
If t is set to 1, then τmin equals τmax and thus no information is gained from the
pheromones. This leads to a blind search and an expected number of iterations
bounded by Θ(n!).
Note that standard application of Markov’s inequality leads to the observa-
tion that the probability that BACO is successful in α · T (n, t) optimization
steps is at least 1 − 1/α.

16
 ·104

(expected) optimization time 5 experiments

T n, n1


4

0 20 40 60 80 100 120 140 160 180 200

problem size n

Figure 1: Optimization time for LeadingOnes and the analytically calculated

expected optimization time using the pheromone ratio t = n1 .

Overall BACO has the same expected optimization time for LeadingOnes
as the (1+1)-EA, assuming both algorithms use their corresponding optimal
parameter settings. In the domain of Sorting however, BACO is less efficient
than the (1+1)-EA and OnePSO by a factor of n/log n. Further research might
lead to a competitive optimization time of BACO for Sorting.

4 Experiments
In this section the theoretical results are validated. For this, an implementa-
tion of BACO written in C++ is used to solve LeadingOnes and Sorting
for a range of problem sizes n. Figure 1 visualizes the optimization time of
the implementation applied to LeadingOnes. We ran 20 repetitions for each
problem size n from 5 up to 200. The analytically calculated expected optimiza-
tion time T (n, t) for LeadingOnes from Lemma 3.6 is plotted in red. Both,
the implementation and the exact formula used the optimal pheromone ratio
t = 1/n. Analogously, Figure 2 shows the optimization times for Sorting. We
ran 40 repetitions for each problem size n from 5 up to 100. The analytically
calculated expected optimization time T (n, t) for Sorting from Lemma 3.10 is
plotted in red. For this experiment the implementation and the exact formula
used the optimal pheromone ratio t = 1/n2 . As can clearly be seen in both plots,
the experimental results fit the theoretical findings.

17
 ·106
(expected) optimization time experiments
T n, n12


1.5

0.5

0 10 20 30 40 50 60 70 80 90 100
problem size n

Figure 2: Optimization time for Sorting and the analytically calculated ex-
pected optimization time using the pheromone ratio t = n12 .

5 Conclusion & Outlook

In this work, the ACO variant Bivalent Ant Colony Optimization (BACO) was
investigated and analyzed based on the problems Sorting and LeadingOnes.
The main characteristic of BACO is the usage of bivalent pheromones, i. e.,
pheromones can only take the values τmin or τmax . We provided a Markov
chain-based approach that allows to deduce an exact formula for the expected
optimization time of BACO and demonstrate it using both problems. The
influence of the pheromone ratio t = τmin/τmax is presented precisely. With the
help of these formulæ we prove the expected optimization time complexities
Θ(n3 ) for Sorting and Θ(n2 ) for LeadingOnes that are achieved using the
optimal values for t, respectively. To the best of our knowledge, bounds for
Sorting and the lower bound for LeadingOnes have been unknown so far.
In the domain of Sorting BACO is still outperformed by the (1+1)-EA and
OnePSO by a factor of n/log n. However, our analysis together with a different
objective function is possibly the right approach to achieve a draw between
those three algorithms. Additionally, the missing lower bound for Leading-
Ones yields the identical complexity Θ(n2 ) as the (1+1)-EA. Experiments with
an implementation of the algorithm validated the results proven by theoretical
analysis.

18
A Infinite sum of matrix powers
Let M be the Markov matrix from Section 3. Then M is an upper triangular
matrix and all entries on the diagonal are strictly smaller than 1 and non-
negative. This implies that (1 − M ) is also an upper triangular matrix and
every diagonal entry is greater than 0. Thus the inverse matrix of (1 − M )
exists and the following identity is well-defined and will be proven afterwards:
∞
X
i · M i = (1 − M )−1 · (1 − M )−1 · M
i=0

Proof.
∞
X ∞

X
(1 − M ) · (1 − M ) · i · Mi = 1 − 2 · M + M2 · i · Mi
i=0 i=1
∞ ∞ ∞
! ! !
X X X
i i+1 i+2
= i·M − 2i · M + i·M
i=1 i=1 i=1
∞ ∞ ∞
! ! !
X X X
i i
= i·M − 2(i − 1) · M + (i − 2) · M i
i=1 i=2 i=3
∞
X
=M+ (i − 2(i − 1) + (i − 2)) · M i = M
i=3

Multiplication by (1 − M )−1 · (1 − M )−1 from the left yields the statement.

Furthermore the infinite sum of powers of M can be reordered since it converges
as the eigenvalues of M are between 0 and 1.

19
References
[1] Abounacer, R., Boukachour, J., Dkhissi, B., and El Hi-
lali Alaoui, A. A hybrid Ant Colony Algorithm for the exam timetabling
problem. Revue Africaine de Recherche en Informatique et Mathématiques
Appliquées 12 (2010), 15–42.
[2] Doerr, B., Happ, E., and Klein, C. A tight analysis of the (1 + 1)-ea
for the single source shortest path problem. In 2007 IEEE Congress on
Evolutionary Computation (2007), pp. 1890–1895.
[3] Doerr, B., and Neumann, F., Eds. Theory of Evolutionary
Computation—Recent Developments in Discrete Optimization. Springer,
2020. Also available at http://www.lix.polytechnique.fr/Labo/
Benjamin.Doerr/doerr_neumann_book.html.
[4] Doerr, B., Neumann, F., Sudholt, D., and Witt, C. On the runtime
analysis of the 1-ant aco algorithm. In Proceedings of the 9th Annual Con-
ference on Genetic and Evolutionary Computation (New York, NY, USA,
2007), GECCO ’07, Association for Computing Machinery, pp. 33–40.
[5] Doerr, B., Neumann, F., Sudholt, D., and Witt, C. Runtime anal-
ysis of the 1-ant ant colony optimizer. Theoretical Computer Science 412,
17 (2011), 1629–1644.
[6] Dorigo, M. Optimization, Learning and Natural Algorithms. PhD thesis,
Dipartimento di Elettronica, Politecnico di Milano, Italy, 1992. In Italian.
[7] Dorigo, M., and Blum, C. Ant colony optimization theory: A survey.
Theoretical Computer Science 344, 2 (2005), 243–278.
[8] Dorigo, M., and Di Caro, G. Ant colony optimization: a new meta-
heuristic. In Proc 1999 Congress on Evolutionary Computation (CEC)
(Washington, DC, USA, 1999), vol. 2, IEEE, pp. 1470–1477.
[9] Dorigo, M., Maniezzo, V., and Colorni, A. Positive feedback as a
search strategy. Tech rep., 91-016, Dip Elettronica, Politecnico di Milano,
Italy (04 1999).
[10] Droste, S., Jansen, T., and Wegener, I. On the analysis of the (1+1)
evolutionary algorithm. Theoretical Computer Science 276, 1 (2002), 51–81.
[11] Gambardella, L., and Dorigo, M. Solving symmetric and asymmetric
tsps by ant colonies. In Proceedings of IEEE International Conference on
Evolutionary Computation (1996), pp. 622–627.
[12] Gutjahr, W. J. First steps to the runtime complexity analysis of ant
colony optimization. Computers & Operations Research 35, 9 (2008), 2711–
2727. Part of Special Issue: Bio-inspired Methods in Combinatorial Opti-
mization.

20
[13] Kötzing, T., Lehre, P. K., Neumann, F., and Oliveto, P. S. Ant
colony optimization and the minimum cut problem. In Proceedings of the
12th Annual Conference on Genetic and Evolutionary Computation (New
York, NY, USA, 2010), GECCO ’10, Association for Computing Machinery,
pp. 1393–1400.

[14] Mühlenthaler, M., Raß, A., Schmitt, M., and Wanka, R. Ex-
act Markov chain-based runtime analysis of a discrete particle swarm opti-
mization algorithm on sorting and OneMax. Natural Computing 21 (2021),
651–677.
[15] Neumann, F., and Witt, C. Runtime analysis of a simple ant colony op-
timization algorithm. In Proceedings of the 17th International Conference
on Algorithms and Computation (Berlin, Heidelberg, 2006), ISAAC’06,
Springer-Verlag, pp. 618–627.
[16] Neumann, F., and Witt, C. Runtime analysis of a simple ant colony
optimization algorithm. Algorithmica 54, 2 (2009), 243–255.

[17] Neumann, F., and Witt, C. Ant colony optimization and the minimum
spanning tree problem. Theoretical Computer Science 411, 25 (2010), 2406–
2413.
[18] Scharnow, J., Tinnefeld, K., and Wegener, I. The analysis of
evolutionary algorithms on sorting and shortest paths problems. J. Math.
Model. Algorithms 3, 4 (2004), 349–366.
[19] Shyu, S. J., Yin, P.-Y., and Lin, B. M. An Ant Colony Optimiza-
tion algorithm for the minimum weight vertex cover problem. Annals of
Operations Research 13, 1 (2004), 283–304.

[20] Stützle, T., and Hoos, H. Max-min ant system and local search for
the traveling salesman problem. In Proceedings of 1997 IEEE international
conference on evolutionary computation (ICEC’97) (1997), IEEE, pp. 309–
314.
[21] Stützle, T., and Hoos, H. H. Max-min ant system. Future Generation
Computer Systems 16, 8 (2000), 889–914.
[22] Sudholt, D., and Thyssen, C. Running time analysis of Ant Colony
Optimization for shortest path problems. Journal of Discrete Algorithms
10 (2012), 165–180.

21