IEEE TRANSACTIONS ON CYBERNETICS, VOL. 43, NO.

1, FEBRUARY 2013 347

Ant Colony Optimization With Combining Gaussian

Eliminations for Matrix Multiplication
Yuren Zhou, Xinsheng Lai, Yuanxiang Li, and Wenyong Dong

Abstract—One of the main unsolved problems in computer We partition two input matrices X and Y and their product
algebra is to determine the minimal number of multiplications Z = XY into quadrants as follows:
which is necessary to compute the product of two matrices. For     
practical value, the small format is of special interest. This leads Z11 Z12 X11 X12 Y11 Y12
to a combinatorial optimization problem which is unlikely solved =
Z21 Z22 X21 X22 Y21 Y22
in polynomial time. In this paper, we present a method called com-
bining Gaussian eliminations to reduce the number of variables in 
this optimization problem and use heuristic ant colony algorithm where Zij = 2k=1 Xik Ykj , i, j = 1, 2.
to solve the problem. The results of experiments on 2 × 2 case To compute the size-n product XY , we recursively compute
show that our algorithm achieves significant performance gains. eight size-n/2 products Zij and a few O(n2 ) additions. If we
Extending this algorithm from 2 × 2 case to 3 × 3 case is also use T (n) to denote the total number of arithmetic operations to
discussed.
compute the product of two n × n matrices, then we have that
Index Terms—Ant colony optimization (ACO), evolutionary n
algorithms, Gaussian eliminations, matrix multiplication, multi- T (n) = 8T + O(n2 ) T (2) = O(1).
plicative complexity, Strassen’s algorithm. 2
A proof by recursion shows that T (n) = O(n3 ), and this
I. I NTRODUCTION
method is no faster than the ordinary one. However, the effi-
ciency can be further improved. We consider calculating the
T HE MULTIPLICATION of two matrices is one of the
most basic operations of linear algebra and scientific
computation, such as the solution of linear equations, matrix
following seven products:

inversion, and so on. It has drawn considerable attention to find P1 = X11 (Y12 − Y22 )
methods to speed up the computation [1]. P2 = (X11 + X12 )Y22
The standard algorithm for multiplying two n × n matrices
requires n3 scalar multiplications and n3 − n2 scalar additions, P3 = (X21 + X22 )Y11
for a total arithmetic operation count of 2n3 − n2 . For a very P4 = X22 (−Y11 + Y21 )
long time, no one doubts that that can be done using less than
n3 multiplications. It was not until 1969 that history changed. P5 = (X11 + X22 )(Y11 + Y22 )
Strassen [2] constructed a recursive algorithm that needs nlog2 7 P6 = (X12 − X22 )(Y21 + Y22 )
multiplications to calculate matrix product after analyzing the
relation between the elements of the matrix in 1969. P7 = (−X11 + X21 )(Y11 + Y12 ).
Strassen’s algorithm is usually described in the divide-and- It turns out that
conquer form. Assume for simplicity that n is a power of
two. Let X = (xij ) and Y = (yij ) be two n × n matrices. Z11 = − P2 + P4 + P5 + P6
Z12 = P1 + P2
Z21 = P3 + P4
Manuscript received February 21, 2012; revised June 14, 2012; accepted
June 22, 2012. Date of publication July 20, 2012; date of current version Z22 = P1 − P3 + P5 + P7 .
January 11, 2013. This work was supported in part by the National Natural
Science Foundation of China under Grants 61170081, 61165003, 61170305,
61070009, and 60873078 and in part by the Natural Science Foundation of
This method needs seven multiplications and 18 additions,
Guangdong Province of China under Grant 9251064101000010. This paper was and we derive the following recurrence for arithmetic opera-
recommended by Editor P. P. Angelov. tions T (n) to multiply two n × n matrices:
Y. Zhou is with the School of Computer Science and Engineering, n
South China University of Technology, Guangzhou 510006, China (e-mail:
[email protected]). T (n) = 7T + O(n2 ).
2
X. Lai is with the School of Computer Science and Engineering, South China
University of Technology, Guangzhou 510006, China, and also with the School
of Mathematics and Computer Science, Shangrao Normal University, Shangrao The solution works out to be T (n) = O(nlog2 7 ) = O(n2.81 ).
334001, China (e-mail: [email protected]). Thus, the aforementioned recursive method yields a faster ma-
Y. Li and W. Dong are with the School of Computer Science, Wuhan trix multiplication algorithm, and it can be extended to matrices
University, Wuhan 430072, China (e-mail: [email protected]; hubei_001@
163.com). of arbitrary size by embedding them into larger matrices of size
Digital Object Identifier 10.1109/TSMCB.2012.2207717 2n × 2n .

2168-2267/$31.00 © 2012 IEEE

Authorized licensed use limited to: ST. JOSEPH ENGINEERING COLLEGE MANGALORE. Downloaded on March 09,2025 at 09:56:53 UTC from IEEE Xplore. Restrictions apply.
 348 IEEE TRANSACTIONS ON CYBERNETICS, VOL. 43, NO. 1, FEBRUARY 2013

Strassen’s work started the search for even faster algo- ACO to solve it. We use two variants of ACO named MMAS
rithms for matrix multiplication. Winograd [3] proposed an and MMAS∗ to find solutions for 2 × 2 case. The experimental
algorithm that required seven multiplications and 15 addi- results show that the run-time could be reduced greatly. What
tions/subtractions in 1973. Since multiplications are more is more, we find a lot of other algorithms that are of the same
expensive than additions/subtractions, this algorithm did not quantity with Strassen’s algorithm.
improve Strassen algorithm too much. Hopcroft and Kerr [4] This paper is organized as follows. In Section II, we briefly
showed that at least seven multiplications were required for describe the formulation of the matrix multiplication problem.
multiplying 2 × 2 matrices. In addition, Bshouty [5] has proved Section III contains the details of the method combining the
that at least 15 additions/subtractions are needed for multiply- Gaussian eliminations to reduce the number of variables in
ing 2 × 2 matrices. A lot of effort has been spent on improving the combinatorial optimization problem introduced by small-
Strassen’s upper bound. In 2003, Cohn and Umans [6] devel- size matrix multiplication. The ACO algorithms with com-
oped a group theoretic approach to fast matrix multiplication. bining Gaussian eliminations are described in Section IV. In
They showed that if there are groups that simultaneously satisfy Section V, the effectiveness of the ACO algorithms for matrix
two conditions, then the group theoretic approach will yield multiplication problem of 2 × 2 case is shown by differ-
nontrivial (< 3) upper bound on the exponent for matrix multi- ent experiments. Finally, Section VI gives some concluding
plication. They later found a group that satisfies the two condi- remarks.
tions [7]. Currently, the best algorithm for matrix multiplication
was proposed by Coppersmith and Winograd [8] since its time
II. P ROBLEM F ORMULATION
complexity was O(n2.376 ). However, most researchers believe
that an optimal algorithm for matrix multiplication will run in Given an arbitrary integer n and two n × n matrices X =
O(n2 ) time. (xij ) and Y = (yij ), the optimal matrix multiplication problem
For the aforementioned algorithms except Strassen’s algo- asks how many essential multiplications are needed to compute
rithm, the constants hidden in the O notation are far too huge the entries of product Z = (zij ) = XY . More precisely, we
to make these algorithms usable in practice. Since the number wish to determinate the smallest number m of products
of multiplications in Strassen’s algorithm is optimal, we have
to look for another small format to obtain faster algorithm of pr = ur (xij )vr (yij )
practical value. The 3 × 3 format is of particular interest. To
with linear forms ur in the xij and vr in the yij such that each
improve Strassen’s upper bound on the matrix multiplication,
entry of XY can be represented by a linear combination of pr .
an algorithm for 3 × 3 matrices with 21 or less multipli-
Let us guess that each product pr (r = 1, . . . , m) can be
cations is required. It is known that the optimal number of
expressed in the form
multiplications of 3 × 3 format is in the interval [19, 23] (see,
⎛ ⎞⎛ ⎞
e.g., [9] and [10]). n n
It is not clear exactly how Strassen discovered the submatrix pr = ⎝ r
αij xij ⎠ ⎝ r
βkl ykl ⎠
products that are the key to make his algorithm work. He i,j=1 k,l=1
probably realized that he wanted to determine each element in
the product using less than eight multiplications. However, the such that each entry of XY can be written as a linear combina-
computer search provides an efficient and interesting method tion of these m products
for fast matrix multiplication problem. In 1970, Brent [11] used n m
r
a least squares minimization technique on a function whose zst = xsp ypt = γst pr
minima correspond with matrix multiplication algorithm to p=1 r=1
rediscover the 2 × 2 case. In 2001, Kolen and Bruce [12] used ⎛ ⎞⎛ ⎞
m n n
evolutionary algorithms to construct 2 × 2 matrix multiplica- ⎝
= r
αij xij ⎠ ⎝ r
βkl ykl ⎠ γst
r
(1)
tion. They presented the representational schema, evaluation
r=1 i,j=1 k,l=1
criteria, and evolution mechanisms employed during search.
Their experiments validated that the evolutionary search can where αij r r
, βkl r
, and γst are coefficients to be specified such
replicate Strassen’s discovery. Recently, Oh and Moon [13] also that the aforementioned equations are identities in the indeter-
use genetic algorithm to reproduce Strassen’s algorithm and minates xsp and ypt for all s, p, t ∈ {1, . . . , n}.
found 608 algorithms that have the same quantity as Strassen’s. Strassen’s algorithm, for example, has n = 2, m = 7, αij r
,
They defined the fitness as the number of product matrix entries r r
βkl , and γst with elements taken from the set {−1, 0, 1}. For the
that can be represented by the linear combinations of bilinear simple case n = 2, it has been proved that the smallest number
products and used Gaussian elimination and linear indepen- of product is seven [4].
dence techniques to help genetic search. In the following, we expend (1) and represent it in the
In this paper, we propose the ant colony optimization (ACO) form of a cubic system. We consider an alternate formulation
with combining Gaussian eliminations to solve the combina- of (1)
torial optimization problem introduced by small-size matrix ⎛ ⎞
multiplication. This method takes advantage of combining n m n

Gaussian eliminations to reduce the number of variables in xsp ypt = ⎝ r r

αij βkl xij ykl ⎠ γst
r
. (2)
the combinatorial optimization problem and then use heuristic p=1 r=1 i,j,k,l=1

Authorized licensed use limited to: ST. JOSEPH ENGINEERING COLLEGE MANGALORE. Downloaded on March 09,2025 at 09:56:53 UTC from IEEE Xplore. Restrictions apply.
 ZHOU et al.: ACO WITH COMBINING GAUSSIAN ELIMINATIONS FOR MULTIPLICATION 349

⎛ γ1 1
γ12 1
γ21 1
γ22
⎞
To expend (2),  we maker use of the vector schema to present 11
the sum term ni,j,k,l=1 αij r
βkl xij ykl . ⎜ 2
γ11 2
γ12 2
γ21 2
γ22 ⎟
⎜ 3 3 3 3 ⎟
For r = 1, . . . , m, let αr = (α11 r r
, . . . , α1n r
, α21 r
, . . . , α2n , ⎜ γ11 γ12 γ21 γ22 ⎟
H=⎜ 4
γ11 4
γ12 4
γ21 4 ⎟
γ22
r r r r r
. . . , αn1 , . . . , αnn ) and βr = (β11 , . . . , β1n , β21 , . . . , β2n r
, ⎜ ⎟
⎜ 5
γ11 5
γ12 5
γ21 5 ⎟
γ22
r
. . . , βn1 r
, . . . , βnn ) be row vectors whose elements are those ⎝ 6 6 6 6 ⎠
r r γ11 γ12 γ21 γ22
of matrices (αij )n×n and (βkl )n×n , respectively, in row order. 7
γ11 7
γ12 7
γ21 7
γ22
Furthermore, we use pr to denote the column vector whose ⎛1 0 0 0⎞
elements are those of αrT βr in row order, i.e.,
⎜0 1 0 0⎟
⎜0 0 0 0⎟
⎜ ⎟
r
pr = (α11 r
β11 r
, . . . , α11 r
β1n r
, α11 r
β21 r
, . . . , α11 r
β2n ,..., ⎜0 0 0 0⎟
⎜ ⎟
⎜0 0 0 0⎟
r
α11 r
βn1 r
, . . . , α11 r
βnn r
, . . . , αnn r
β11 r
, . . . , αnn r
β1n , ⎜0 0 ⎟
⎜ 0 0⎟
⎜1 0 0⎟
r
αnn r
β21 r
, . . . , αnn r
β2n r
, . . . , αnn r
βn1 r
, . . . , αnn r T
βnn ) (3) ⎜ 0 ⎟
⎜0 1 0⎟
D=⎜ ⎟
0
⎜0 0 (5)
1 0⎟
⎜ ⎟
which is the vector representation for the sum term ⎜0 0 0 1⎟
n ⎜ ⎟
r r ⎜0 0 0 0⎟
i,j,k,l=1 αij βkl xij ykl . ⎜0 0 ⎟
In a similar manner, the entry of product XY can be ⎜ 0 0⎟
⎜0 0 0⎟
expressed  by the vector schema. For example, for n = 2, ⎜ 0 ⎟
⎜0 0 0 0⎟
the entry 2i,j,k,l=1 ηijkl xij ykl of product XY , where ηijkl ⎝ ⎠
0 0 1 0
takes value from {0, 1}, can be expressed as (x11 y11 , x11 y12 , 0 0 0 1
x11 y21 , x11 y22 , x12 y11 , x12 y12 , x12 y21 , x12 y22 , . . . , x22 y11 ,
x22 y12 , x22 y21 , x22 y22 )(η1111 , η1112 , η1121 , η1122 , η1211 , Since both matrices H and D have n2 column vectors,
η1212 , η1221 , η1222 , . . . , η2211 , η2212 , η2221 , η2222 )T . Hence, we denote D = (d1 , . . . , dn2 ) and H = (h1 , . . . , hn2 ). Then,
the vector schema representation for x11 y11 + x12 y21 is (1, 0, system (4) consists of n2 systems
0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0)T , since the first term x11 y11
and the seventh term x12 y21 appear in x11 y11 + x12 y21 . P hk = dk (k = 1, . . . , n2 ). (6)
Similarly, the vector representations for x11 y12 + x12 y22 , r r
x21 y11 + x22 y21 , and x21 y12 + x22 y22 are (0, 1, 0, 0, 0, 0, 0, 1, Note that, when coefficients αij and βkl are held fixed, each
0, 0, 0, 0, 0, 0, 0, 0)T , (0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0)T , system of (6) is a system of linear equations, which can be
and (0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1)T , respectively. solved efficiently by Gaussian eliminations. For simplicity, in
r r
Let n4 × m matrix P = (p1 , . . . , pm ). Defining the matrix the following, we limit the elements αij and βkl to the set
D to be n4 × n2 matrix whose column vectors correspond to {−1, 0, 1}, and then, our objective is to assign −1, 0, and
r r
the entries of product XY and matrix H to be m × n2 matrix 1 to variables αij and βkl such that linear systems (6) are
whose elements are γst r
(r = 1, . . . , m; s, t = 1, . . . , n), we solvable.
may then express (2) as the following system: A natural generalization of the aforementioned problem is to
ask whether there exists an assignment of variables such that
at least a certain number of linear systems are solvable. We
PH = D (4) formulate the number of solvable linear systems as the value
of an objective function to be maximized so the matrix multi-
which is a system of n6 cubic equations with 3n2 m unknowns. plication problem is transformed into a maximization problem.
r
For example, for n = 2 and m = 7, For given matrix P which is determined by coefficients αij
r
and βkl in (4), we use the notation f itness(P ) to denote the
⎛ α1 β 1 α211 β11
2 α311 β11
3 α411 β11
4 α511 β11
5 α611 β11
6 7 ⎞
α711 β11 number of solvable linear systems which is an integer from
11 11
1 β1 α211 β12
2 α311 β12
3 α411 β12
4 α511 β12
5 α611 β12
6 α711 β12
7 zero to n2 . The aim is then to find an assignment of variables
⎜ α11 12 ⎟
⎜ α111 β21
1 2 2
α11 β21 3 3
α11 β21 4 4
α11 β21 5 5
α11 β21 6 6
α11 β21 7 7 ⎟
α11 β21
r
αij r
and βkl to maximize f itness(P ). Since the number of
⎜ 1 1 ⎟
⎜ α11 β22 2 2
α11 β22 3 3
α11 β22 4 4
α11 β22 5 5
α11 β22 6 6
α11 β22 7 ⎟
α711 β22
2
all possible assignments is 32n m , the exhaustive search is
⎜ 1 1 7 ⎟
⎜ α12 β11 α212 β11
2 α312 β11
3 α412 β11
4 α512 β11
5 α612 β11
6 α712 β11 ⎟ impractical, even for the simplest case n = 2 and m = 7. In
⎜ α1 β 1 7 ⎟
⎜ 12 12 α212 β12
2 α312 β12
3 α412 β12
4 α512 β12
5 α612 β12
6 7
α12 β12 ⎟ the following sections, we will use the heuristic ant colony
⎜ α1 β 1 2 2 3 3 4 4 5 5 6 6 7 ⎟
α712 β21
⎜ 12 21 α12 β21 α12 β21 α12 β21 α12 β21 α12 β21 ⎟ algorithm with combining Gaussian eliminations to solve this
⎜ α1 β 1 2 2 3 3 4 4 5 5 6 6 7 ⎟
α712 β22
P =⎜ ⎟
12 22 α12 β22 α12 β22 α12 β22 α12 β22 α12 β22
⎜ α121 β11
1 7 ⎟ problem.
α221 β11
2 α321 β11
3 α421 β11
4 α521 β11
5 α621 β11
6 α721 β11
⎜ 1 1 ⎟
⎜ α21 β12 α221 β12
2 α321 β12
3 α421 β12
4 α521 β12
5 α621 β12
6 7 ⎟
α721 β12
⎜ 1 1 7 ⎟
⎜ α21 β21 2 2
α21 β21 3 3
α21 β21 4 4
α21 β21 5 5
α21 β21 6 6
α21 β21 α721 β21 ⎟ III. C OMBINE G AUSSIAN E LIMINATIONS TO R EDUCE THE
⎜ α1 β 1 7 ⎟
⎜ 21 22 2 2
α21 β22 3 3
α21 β22 4 4
α21 β22 5 5
α21 β22 6 6
α21 β22 α721 β22 ⎟ N UMBER OF VARIABLES IN M ATRIX
⎜ α1 β 1 α222 β11
2 α322 β11
3 α422 β11
4 α522 β11
5 α622 β11
6 7 ⎟
α722 β11
⎜ 22 11 ⎟ M ULTIPLICATION P ROBLEM
⎜ α1 β 1 α222 β12
2 α322 β12
3 α422 β12
4 α522 β12
5 α622 β12
6 7 ⎟
α722 β12
⎝ 221
12
1 2 2 3 3 4 4 5 5 6 6
⎠
α22 β21 α22 β21 α22 β21 α22 β21 α22 β21 α22 β21 α722 β21
7
Computational difficulty in solving the matrix multiplication
α122 β22
1 2 2
α22 β22 3 3
α22 β22 4 4
α22 β22 5 5
α22 β22 6 6
α22 β22 α722 β22
7
problem is caused by problem dimension, i.e., the number
Authorized licensed use limited to: ST. JOSEPH ENGINEERING COLLEGE MANGALORE. Downloaded on March 09,2025 at 09:56:53 UTC from IEEE Xplore. Restrictions apply.
 350 IEEE TRANSACTIONS ON CYBERNETICS, VOL. 43, NO. 1, FEBRUARY 2013

of variables in the the combinatorial optimization problem clearly as follows:

introduced by matrix multiplication. If the number of variables ⎛ 1 1 2 2 3 3 ⎞
in the combinatorial optimization problem is small, then the α11 β11 α11 β11 α11 β11 1 0 0 0
⎜ α11 β12 α11 β12 α11 β12
1 1 2 2 3 3
0 1 0 0⎟
matrix multiplication problem can be solved with a reasonable ⎜ 1 1 ⎟
⎜ α11 β21 α11 2 2
β21 3
α11 3
β21 0 0 0 0⎟
expenditure of effort. In the following, we present a method ⎜ 1 1 ⎟
⎜ α11 β22 α11 2 2
β22 3
α11 3
β22 0 0 0 0⎟
called combining Gaussian eliminations to reduce the number ⎜ 1 1 ⎟
⎜ α12 β11 α12 2 2
β11 3
α12 3
β11 0 0 0 0⎟
of the variables in the combinatorial optimization problem. This ⎜ 1 1 ⎟
⎜ α12 β12 α12 2 2
β12 3
α12 3
β12 0 0 0 0⎟
method and its analysis also provide some insights regarding ⎜ 1 1 ⎟
⎜ α12 β21 α12 2 2
β21 3
α12 3
β21 1 0 0 0⎟
the correlations among the variables in this problem. ⎜ 1 1 ⎟
⎜ α12 β22 α12 2 2
β22 3
α12 3
β22 0 1 0 0⎟
For the analysis and our later discussion, the following ⎜ 1 1 ⎟F
⎜ α21 β11 α21 2 2
β11 3
α21 3
β11 0 0 1 0 ⎟ 7×4
lemma plays a key role. Here and in the rest of this paper, ⎜ 1 1 ⎟
⎜ α21 β12 α21 2 2
β12 3
α21 3
β12 0 0 0 1⎟
we consider the optimal matrix multiplication problem with ⎜ 1 1 ⎟
⎜ α21 β21 α21 2 2
β21 3
α21 3
β21 0 0 0 0⎟
the special case n = 2 and m = 7. However, our analysis and ⎜ 1 1 ⎟
⎜ α21 β22 α21 2 2
β22 3
α21 3
β22 0 0 0 0⎟
results hold in general for all n and m. Recall that our aim is to ⎜ 1 1 ⎟
assign −1, 0, and 1 to variables αijr r ⎜ α22 β11 α22 2 2
β11 3
α22 3
β11 0 0 0 0⎟
and βkl (r = 1, . . . , 7; i, j, ⎜ 1 1 ⎟
k, l = 1, 2) such that the following linear system is solvable: ⎜ α22 β12 α22 2 2
β12 3
α22 3
β12 0 0 0 0⎟
⎝ 1 1 2 2 3 3 ⎠
α22 β21 α22 β21 α22 β21 0 0 1 0
1 1 2 2 3 3
α22 β22 α22 β22 α22 β22 0 0 0 1
P16×7 H7×4 = D16×4 (7) ⎛ 4 4 5 5 6 6 7 7 ⎞
α11 β11 α11 β11 α11 β11 α11 β11
⎜ α11 β12 α11 β12
4 4 5 5 6 6
α11 β12 7 7
α11 β12 ⎟
⎜ 4 4 7 ⎟
⎜ α11 β21 α11 β21
5 5 6 6
α11 β21 7
α11 β21 ⎟
where matrices P , H, and D are defined as (5). ⎜ 4 4 7 ⎟
⎜ α11 β22 α11 β22
5 5 6 6
α11 β22 7
α11 β22 ⎟
Lemma 1: Given matrix P of (7), assume that the set of ⎜ 4 4 7 ⎟
vectors S = {p1 , p2 , . . . , p7 } is linearly independent, where pi ⎜ α12 β11 α12 β11
5 5 6 6
α12 β11 7
α12 β11 ⎟
⎜ 4 4 7 ⎟
⎜ α12 β12 α12 β12
5 5 6 6
α12 β12 7
α12 β12 ⎟
(i = 1, . . . , 7) is the ith column vector of P . Then, linear matrix ⎜ 4 4 7 ⎟
⎜ α12 β21 α12 β21
5 5 6 6
α12 β21 7
α12 β21 ⎟
(7) is solvable if and only if the linearly independent set S ⎜ 4 4 7 ⎟
can be partitioned into two subsets S1 = {pi1 , pi2 , pi3 } and ⎜ α β22 α12 β22
5 5 6 6
α12 β22 7
α12 β22 ⎟
= ⎜ 12 7 ⎟. (9)
S2 = {pi4 , . . . , pi7 } such that each element pik (k = 4, . . . , 7) ⎜ α21 β11 α21 β11
4 4 5 5 6 6
α21 β11 7
α21 β11 ⎟
⎜ 4 4 7 ⎟
⎜ α21 β12 α21 β12
5 5 6 6
α21 β12 7
α21 β12 ⎟
of subset S2 can be written as a linear combination of vectors ⎜ 4 4 7 ⎟
⎜ α21 β21 α21 β21
5 5 6 6
α21 β21 7
α21 β21 ⎟
pi1 , pi2 , pi3 , d1 , d2 , d3 , d4 . ⎜ 4 4 7 ⎟
⎜ α21 β22 α21 β22
5 5 6 6
α21 β22 7
α21 β22 ⎟
Proof: (⇒) Suppose that linear matrix (7) is solvable, i.e., ⎜ 4 4 7 ⎟
⎜ α22 β11 α22 β11
5 5 6 6
α22 β11 7
α22 β11 ⎟
each di (i = 1, . . . , 4) can be written as a linear combination ⎜ 4 4 7 ⎟
of vectors p1 , . . . , p7 . Let S  = {p1 , . . . , p7 , d1 , . . . , d4 }; then, ⎜ α22 β12 α22 β12
5 5 6 6
α22 β12 7
α22 β12 ⎟
⎝ 4 4 5 5 6 6 7 7 ⎠
α22 β21 α22 β21 α22 β21 α22 β21
the rank of S  is seven. Since vectors d1 , . . . , d4 are linearly 4 4 5 5 6 6 7 7
α22 β22 α22 β22 α22 β22 α22 β22
independent, they can be expended to a basis of S  . We denote
this basis by set {d1 , . . . , d4 , pi1 , pi2 , pi3 }; hence, the result
Since the elements of pi are those of αiT βi in row order and
follows. r r
αij and βkl are limited to set {−1, 0, 1}, the number of all
(⇐) Suppose that each element pik (k = 4, . . . , 7) of
subset S2 can be written as a linear combination of possible pi ’s is 34 × 34 . Eliminating zero vector and vectors
vectors pi1 , pi2 , pi3 , d1 , d2 , d3 , d4 ; then, the rank of S  = with reversed signs, the number of unique pi ’s is 1600. We
{p1 , . . . , p7 , d1 , . . . , d4 } is not greater than seven. Since the denote these vectors as {q1 , . . . , q1600 } and use Q to denote the
subset S of S  is linearly independent and its cardinality is matrix whose column vectors are qi ’s, i.e., Q = (q1 · · · q1600 ).
seven, it is a basis of S  . This completes the proof.  We now consider the following problem: Given vectors
According to Lemma 1, the matrix multiplication problem p1 , p2 , p3 , how can we find vectors p4 , . . . , p7 such that matrix
for order 2 and 7 multiplications is equivalent to find lin- (8) is solvable (if these vectors exist)?
early independent vectors {pi1 , pi2 , . . . , pi7 }, such that each To solve the aforementioned question, it suffices to find
pik (k = 4, . . . , 7) can be written as a linear combination of which of following 1600 systems of linear equations is
vectors pi1 , pi2 , pi3 , d1 , d2 , d3 , d4 . In the following, we simplify solvable:
the notation by replacing pik with pk (k = 1, 2, . . . , 7) in an
R16×7 δi = qi (i = 1, . . . , 1600). (10)
unambiguous context. Then, our objective is converted to find
linearly independent vectors {p1 , p2 , . . . , p7 }, such that the
We use Gaussian eliminations to solve the system of lin-
following matrix equation:
ear equations. In general, we need to apply 1600 rounds of
Gaussian eliminations to solve 1600 systems. However, since
R16×7 F7×4 = E16×4 (8) the matrix R16×7 is the same, the systems of linear equations
(10) do not need to be solved separately. We can combine 1600
rounds of Gaussian eliminations to one round.
is solvable, where R16×7 = (p1 , p2 , p3 , d1 , . . . , d4 ) and Formally, we combine two matrices R16×7 and Q16×1600 to
E16×4 = (p4 , . . . , p7 ). Matrix (8) can be expressed more form the augmented matrix (R16×7 Q16×1600 ) and apply one
Authorized licensed use limited to: ST. JOSEPH ENGINEERING COLLEGE MANGALORE. Downloaded on March 09,2025 at 09:56:53 UTC from IEEE Xplore. Restrictions apply.
 ZHOU et al.: ACO WITH COMBINING GAUSSIAN ELIMINATIONS FOR MULTIPLICATION 351

round of Gaussian eliminations (through the use of elementary

row
 operations) tothis matrix until it has the reduced form
T7×7 G17×1600
, where T7×7 is the matrix in upper tri-
Q9×7 G29×1600
angular or row echelon form and Q9×7 is the matrix whose
components are all zeros.
 G1 
Denote G = G7×1600
2 = (g1 · · · g1600 ). Recall that R = Fig. 1. Construction graph for matrix multiplication problem.
9×1600
(p1 , . . . , p3 , d1 , . . . , d4 ). Then, vector qi can be written as a
linear combination of vectors p1 , p2 , p3 , d1 , . . . , d4 if and only IV. A NT C OLONY A LGORITHM FOR M ATRIX
if its corresponding vector gi has the form (∗, . . . , ∗, 0, . . . , 0)T . M ULTIPLICATION P ROBLEM
     
7 9 Ant colony algorithm is a relatively new metaheuristic and
We use G to denote the set of these gi ’s, i.e., G = {gi |gi ∈ belongs to the class of biology-inspired methods that are de-
{g1 , . . . , g1600 } and has the form (∗, . . . , ∗, 0, . . . , 0)T }. Note veloped and based on biological activities. When ants search
     
7 9 for food, they are capable of finding the shortest pathway in a
that the rank of G is, at most, seven. set of alternative pathways from their nest by exchanging in-
By Lemma 1, for given order 2 and seven multiplications, formation via pheromones. The basic idea of ACO is to imitate
the matrix multiplication problem is equivalent to find vectors the cooperative behavior of ant colonies [14]. The ACO is a
r r
p1 , p2 , p3 (which are determined by variables αij and βkl (r = population-based randomized search approach using positive
1, 2, 3; i, j, k, l = 1, 2)) such that the rank of G equals seven. feedback as well as greedy search. It has been applied to a range
Then, the matrix multiplication problem can be converted into of complicated optimization problems and has been shown to be
r
an optimization problem: Find an assignment of variables αij both robust and versatile.
and βkl r
such that the rank of G , which is defined as the An ACO algorithm contains two basic elements: the prob-
objective function, is maximized. The number of unknown abilistic mechanism parameterized by pheromone vector to
variables in this optimization problem is 3 × 8. Since the construct solutions and the updating rule for pheromone vector.
number of unknown variables in original (7) is 7 × 8, the search It can be formalized as follows.
space is reduced greatly.
r r
In summary, given αij and βkl (r = 1, 2, 3; i, j, k, l =
1, 2), the objective function of the maximization problem in- The ACO Algorithm
troduced by the matrix multiplication problem (i.e., the rank of Begin
G ) is computed by the following steps. Initialization: Set parameters, initialize pheromones;
While (termination condition does not hold) do
Construct solutions;
Objective function value calculation Update the pheromones;
r r Endwhile
Input: αij and βkl .
End
Step 1) Calculate p1 , p2 , p3 according to (3).
Step 2) Compose the augmented matrix (p1 , p2 , p3 , d1 , . . . ,
d4 |q1 , . . . , q1600 ) = (R|Q). There are several ACO variants in the literature. We intro-
Step 3) Perform elementary row operations to transform the duce a graph-based ant system as given in [15] to solve the
augmented matrix (R|Q) into (R |G), where matrix combinatorial optimization problem introduced by the matrix
R is in an echelon row form. multiplication problem.
Step 4) Remove any column vector u = (u1 , u2 , . . . , u16 )T We consider a general optimization problem to be maximized
from G that contains at least an element ui = 0(8 ≤ f : {−1, 0, 1}n → R and construct the chain graph to represent
i ≤ 16). this function. The chain graph is a directed multigraph G =
Step 5) Return the rank of G the set of the remaining (V, E) with vertex set V = {vi |i = 0, 1, . . . , n, } and edge
vectors. set E = {ei,j |i = 1, . . . , n; j = −1, 0, 1} (see Fig. 1). This
graph contains a sequence of vertices, and successive vertices
are jointed by three parallel edges. The vertex v0 serves as
the starting vertex, and vertex vi (i = 1, . . . , n) corresponds to
As we can see from earlier discussion, the matrix multi- variable xi in function f . Vertex vi can be reached from vi−1
plication problem for order 2 and 7 multiplications can be by three edges: ei,−1 , ei,0 , and ei,1 . In case that ei,j is chosen,
converted to find assignments of variables αij r r
and βkl (r = variable xi is set to j (j = −1, 0, 1) in the constructed solution.
1, 2, 3; i, j, k, l = 1, 2), such that the objective function value In the following, we give a more detailed description of the
described earlier is maximized. Since the complete search construction and pheromone update procedures.
approach often becomes intractable on hard combinatorial Construct an ant solution: ACO algorithm constructs a so-
problem, in this paper, we employ an incomplete approach lution by traversing the nodes v0 , v1 , v2 , . . . , vn . In every step,
for solving this maximization problem based on the ACO the ant chooses an edge with a probability proportional to the
metaheuristic [14]. pheromone value of that edge. Pheromones are positive real
Authorized licensed use limited to: ST. JOSEPH ENGINEERING COLLEGE MANGALORE. Downloaded on March 09,2025 at 09:56:53 UTC from IEEE Xplore. Restrictions apply.
 352 IEEE TRANSACTIONS ON CYBERNETICS, VOL. 43, NO. 1, FEBRUARY 2013

values associated with the edges of the construction graph. 1-ANT MMAS
Formally, we denote pheromones by a function τ : E → R+ . Begin
At vertex vi−1 , edge ei,j is then chosen according to proba- For all ei,j , initialize τ (ei,j );
bility τ (ei,j )/(τ (ei,−1 ) + τ (ei,0 ) + τ (ei,1 )). Thus, a walk of Construct an initial solution x;
the ant produces a solution vector x = (x1 , x2 , . . . , xn ) ∈ While (termination condition does not hold){
{−1, 0, 1}n , and we denote the path by P (x). Construct a new solution x ;
Update pheromone: A pheromone τ (ei,j ), associated with If f (x ) ≥ f (x) then x := x ;
edge ei,j , is initiated randomly and is then changed dynamically Update the pheromone values w.r.t. x;
during the run of the algorithm. The aim of the pheromone }
update is to strengthen the pheromone values on the edges End
participating to good solutions and to decrease these with bad 1-ANT MMAS∗
ones. More precisely, these changes are determined by the Begin
evaporation rate ρ ∈ [0, 1]: A ρ-fraction of all pheromones For all ei,j , initialize τ (ei,j );
evaporates, and some pheromone is added to the edges that Construct an initial solution x;
belong to best so-far path. In order to prevent premature con- While (termination condition does not hold){
vergence, Stützle and Hoos [16] proposed the max–min ant Construct a new solution x ;
system (MMAS) that imposes explicit limit on the pheromone If f (x ) > f (x) then x := x ;
to ensure that each search point has a positive probability Update the pheromone values w.r.t. x;
of being chosen in the next step. In this paper, we use the }
max–min pheromone update rule and restrict each pheromone End
to the interval [1/n, 1 − (1/n)]. This choice is inspired by the n-ANT MMAS
standard mutation in the evolutionary algorithm which flips Begin
each bit of solution vector (x1 , x2 , . . . , xn ) with probability For all ei,j , initialize τ (ei,j );
1/n. Depending on whether edge ei,j is contained in the path Construct an initial solution x;
P (x) of the constructed solution x, the pheromone update is While (termination condition does not hold){
performed as follows: Construct n new solutions;
   Select the best new solution x ;
 min (1−ρ)τ (ei,j )+ρ,1− n1 if ei,j ∈ P (x) If f (x ) ≥ f (x) then x := x ;
τ (ei,j ) =
max (1−ρ)τ (ei,j ), n1 otherwise Update the pheromone values w.r.t. x;
(11) }
where ρ is the evaporation rate of the pheromone. End
Since the objective function’s value of the combinatorial n-ANT MMAS∗
optimization problem introduced by the matrix multiplication Begin
problem with size n and multiplications m is an integer between For all ei,j , initialize τ (ei,j );
zero and m, it is obviously a plateau function. Plateau is a Construct an initial solution x;
region in the search space, in which all search points have the While (termination condition does not hold){
same fitness. Construct n new solutions;
Commonly, there exist two acceptance criteria for genetic Select the best new solution x ;
algorithms, ACOs, and other heuristic algorithms. One is that If f (x ) > f (x) then x := x ;
an algorithm accepts a new solution which is not bad compared Update the pheromone values w.r.t. x;
with the current best one, and the other is that the algorithm }
accepts a new solution only when it is strictly better than the End
current best one. For a plateau function, acceptance criterion
has a notable impact on evolutionary algorithms’ performances, We use the aforementioned algorithms to solve the matrix
particularly the run-time of the algorithms [17], [18]. In [19], multiplication problem. For the matrix multiplication problem
Neumann et al. have investigated that the acceptance criterion with order 2 and 7 multiplications, a solution x means an
r r
of accepting a new solution as good as the current best one assignment of variables αij and βkl (r = 1, 2, 3; i, j, k, l =
could drastically reduce the run-time of MMAS. 1, 2) to −1, 0, and 1, and the objective function f (x) is de-
In this paper, we use two kinds of ant colony algorithms termined by the objective function value calculation algorithm
called MMAS and MMAS∗ to solve matrix multiplication in Section III. The experimental results are given in the next
problem of case n = 2. The MMAS algorithms accept a new section.
solution that is not bad compared with the current best one.
The MMAS∗ algorithms accept a new candidate solution that
V. E XPERIMENTAL R ESULTS
is strictly better than the current best one. Each kind contains
three different numbers of ants, i.e., 1, 15, and 30 ants. MMAS We now experimentally compare the different algorithms
and MMAS∗ are similar to or the same as MMAS and MMAS∗ for the matrix multiplication problem of the case n = 2 and
discussed in [19] and [20]. Both of them are described as m = 7 that we have presented in this paper. We first give a brief
follows. description of main experimental results.
Authorized licensed use limited to: ST. JOSEPH ENGINEERING COLLEGE MANGALORE. Downloaded on March 09,2025 at 09:56:53 UTC from IEEE Xplore. Restrictions apply.
 ZHOU et al.: ACO WITH COMBINING GAUSSIAN ELIMINATIONS FOR MULTIPLICATION 353

TABLE I
D IFFERENT T YPES OF 2 × 2 M ATRIX M ULTIPLICATION A LGORITHMS

In [13], Oh and Moon used genetic algorithms to find so- find a solution. In extreme case, the run-times for MMAS and
lutions for matrix multiplication of case n = 2 and m = 7. In MMAS∗ algorithms are all less than 1 s.
general, it takes a few hours to find a solution, and in extreme Table I shows the ten types of solutions found by both
case, the run-time is 10 s. In our experiments, results show MMAS and MMAS∗ for matrix multiplication problem of case
that the run-time could be reduced greatly. It takes about from n = 2 and m = 7. Besides the nine types of solutions found in
seconds to less than 20-s mean time for MMAS algorithms [13], both of MMAS and MMAS∗ find a new solution type of
and dozens of minutes in general for MMAS∗ algorithms to 2266668 (type 4).
Authorized licensed use limited to: ST. JOSEPH ENGINEERING COLLEGE MANGALORE. Downloaded on March 09,2025 at 09:56:53 UTC from IEEE Xplore. Restrictions apply.
 354 IEEE TRANSACTIONS ON CYBERNETICS, VOL. 43, NO. 1, FEBRUARY 2013

TABLE II TABLE III

M EAN N UMBERS OF I TERATIONS (M EAN I TER #), M EAN T IMES , AND S MALLEST AND L ARGEST N UMBERS OF I TERATIONS (S MALLEST # AND
T HEIR MSE S OF D IFFERENT MMAS A LGORITHMS TO F IND A S OLUTION L ARGEST #, R ESPECTIVELY ) AND THE S HORTEST AND L ONGEST T IMES
FOR 2 × 2 C ASE OVER 50 I NDEPENDENT RUNS . (S HORTEST T AND L ONGEST T, R ESPECTIVELY ) OF D IFFERENT MMAS
T HE U NIT OF T IME IS S ECOND A LGORITHMS TO F IND A S OLUTION FOR 2 × 2 C ASE OVER
50 I NDEPENDENT RUNS . T HE U NIT OF T IME IS S ECOND

In the following, the effect of ACO algorithm parameters on MMAS could find a solution for matrix multiplication problem
performance and comparisons of MMAS and MMAS∗ will be of case n = 2 and m = 7 with the smallest mean number of
discussed by various experiments. iterations when ρ = 0.05. While 15-ANT MMAS needs the
In Section V-B, we investigate 1-ANT MMAS, 15-ANT largest mean number of iterations to find a solution when ρ is
MMAS, and 30-ANT MMAS with different evaporation rates set to 0.05, 30-ANT MMAS needs the largest mean number of
ρ, and in Section V-C, we investigate 1-ANT MMAS∗ , iterations to find a solution when ρ is set to 0.01. With ρ taking
15-ANT MMAS∗ , and 30-ANT MMAS∗ with respect to their value from 0.1 to 1, 15-ANT MMAS and 30-ANT MMAS
iterations and times needed to find a solution. Furthermore, in could find a solution with relatively smaller iterations. The more
Section V-D, we compare MMAS and MMAS∗ with respect the number of ants is, the smaller the number of iterations to
to the mean iterations and the mean times to find solutions for find a solution is.
matrix multiplication problem of case n = 2 and m = 7. Since 30-ANT MMAS and 15-ANT MMAS try 30 and 15
In Section V-E, we further investigate the distribution of candidate solutions in one iteration, respectively, while 1-ANT
solutions found by 1-ANT MMAS, 15-ANT MMAS, and MMAS tries only one candidate solution in one iteration, the
30-ANT MMAS with different evaporation rates ρ. mean time needed for 1-ANT MMAS to find a feasible solution
is the shortest among these three algorithms, as shown in
Table II. In addition, the shortest mean time for 1-ANT MMAS
A. Experiment Setting
to find a solution for matrix multiplication of case n = 2 and
Experimental computer is an Intel Xeon 2.00-GHz server m = 7 is 2.2896 when ρ is set to 0.05.
with 4-GB RAM. The evaporation rates are ρ = 0.01, 0.05, 0.1, Table III shows that 1-ANT MMAS could find a solution
0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, and 1.0. from less than 1 s to less than 1 min and, for 15-ANT MMAS
The n’s of n-ANT MMAS and n-ANT MMAS∗ are 1, 15, and 30-ANT MMAS, the times to find a solution range from
and 30. less than 1 s to several minutes.
Mean square error (mse) is the measure of stability per-
formance of an algorithm. Table II also shows the mse of
B. 1-ANT MMAS, 15-ANT MMAS, and 30-ANT MMAS for
three MMASs. As far as the stability performance (mse) is
Matrix Multiplication Problem of Case n = 2 and m = 7
concerned, we could see that 1-ANT MMAS is the worst,
In this subsection, we investigate 1-ANT MMAS, 15-ANT 30-ANT MMAS is the best, and 15-ANT MMAS is medium in
MMAS, and 30-ANT MMAS on finding solutions for matrix stability measured by the mse for the number of iterations. This
multiplication problem of case n = 2 and m = 7. may be because the more ants an algorithm contains, the more
Table II and III show the details of experimental results of candidate solutions are constructed and tried in one iteration,
50 independent runs. From Table II, we can find that 1-ANT and the more stable this algorithm is.
Authorized licensed use limited to: ST. JOSEPH ENGINEERING COLLEGE MANGALORE. Downloaded on March 09,2025 at 09:56:53 UTC from IEEE Xplore. Restrictions apply.
 ZHOU et al.: ACO WITH COMBINING GAUSSIAN ELIMINATIONS FOR MULTIPLICATION 355

TABLE IV TABLE V
M EAN N UMBERS OF I TERATIONS (M EAN I TER #), M EAN T IMES , AND L ARGEST AND S MALLEST N UMBERS OF I TERATIONS (S MALLEST # AND
T HEIR MSE OF D IFFERENT MMAS∗ A LGORITHMS TO F IND A S OLUTION L ARGEST #, R ESPECTIVELY ) AND THE S HORTEST AND L ONGEST T IMES
FOR 2 × 2 C ASE OVER 50 I NDEPENDENT RUNS . (S HORTEST T AND L ONGEST T, R ESPECTIVELY ) OF D IFFERENT MMAS∗
T HE U NIT OF T IME IS S ECOND A LGORITHMS TO FIND A S OLUTION FOR 2 × 2 C ASE OVER
50 I NDEPENDENT RUNS . T HE U NIT OF T IME IS S ECOND

C. 1-ANT M M AS ∗ , 15-ANT M M AS ∗ , and 30-ANT

TABLE VI
M M AS ∗ for Matrix Multiplication of Case n = 2 and m = 7 R ATIO OF THE M EAN N UMBER OF I TERATIONS AND THE M EAN T IME
B ETWEEN MMAS∗ AND MMAS. T HE NAIVE DATA
In this subsection, we investigate 1-ANT MMAS∗ , 15-ANT C OME F ROM TABLES II AND IV
MMAS∗ , and 30-ANT MMAS∗ on finding solutions for matrix
multiplication of case n = 2 and m = 7. Table IV and V show
the experimental results of 50 independent runs.
As shown in Table IV, 1-ANT MMAS∗ needs in general the
largest mean number of iterations to find a solution, particularly
when ρ = 0.01, where the mean number of iterations reaches
1.1452e + 6. 30-ANT MMAS∗ needs in general the smallest
mean number of iterations. In addition, the mean number of
iterations for 15-ANT MMAS∗ is in general less than that of
1-ANT MMAS∗ but larger than that of 30-ANT MMAS∗ . The
case of the mse of the number of iterations is similar to that of
the mean number of iterations.
For the mean time needed to find a solution for matrix
for 1-ANT MMAS∗ to find a solution for matrix multiplication
multiplication of case n = 2 and m = 7, 1-ANT MMAS∗ needs
of case n = 2 and m = 7 is nearly hundreds of times as large
thousands of seconds, approximate to dozens of minutes. The
as that of 1-ANT MMAS. For 15-ANT MMAS∗ and 30-ANT
mean times for 15-ANT MMAS∗ range from minutes to hours,
MMAS∗ , the mean number of iterations for finding a solution is
and the mean times for 30-ANT MMAS∗ range from dozens of
from dozens to thousands of times as large as those of 15-ANT
minutes to hours.
MMAS and 30-ANT MMAS, respectively. Furthermore, the
From Table V, we could see that the shortest times to find
MMAS algorithms need shorter mean times to find a solution
a solution are less than 1 s for 1-ANT MMAS∗ , 15-ANT
for matrix multiplication problem of case n = 2 and m = 7,
MMAS∗ , and 30-ANT MMAS∗ . However, the longest times for
while MMAS∗ algorithms need longer mean times.
MMAS∗ algorithms are several hours.
We can conclude that, for the matrix multiplication problem,
the MMAS algorithms are faster than MMAS∗ algorithms.
D. Comparison of MMAS and M M AS ∗
The reason why MMAS algorithms are faster than MMAS∗
Table VI shows the ratio of the mean number of iterations algorithms may be that MMAS algorithms accept a candidate
of MMAS∗ to that of MMAS and also the ratio of the mean solution which is not worse than the current best candidate so-
time of MMAS∗ to that of MMAS. The source data come from lution, which makes MMAS algorithms try different candidate
Tables II and IV. It is shown that the mean number of iterations solutions as many as possible.
Authorized licensed use limited to: ST. JOSEPH ENGINEERING COLLEGE MANGALORE. Downloaded on March 09,2025 at 09:56:53 UTC from IEEE Xplore. Restrictions apply.
 356 IEEE TRANSACTIONS ON CYBERNETICS, VOL. 43, NO. 1, FEBRUARY 2013

Fig. 2. Distribution of different solution types found by 1-ANT MMAS. For Fig. 4. Distribution of different solution types found by 30-ANT MMAS. For
each ρ, the algorithm is terminated after having found 2383 solutions. each ρ, the algorithm is terminated after having found 2383 solutions.

more solutions of type 6 than with a high value of ρ, and the

case for types 7 and 10 is contrary to that of type 6.

VI. C ONCLUSION AND D ISCUSSION

In this paper, we have proposed a method of combining
Gaussian eliminations to reduce the number of variables in
matrix multiplication problem, which shrinks the search space.
In the case of n = 2 and m = 7, the number of variables could
be reduced from 7 ∗ 8 to (7 − 4) ∗ 8. With the method of com-
bining Gaussian eliminations, two ACO algorithms considered
in this paper, namely, MMAS and MMAS∗ , could find solutions
for matrix multiplication problem of case n = 2 and m = 7
at high speeds. This speedup is not the result of sophisticated
learning strategies but rather of efficient reducing the search
space.
Kolen and Bruce [12] and Oh and Moon [13] have also
Fig. 3. Distribution of different solution types found by 15-ANT MMAS. For
each ρ, the algorithm is terminated after having found 2383 solutions.
employed evolutionary algorithms to find matrix multiplication
algorithms of the same complexity as Strassen’s algorithm.
However, there exist 56 variables in the search spaces of their
methods, which is much more than that of our method. By re-
E. Distributions of Different Solution Types Found by 1-ANT ducing the number of variables and together with the combining
MMAS, 15-ANT MMAS, and 30-ANT MMAS Gaussian eliminations, our method is rather fast.
In this subsection, we investigate the distributions of solu- What is more, our method found ten types of solutions of the
tions found by 1-ANT MMAS, 15-ANT MMAS, and 30-ANT same complexity as Strassen’s algorithm, whereas the method
MMAS with different evaporation rates and make a comparison used by Kolen and Bruce found one solution of the same
on the distributions of different solution types found by three complexity as Strassen’s algorithm [12], and the method used
MMAS algorithms. by Oh and Moon found nine types of solutions of the same
For each ρ, three ACO algorithms, namely, 1-ANT MMAS, complexity as Strassen’s algorithm [13].
15-MMAS, and 30-MMAS, are terminated after having For matrix multiplication problem of special interest case
found 2383 solutions, respectively. The numbers of differ- n = 3 and m = 23, by using the method of combining Gaus-
ent solution types found by these algorithms are shown in sian eliminations, we can reduce the number of variables from
Figs. 2–4. 23 ∗ 18 to (23 − 9) ∗ 18. However, we could not find a solution
We see that, with the value of ρ increasing, the numbers for the matrix multiplication problem of the case n = 3 in an
of different solution types became more well distributed. It is acceptable time. This has two reasons. First, the size of search
also shown that, with the value of ρ increasing, the number of space for the case n = 3 and m = 23 is 314∗18 , which is too
solution type 6 decreases, the numbers of solution types 7 and large relative to that of case n = 2 and m = 7. Second, the scale
10 increase, that of type 1 almost remains the same number, etc. of matrix of Gaussian eliminations is too large: The number
2
This may provide hints that three MMAS algorithms are suited of row is 81, and the number of column is ((39 − 1)/2) =
to find solution of type 1 with all values of ρ considered in this 96 845 281. Thus, the case of n = 3 is still a great challenge for
paper, with a low value of ρ, MMAS algorithms tend to find intelligent algorithms.
Authorized licensed use limited to: ST. JOSEPH ENGINEERING COLLEGE MANGALORE. Downloaded on March 09,2025 at 09:56:53 UTC from IEEE Xplore. Restrictions apply.
 ZHOU et al.: ACO WITH COMBINING GAUSSIAN ELIMINATIONS FOR MULTIPLICATION 357

ACKNOWLEDGMENT Yuren Zhou received the B.Sc. degree in mathemat-

ics from Peking University, Beijing, China, in 1988
The authors would like to thank the anonymous reviewers and the M.Sc. degree in mathematics and the Ph.D.
and the Associate Editor for their valuable comments and degree in computer science from Wuhan University,
Wuhan, China, in 1991 and 2003, respectively.
suggestions that help improve this paper. He is currently a Professor with the School of
Computer Science and Engineering, South China
University of Technology, Guangzhou, China. His
R EFERENCES current research interests are focused on design and
[1] J.M Landsberg, “Geometry and the complexity of matrix multiplication,” analysis of algorithms, evolutionary computation,
Bull. Amer. Math. Soc., vol. 45, no. 2, pp. 247–284, Jan. 2008. and data mining.
[2] V. Strassen, “Gaussian elimination is not optimal,” Numer. Math, vol. 13,
no. 4, pp. 354–356, Aug. 1969.
[3] S. Winograd, “On multiplication of 2 × 2 matrices,” Linear Algebra
Appl., vol. 4, no. 4, pp. 381–388, May 1971.
[4] J. Hopcroft and L. Kerr, “Some techniques for proving certain simple Xinsheng Lai received the M.S. degree in com-
programs optimal,” in Proc. IEEE 10th Annu. Symp., Switching Automata, puter applied technology from Guizhou University,
Oct. 1969, pp. 36–45. Guiyang, China, in 2004. He is currently working
[5] N. H. Bshouty, “On the additive complexity of 2 × 2 matrix multiplica- toward the Ph.D. degree at South China University
tion,” Inf. Process. Lett., vol. 56, no. 6, pp. 329–335, Dec. 1995. of Technology, Guangzhou, China.
[6] H. Cohn and C. Umans, “A group-theoretic approach to fast matrix mul- He is also with the School of Mathematics and
tiplication,” in Proc. 44th IEEE ASFCS, 2003, pp. 438–449. Computer Science, Shangrao Normal University,
[7] H. Cohn, R. Kleinberg, B. Szegedy, and C. Umans, “Group-theoretic Shangrao, China. His main research interests include
algorithms for matrix multiplication,” in Proc. 46th IEEE ASFCS, 2005, evolutionary computation and neural computation
pp. 23–25. and their applications in real world.
[8] D. Coppersmith and S. Winograd, “Matrix multiplication via arithmetic
progression,” J. Symbol. Comput., vol. 9, no. 3, pp. 251–280, Mar. 1990.
[9] J. Laderman, “A non-commutative algorithm for multiplying 3 × 3 ma-
trices using 23 multiplications,” Bull. Amer. Math. Soc, vol. 82, no. 1,
pp. 126–128, Jan. 1976. Yuanxiang Li was born in 1962. He received the
[10] M. Blaser, “On the complexity of the multiplications of matrices of small Ph.D. degree from the Department of Computer
formats,” J. Complexity, vol. 19, no. 1, pp. 43–60, Feb. 2003. Science, Wuhan University (WHU), Wuhan, China,
[11] R. P. Brent, “Algorithms for Matrix Multiplication,” Comput. Sci. Dept. in 1993.
Stanford Univ., Stanford, CA, Tech. Rep. CS 157, Mar. 1970. He is currently a Professor of computer science
[12] J. F. Kolen and P. Bruce, “Evolutionary search for matrix multiplication with WHU. His research fields are parallel comput-
algorithms,” in Proc. 14th Int. Florida Artif. Intell. Res. Soc. Conf., 2001, ing, evolutionary computation, and cellular automata
pp. 161–165. modeling for complex systems.
[13] S. Oh and B. R. Moon, “Automatic reproduction of a genius algorithm: Dr. Li was the recipient of the National Nat-
Strassen’s algorithm revisited by genetic search,” IEEE Trans. Evol. ural Science Prize of China for the project
Comput., vol. 14, no. 2, pp. 246–251, Apr. 2010. “Asynchronous Parallel Algorithms and Domain De-
[14] M. Dorigo and T. Stützle, Ant Colony Optimization. Cambridge, MA: compositions” and the Advanced Prize of Science and Technology of the
MIT Press, 2004. National Educational Ministry for the project “Parallel Computational Models
[15] W. J. Gutjahr, “A graph-based ant system and its convergence,” Future and Algorithms for Simulating Complex Systems.”
Gener. Comput. Syst., vol. 16, no. 9, pp. 873–888, Jun. 2000.
[16] T. Stützle and H. H. Hoos, “Max–min ant system,” Future Gener. Comput.
Syst., vol. 16, no. 8, pp. 889–914, Jun. 2000.
[17] T. Friedrich, N. Hebbinghaus, and F. Neumann, “Comparison of sim-
ple diversity mechanisms on plateau functions,” Theor. Comput. Sci., Wenyong Dong received B.S. degree in computer
vol. 410, no. 26, pp. 2455–2462, Jun. 2009. engineering from Wuhan University of Science and
[18] T. Jansen and I. Wegener, “Evolutionary algorithms: How to cope with Technology, Wuhan, China, in 1996 and the M.S. and
plateaus of constant fitness and when to reject strings of the same fitness,” Ph.D. degrees in computer science from the Wuhan
IEEE Trans. Evol. Comput., vol. 5, no. 6, pp. 589–599, Dec. 2001. University (WHU), Wuhan, in 1999 and 2002,
[19] F. Neumann, D. Sudholt, and C. Witt, “Analysis of different MMAS ACO respectively.
algorithms on unimodal functions and plateaus,” Swarm Intell., vol. 3, He is a Professor with the Department of Com-
no. 1, pp. 35–68, Mar. 2009. puter Science, WHU. He has published more than
[20] T. Kötzing, F. Neumann, D. Sudholt, and M. Wagner, “Simple max–min 20 papers, two books on mathematic modeling,
ant systems and the optimization of linear pseudo-Boolean functions,” in simulation optimization, intelligent computing, ma-
Proc. Int. Workshop FOGA, 2011, pp. 209–218. chine learning, and distributed systems.

Authorized licensed use limited to: ST. JOSEPH ENGINEERING COLLEGE MANGALORE. Downloaded on March 09,2025 at 09:56:53 UTC from IEEE Xplore. Restrictions apply.