International Conference on Software Engineering and Computer Science (ICSECS2013)

Analysis of the Time Complexity of Strassen Algorithm

Xiang Wang
School of Information and Electronic Engineering
Tianjin Vocational Institute
Tianjin, China
e-mail: [email protected]

Abstract—Algorithms of matrix multiplication are widely used The function of matrix multiplication is as follows:
in software engineering field. Application of Strassen template<class Type>
algorithm makes a significant contribution to optimize the Type M (Type a[][N], Type b[][N],int m, int n, int p)
algorithm . Therefore, thorough study based on time
{
complexity of matrix multiplication algorithm is very
important. This paper talks about the time complexity of
Type c[][N];
Strassen algorithm and general algorithm for matrix for(int k=0;k<m;k++ )
multiplication, and makes a comparison between the two for(int u=0;u<n;u++ )
algorithm routines so as to discuss the advantages and for(int v=0;v<p;v++)
disadvantages of Strassen algorithm. The rational utilization c[k][u]=c[k][u]+a[k][v]*b[v][u];
plan of matrix multiplication algorithms is also discussed. return *c;
}
Keywords-algorithms for matrix multiplication; time
complexity; Strassen algorithm
We can obtain the following conclusion very easily:
time complexity of a general algorithm for matrix
I. A GENERAL ALGORITHM FOR MATRIX multiplication depends on the rows and columns of matrix A
MULTIPLICATION and matrix B., and generally we can explain the time
Let A=(aij)m × n be an m × n matrix, and let B=(bij)n × p complexity through the following expression, i.e.,
be an n × p matrix. Given an (m×p) matrix A with r row O(m×n×p). If matrix A and matrix B are both n × n matrices,
partitions and s column partitions and a (p×n) matrix with s A×B is just a special case of matrix multiplication. And if n
row partitions and t column partitions that are compatible is a power of 2, both matrix A and matrix B can be
with the partitions of A, the matrix product C=A×B. partitioned into 4 n/2×n/2 blocks. The partitioned matrix can
generally we can explain the matrix multiplication through then be written as
the following expression, i.e.,
A A12  B B12 
 A11 A12  A1s   B11 B12  B1t  A   11  B   11 
     A21 A22   B21 B22 
A A22  A2 s  B B22  B2t  .
A   21 B   21  A B  A12 B21 A11B12  A12 B22 
        AB   11 11 
A21B12  A22 B22 
    (3)
A
 r1

Ar 2  Ars  
 Bs1 Bs 2  Bst   A21B11  A22 B21
 C11 C12  C1t 
  In this case, if we adopt the general algorithm for matrix
 C 21 C 22  C 2t  multiplication, then we need make eight n/2×n/2 Matrix
AB  
 
Multiplications and four Matrix additions (or matrix
 
  subtractions). Let T(n) be time complexity of matrix
C  multiplication, we can explain the T(n) through the
 r1 C r 2  C rt  (1) following recursive expression, i.e.,

Thus AB can be formed blockwise, yielding as an (m×p)

 1 n 1
matrix with r row partitions and t column partitions. The  2
matrices in your matrix are calculated by multiplying: T ( n)    n   n  (4)
8T   4  n 1

  2 2
s
Cij   Ai Bj and i=1,…,r;j=1,…,t. (2)
and generally we can explain the time complexity
 1
through the following expression, i.e., O(n3).

© 2013. The authors - Published by Atlantis Press 103

 II. ANALYSIS OF TIME COMPLEXITY OF THE STRASSEN 2
 2k   2k 
T(n)  7T   18 
ALGORITHM
Strassen algorithm used for matrix multiplication, which
is invented by the German mathematician Volker Strassen  2   2 
in 1969, is considered as one of the fastest and best matrix
   
 7T 2 k -1  18 2 k -1
2

 77T2   182    182 

multiplication algorithms. We define 7 new matrices:
F1=A11(B12-B22), k -2 k -2 2 k -1 2
F2=(A11+A12)B22,

 7 T2   7  182   182 

F3=(A21+A22)B11,
F4=A22(B21-B11), 2 k -2 k -2 2 k -1 2
F5=(A11+A22)(B11+B22),
F6=(A12-A22)(B21+B22), 
   
 7 2 7T 2 k -3  18 2 k -3
2

 7  182   182 
F7=(A11-A21)(B11+B12)
Thus k -2 2 k -1 2

 F  F  F2  F6 F1  F2   7 T2   7  182 
3 k -3 2 k -3 2

AB   5 4 
 F3  F4 F1  F5  F3  F7 
(5)
 7  182   182 
k -2 2 k -1 2

 
k 1
    7 i  18 2 k -i-1
2
The numbers of n/2×n/2 matrix multiplication may set
back to 7 through the Strassen algorithm, when compared i 0
with the general algorithm for matrix multiplication. This
  7
k 1 i
meantime, there are 18 matrix addition or matrix subtraction. 2
 18 2 k
As n is a power of 2, if let k be nonnegative integer, then n
is k-th power of 2. If k=0, then the time complexity of 2  i 0
i 1 2

matrix multiplication is 1. If k>0, generally we can explain i

 2    
k 1
time complexity of matrix multiplication through the 9 7 k 2
following recursive expression,
2 4 i 0  
k
2 7
n n 1  
7T   18 
i.e., 2 2 (6)
  2k   
9
2
2
  4
7
1
so we can explain the T(n) through the following
4
recursive expression,  67  6 4
k k
Since
 1 n 1
 2
7 k  7 log2 n
T ( n)    n  n
7T   18  n 1

 2 2 log 2 n
i.e., (7)  n logn 7
If n>1, then  n log2 n logn 7
log2 7
2 log2 n
n n
T(n)  7T   18  n log2 n

2 2 (8)  n log2 7

We come next to the analysis of equation(8). We replace  n 2.807355
variable n with 2k.
 n 2.81
and

104
 4 k  2 2 k  2 2 log2 n 1  2k
 
 2  k 2

1 2
2 log2 n 2
n 2
,
 23k  2 2 k
Then, we have
 23 log2 n  2 2 log2 n
 1 n 1
T (n)   2.81  n3  n 2
6n  6n n 1
2
(9) Thus, we have
This meantime, we can explain the time complexity
through the following expression, i.e., O(n2.81). It is evident  1 n 1
that the Strassen algorithm was more efficient than the T ( n)   3 (11)
n  n n 1
2
general algorithm for matrix multiplication. In fact, if n is
not a power of 2, the matrix can be embedded into another
matrix (let the matrix be matrix E) whose number of Let us begin to consider the behavior of a new function:
dimension is a power of 2. In matrix multiplication, if the
dimension of matrix E take minimum value within the
design requirement above is achieved, the dimension will K (n)  (n 3  n 2 )  (n 2.81  6n 2 )
increase twofold at most. Thus, the time complexity of i.e., K (n)  n  n  5n 2
3 2.81
(12)
Strassen algorithm can still be described as O(n2.81).
III. VARIATION TRENDS OF TIME COMPLEXITY OF THE We can easily get the first order derivative and second
STRASSEN ALGORITHM order derivative of the function, such that
For the general algorithm for matrix multiplication, let n
be a power of 2. If let k be nonnegative integer, then n is k-th K (n)  3n 2  2.81n1.81  10n (13)
power of 2, i.e., n=2k. When n>1, we can deeply analyze K (n)  6n  5.0861n 0.81  10 (14)
time complexity of the general algorithm for matrix
multiplication through the following recursive expression,
We can obtain that if n>1, then K (n)  0 and
n n
2
K (n)  0 . This implies that Strassen algorithm is always
T(n)  8T   4  better than the general algorithm for matrix multiplication in
 2 2 (10) terms of time complexity. And the advantage of the Strassen
algorithm will became more obvious with the increase of
We come next to the analysis of equation(10). We dimension of matrix. Strassen algorithm adopts the method
replace variable n with 2k. of recursive algorithms, which is helpful to the
implementation and analysis of the algorithm. However, the
2 method make the algorithm require a great number of
 2k   2k  dynamic two dimensional arrays so as to assign memory
T(n)  8T    4  space. which enhance the time complexity and space
 2   2  complexity when the dimension of matrix is smaller.
   
 8T 2 k 1  4 2 k 1
2 However, the general algorithm for matrix multiplication is
no problem on this aspect. Tests show when the dimension
of matrix is less than 500, this situation will get even worse.
 
k 1
    8i  4 2 k-i-1
2
In order to solve this problem, we can specify a numeric
value (such as 500) for algorithm routine, and we will adopt
i 0
Strassen algorithm or the general algorithm for matrix

 
k 1
k 2 8i multiplication based on the comparisons between the
42
2 
dimension and the numeric value.
i 1 2
i 0 The function of Strassen algorithm is as follows:

 2   2
k 1
k 2 i

i 0

105
 subtraction) to decrease the number of block matrix
template<class Type> multiplication, and adopts divide and conquer algorithm to
void S (int n,Type A[][N],Type B[][N],Type AB[][N]) complete matrix multiplication through recursive function.
{ Such methods can effectively reduce the time complexity of
...// definition of related array matrix multiplication. And Within the algorithm program,
int i,j; the computing process is accomplished by the block
if (n==2) matrixes(A11,A12,B11,B12)that act as middle variables,
...;//the general algorithm for matrix multiplication and recursive function is accomplished by S(n/2,A11,BB,M i)
else (i=1,2,3,4,5,6,7).While we pay particular attention to the
{ obvious advantages of Strassen algorithm, we should notice
for(i=0;i<n/2;i++) that Strassen algorithm need to create a large number of
for(j=0;j<n/2;j++) dynamic two dimensional arrays, which takes more
{ computing time to assign memory space. And, the higher
A11[i][j]=A[i][j]; the order of the matrices, the more memory space Strassen
A12[i][j]=A[i][j+n/2]; algorithm program will require. Experiments indicate the
A21[i][j]=A[i+n/2][j]; algorithm of Strassen is not much faster than the general
A22[i][j]=A[i+n/2][j+n/2]; matrix multiplication algorithm when the order of the
B11[i][j]=B[i][j]; matrices is relatively small (usually n<45). Therefore, it is
B12[i][j]=B[i][j+n/2]; necessary to improve Strassen algorithm through setting a
B21[i][j]=B[i+n/2][j]; boundary for the order of the matrices. When the order of
B22[i][j]=B[i+n/2][j+n/2]; the matrices is bigger than the boundary we adopt Strassen
} algorithm, otherwise we adopt the general matrix
//the matrix is partitioned into 4 blocks. multiplication algorithm.
S (n/2,A11,BB,M1);
...;// BB=B12-B22 REFERENCES
S (n/2,AA,B22,M2); [1] Aho, A.V., Hopcroft, J.E.,and Ullman, J.D.(2007). The Design and
...;//AA=A11+A12 Analysis of Computer Algorithms(Huang, L.P., Wang, D.J., and
Zhang, S., Trans.). Beijing, China: China Machine Press.(1974).
[2] Deng, X.Y.,and Wan, T.T.(2006). Design and Analysis of Algorithms.
Beijing, China: Metallurgical Industry Press.
S (n/2,AA,B11,M3); [3] Wang, X.D.(2001). The Design and Analysis of Computer
...;// AA=A21+A22 Algorithms. Beijing, China: Publishing House of Electronics Industry.
S (n/2,A22,BB,M4); [4] Liu, J.(2003). An Introduction to Computer Algorithm—Techniques
...;// BB=B21-B11 of Design and Analysis. Beijing, China:Science Press.
S (n/2,AA,BB,M5); [5] Su, D.F.,and Zhong, C.(2001).The Design and Analysis of Computer
...;//AA=A11+A22 Algorithms. Beijing, China: Publishing House of Electronics Industry.
...;//BB=B11+B22 [6] E.Horowitz.,and S.Sahni.(1978). Fundamentals of Computer
S (n/2,AA,BB,M6); Algorithms. U.S.A.: Compter Science Press.
...;//AA=A12-A22 [7] E.Horowitz., S.Sahni.,and S.Rajasekaran.(1996). Computer
Algorithms/C++. U.S.A.: Compter Science Press.
...;//BB=B21-B22
[8] T.H.Cormen., C.Leiserson.,and C.Stein.(2001). Introduction to
S (n/2,AA,BB,M7); Algorithms. U.S.A:The MIT Press.
...;//AA=A11-A21
[9] Alfred V.Aho.,John E. Hopcroft., and Jeffrey D.Ullman.(1983). Data
...;//BB=B11-B12 Structures and and Algorithms. U.S.A: Addison-Wesley.
...;//AB11=M5+M4-M2+M6 [10] Sara Baase., and A.Gelder.(2000). Computer Algorithms:Introduction
...;//AB12=M1+M2 to Design and Analysis. U.S.A: Addison-Wesley.
...;//AB21=M3+M4
...;//AB22=M5+M1-M3-M7
for(i=0;i<n/2;i++)
for(j=0;j<n/2;j++)
{
AB[i][j]=AB11[i][j];
AB[i][j+n/2]=AB12[i][j];
AB[i+n/2][j]=AB21[i][j];
AB[i+n/2][j+n/2]=AB22[i][j];
}
//store the result in matrix AB
}
} The program above shows that Strassen algorithm
increases the number of block matrix addition(or

106