Journal of Applied Computer Science & Mathematics, no.

11 (5) /2011, Suceava

Simplified Optimal Parenthesization Scheme for Matrix Chain Multiplication

Problem using Bottom-up Practice in 2-Tree Structure
Biswajit BHOWMIK
Department of Computer Science & Engineering, Indian Institute of Technology Guwahati,
Assam – 781 039, India
[email protected]

Abstract–Dynamic Programming is one of the sledgehammers II. TRADITIONAL APPROACH

of the algorithms craft in optimizations. The versatility of the
dynamic programming method is really appreciated by Let Amxn and Bnxp be two matrices, then their product C =
exposure to a wide variety of applications. In this paper a AB is an mxp matrix. This C can be computed in O(nmp)
modified algorithm is introduced to provide suitable procedure
that ensures how to multiply a chain of matrices. The model
time, using very traditional elegant matrix multiplication
Optimal Parenthesization Scheme using 2-Tree Generation scheme [7][8] that runs in O(n3) time. For more than two
(OPS2TG) acts as one relevancies of the proposed algorithm. A matrices say X, Y, Z conforming matrix multiplication
new approach for how to break the matrix chain and how to condition, the purpose can be achieved in two ways as either
insert parenthesis in it is also designed in this model. The ((XY)Z) or (X(YZ)) as matrix multiplication is associative.
comparison study between the proposed model and the classical Similarly, for four matrices say A, B, C, and D, they may be
approach shows acceptance of the model suggested here. multiplied either of the orders: (((AB)C)D), ((AB)(CD)),
(A((BC)D)), ((A(BC))D), or (A(B(CD))) [7][9]. In all these
Keywords: Scalar Multiplication, Optimal Parenthesization, cases however multiplication doesn’t affect the final
Chain Matrix, Solution Matrix, Optimal Cost, Sequence of
Decisions, Optimal Solution.
outcome. But the choice can affect the running time to
compute it [6][10]. If the matrices are of different
I. INTRODUCTION dimensions, the number of operations termed as scalar
multiplications (which makes a huge difference in costs) is
Dynamic Programming (DP) is one of the elegant affected by the order in which the product is computed [10].
algorithm design standards and is a powerful tool which This computation for an optimal order which minimizes the
yields classic algorithms for a variety of combinatorial total number of scalar multiplication operations depends fully
optimization problems such as shortest path problems, on parenthesizing the chain of the matrices. Thus, it
traveling salesman problem, knapsack problem, etc including minimizes the number of scalar multiplications resulting
underlying Matrix Chain Multiplication (MCM) problem. minimum cost [11].
The few past decades throughout many "fast" algorithms Now, a sequence of n matrices requires n-1
have been proposed for matrix multiplications [1]. Some of multiplications. The cost of multiplying this chain obviously
them are acceptable to some extent. Some of them even are in depends on the order in which the n-1 multiplications are
use till now. In this paper an alternative approach has been carried out. The rearrangement which results out the
described extending a similar approach [2][3][4]. The minimum number of scalar multiplications can be found in
proposed is named Optimal Parenthesization Scheme using many ways. In the sequence there are n-1 places where each
2-Tree Generation (OPS2TG) model. parenthesization defines a set of n-1 matrix multiplications.
The paper is organized in the following way. First, a The sequence is segmented with the outermost pair of
general insight is provided into the traditional approach with parentheses as just after the 1st matrix, just after the 2nd
a simple example. Second, a formal modification of the matrix… and just after the (n-1)th matrix. In this way we are
existing algorithm is given to show computation of cost of able to pick the best parenthesized matrix order [2][4][7][12].
scalar multiplications for the chain of matrices conformable A question may arise in mind about how many
to multiplication in tabular form and enhances constructing parenthesization are possible? Eqn. (1) gives the answer.
solution table under certainty [2][5][6][7]. Third our re- In general, the number of orderings is equal to the number
devised algorithm is stated that provides us simple solution of ways to place parentheses to multiply the n matrices in
for optimal parenthesization of matrices in the chain. The every possible way. Let us suppose P(n) be the number of to
paper ends with comparison study between traditional fully parenthesize a product of n matrices. Suppose we want
approach, dynamic programming approach [2][[7], and our to perform the multiplication (MiM2…Mk)(Mk+iMk+2…
proposed approach. Mn). Then, there are P(k) ways to parenthesize the first k
matrices. For each one of the P(k) ways, there are P(n-k)

  9
 Computer Science Section
 

ways to parenthesize the remaining n-k matrices, thus for a m[i][j]= m[ i ][ k ] + m[ k + 1 ][ j ] + Pi−1PkPj .
total of P(k)P(n - k) ways. Since k can be assumed any value Thus, m[i][j] could be represented by the recurrence as:
between 1 and n – 1, the overall number of ways to
parenthesize the n matrices is given by the summation: ⎧ 0, i = j

( )
⎧ 1, n = 1 ⎪
⎪ n −1 ⎪ ⎧ m[i][k ] +
= Ω 4 3 / 2        (1)
n
P (n) = ⎨ m[i][ j ] = ⎨ ⎪ (2)
⎪⎩ ∑
P ( k ) P ( n − k ), n ≥ 2 n min i≤k < j ⎨ m[ k + 1][ j ] + if i < j ..
⎪
k =1
⎪ ⎪ Pi −1 Pk Pj
(Using Catalan numbers and Stirlings’ formula)[2][4]. ⎩ ⎩
It will be heuristics to find the optimal solution that
computes all the rearrangements. Both the number of B. Computation of Scalar Multiplications
solutions and the brute-force method of exhaustive searching The value at m[1][n] is our desired optimal number of
as expected will become a poor tactic to determine optimal scalar multiplications. But the computational procedure of
parenthesization in the matrix chain [2][13]. m[1][n] for M1M2………Mn using m[i][j], takes O(n2) time
which is not better than brute force method of checking each
III. PROPOSED MODEL way of parenthesizing the product [1][2]. For improvement,
instead of computing m[i][j] recursively we construct this
Dynamic programming [14] is one of the suitable table m shown in Table II using modified algorithm
standards and powerful technique for optimal matrix OSM_MCM() that goes after bottom up approach at stages in
parenthesization, which is discussed in detail in the proposed computation. At the same time another table S[1…n][1…n]
OPS2TG model. The results and their analysis reveal called “solution matrix” is constructed. The S[i][j] keeps the
considerable amount of time reduction compared with the position of parenthesization (inserting parenthesis in chain) k
existing algorithms [2][9]. Efficiency for optimal solution can in the chain. This table S shown in Table III is used for
be achieved through: finding optimal solution.
(a) Recursive definition for cost of subproblems
computation. Algorithm: OSM_MCM(C)
(b) Computing Optimal cost of scalar multiplication using Description: OSM_MCM means Optimal Scalar
bottom-up. Multiplication for MCM.
(c) Generating Tree structure for the chain. Input: C = matrix chain
(d) The optimal parenthesization using bottom-up. n = number of matrices.
P = i/p sequence represents matrices dimension.
A. Formulating the Problem Output: Tables m and S.
DP approach is basically a potent algorithmic stage wise
search method of optimization problems whose solutions (1) Repeat for i = 1 to n // fill diagonal of m
may be viewed as the result of a sequence of decisions i.e. (2) m[i][i] = 0
solutions that recursively solve sub-problems which overlap (3) Initialize x = 2
[7][15]. Naturally the number of scalar multiplications is (4) Repeat for i = 1 to n – 1 // fill m[i][j] when i < j
computed recursively in terms of the optimal solutions to (5) Initialize j = x
subproblems. The partial solution is stored in a table m (6) Repeat while j ≤ n
namely “cost multiplication” table. The result is then fed to (7) m[i][j] = ∞
derive optimal cost of solution. (8) x=x+1
Let us consider M1M2…….MiMi+1……… Mn a given (9) Initialize x = 2
chain of n matrices and Mij, 1≤i≤ j≤n, is the expected (10) Repeat while x ≤ n through line 14
resultant matrix. Its cost can be obtained as: Suppose m[i][j] (11) Initialize
is the minimum number of scalar multiplications needed to (12) i=1
find Mij, and P is a list that contains dimension values of the (13) j=x
matrices in chain. Here Pi−1xPi represents dimension of Mi (14) Repeat while (i ≤ n) through line 19
matrix. Now, m[i][j] can recursively be defined at the same (15) If j ≤ n
time like below: (16) Repeat for k = i to j – 1 //check all possible splits
(a) If i = j, m[i][j] = 0. The problem becomes trivial. No (17) m[i][j]=min(m[i][j],m[i][k]+m[k+1][j]+Pi-PkPj)
need of scalar multiplication. (18) S[i][ j] = k
(b) If i < j, we assume that the optimal parenthesization (19) Initialize j = j + 1
splits MiMi+1…..Mj between Mk and Mk+1, i ≤ k < j. (20) Output “Cost of multiplication” = m[1][n]
Then m[i][j] would be a minimum cost of computation of (21) Exit.
the sub problems Mik and Mk+1j + The cost of multiplying
these two matrices together. i.e. Complexity Analysis: Every entry in the table m is 0 along

  10
 Journal of Applied Computer Science & Mathematics, no. 11 (5) /2011, Suceava

the main diagonal and unknown value (∞) in the rest of cells represented by (S, i, j). The leaves are the matrices of the
above this diagonal shown in Table I(a). It is seen that only chain. Thus the leaf (S, i, i) represents a matrix Mi.
the upper triangular matrix [16] is sufficient to find cost of
scalar multiplication. The modified structure of m is shown (1) Initialize
in Table I(b). The m in Table I is constructed for n = 5. Thus (2) i=1
lines 1 to 8 halve both time and space complexities to (3) j=n
construct the table m. And, it is Θ(n2). Now lines 10 and 14 (4) Construct root node of the 2-tree
iterate at most n-1 and n times respectively whereas looping (5) R = (S, i, j) // R = current root node of the tree
varies in line 16. Although the order is cubic but it would be (6) k = S[i][j]
Θ(n3). Thus, finding these time and space complexities of (7) Recursively construct children nodes of R in the 2-
Algorithm OSM_MCM is straightforward. For some constant tree at tentative k.
c > 0, the running time T(n) of the algorithm is proportional (8) Lchild = TT_MCM(S, i, k) // Left child node
to: (9) Rchild = TT_MCM(S, k+1, j) // Right child node
n n n− x
cn3 − cn (10) Exit.
T (n) = ∑∑∑ = Θ(n3 )         … … (3) Complexity Analysis: The algorithm generates a 2-tree
x=2 i =1 k =1 10
through recursion. Complexity of any binary tree
TABLE I: INITIAL CONFIGURATION OF M. construction will be applicable undoubtedly. Thus the
running time will be Ω(nlogn). The generic view of a 2-tree
shown in Fig. 1 is generated by the algorithm TT_MCM().
The node (S, i, j) represents multiplication of the chain
MixMi+1x……Mj

D. Optimal Parenthesization
This section describes how a parenthesis is inserted in the
chain i.e. a chain of matrices is parenthesized. The algorithm
given below inserts parenthesis in the 2-tree generated in
(a)
previous algorithm in bottom-up fashion.
Algorithm: OP_MC(T) // OP_MC means Optimal
Parenthesization on Matrix Chain

Input: The generated 2-tree

Output: Parenthesized matrix chain
(1) Repeat for i = 1 to n
(2) Replace the leaf (S, i, i) by Mi
(3) Repeat for x = last level h to root level // bottom up
(b) fashion.
(4) Parenthesize all leaves say A, B at x with same
Similarly space complexity for S can be put up as well. parent as (AB) from left to right.
n n −i
cn2 − cn (5) h = h -1
∑∑
i =1 j =1 5
= Θ(n2 )       (4) (6) Take (AB) as new leaf at level h in place of parent
node of A, and B.
(7) Exit.
C. Construction of Tree Structure
The algorithm that directly shows the next step after the Complexity Analysis: It is clearly observed that
computation of most favorable scalar multiplications how to complexity of this algorithm is primarily due to
construct an optimal solution with the help of the table parenthesizing the contemporary leaves climbing up the tree.
S[1..n][1..n] is described in this section. The algorithm The parenthesization starts at lowest level h and ends at root
generates a 2-tree [17] as the output when executes. level. Parenthesizing these leaves at every level requires
constant time. Thus the running time of the algorithm is O(h)
Algorithm: TT_MCM(S, 1, n) // TT_MCM means 2- = O(logn).
Tree generation for MCM
Input: S = nxn matrix obtained in the previous algorithm.
Output: Generated 2-Tree.
Description: the algorithm generates a 2-tree based on the
solution matrix S. All the internal nodes including root are

  11
 Computer Science Section
 

S[3][5] = 3

m[1][4] = 38
S[1][4] = 3

m[2][5] = 38
S[2][5] = 2
⎧ m [1][1] + m [ 2 ][ 5 ] + P0 P1 P5
⎪ m [1][ 2 ] + m [ 3 ][ 5 ] +
⎪ P0 P2 P5
m [1][ 5 ] = min ⎨
⎪ m [1][ 3 ] + m [ 4 ][ 5 ] + P0 P3 P5
Fig. 1: Typical 2-Tree for Optimal Parenthesization
⎪⎩ m [1][ 4 ] + m [ 5 ][ 5 ] + P0 P4 P5
IV. NUMERICAL RESULTS = 40
S[1][5] = 4
Suppose, the chain of 5 matrices <M1, M2, M3, M4, M5>
be ABCDE. Their dimensions are 1x5, 5x4, 4x3, 3x2, and 2x1 Thus, the tables’ m and S can be represented as in Table II
respectively. Going through the algorithm OSM_MCM() the and Table III respectively.
optimal scalar multiplications can be put in the table m Taking advantage of S, the TT_MCM() procedure
shown in Table II using initial arrangement of m in Table recursively builds the preferred 2-tree similar to fig. 1 and it
I(b). Subsequent solution table S is constructed as shown in is shown in fig. 2.
Table III. The value at m[1][7] cell in Table II incurs cost of Undergoing OP_MC() algorithm on the above tree in Fig.
computing the chain. As a sample, few partial solutions are 2, the chain can be parenthesized as close to Fig. 4 in
derived below. consequence. All the leaves are first replaced by the matrices
shown in Fig. 3, and Fig. 4(d) gives an idea about optimal
P = <P0, P1, P2, P3, P4, P5> = <1, 5, 4, 3, 2, 1>. parenthesized chain.
Exploiting equation (2),
TABLE II: MATRIX M FOR SCALAR MULTIPLICATIONS.
m[1][1] = m[2][2] = …m[5][5] = 0.
m[1][2] = m[1][1]+m[2][2]+P0P1P2 = 0+0+1x5x4 = 20
S[1][2] = k = 1

Similarly,
m[2][3] = P1P2P3 =5x4x3 = 60
S[2][3] = k = 2

m[3][4] = P2P3P4 =4x3x2 = 24 TABLE III: SOLUTION MATRIX S FOR 2-TREE GENERATION.
S[3][4] = k = 3

m[4][5] = P3P4P5 =3x2x1 = 6

S[4][5] = k = 4

⎧ m [1][1] + m [ 2 ][ 3 ] + P0 P1 P3
m [1][ 5 ] = min ⎨
⎩ m [1][ 2 ] + m [ 3 ][ 3 ] + P0 P2 P3
⎧ 0 + 60 + 1 × 5 × 3
= min ⎨
⎩ 20 + 0 + 1 × 4 × 3 V. COMPARISION STUDY
= 32
S[1][3] = k = 2 So far we have demonstrated an algorithm that computes
the optimal cost for multiplying a chain of matrices. It is not
Proceeding in the same way, hard to modify the algorithm so that it will also compute the
m[2][4] = 64 actual order of matrix multiplications [4]. The Table IV
S[2][4] = 2 summarizes the main result.

m[3][5] = 18

  12
 Journal of Applied Computer Science & Mathematics, no. 11 (5) /2011, Suceava

(b)

Fig. 2: 2-Tree for Optimal Parenthesization.

(c)

(d)
Fig. 4: Insertion of Parenthesis in the Chain

TABLE IV: COMPARING COMPLEXITIES FOR DIFFERENT

APPROACHES.
Traditional Classical DP Proposed
Approach Approach Model
Time
Complexity O(n3) Θ(n3) Θ(n3)
Space
Complexity O(n2) Θ(n2) Θ(n2)

Fig. 3: Replacement of Leaves Nodes of 2-tree by Matrices. Although complexities are seemed to be alike still the
proposed approach is better one at least in four respects. First,
a complexity represents asymptotic behavior of the proposed
algorithm. Upon computation exact numerical complexity
value would be lower. Second the approach conveys us well
again towards simple understanding of the problem. Third the
table m can be worked out in easy way. Finally the model
suggested has introduced 2-tree that makes a sense how
contemporary two matrices can be parenthesized in a chain.
Thus in general, proposed model shows simple method to
solve a chain of matrices. Surprisingly, this problem could be
solved in O(nlogn) time by Hu –Shing’s suggested model [4].

VI. CONCLUSION

Matrix Chain Multiplication problem involves the question

how the optimal sequence for performing a series of
(a) operations can be determined. This general class of problem

  13
 Computer Science Section
 

is not only important in compiler design for code [5] David B. Wagner, “Dynamic Programming”, The
optimization and in databases for query optimization but also Mathematica Journal, Miller Freeman Publications, 1995.
in task scheduling and allied applications. The model in fact [6] Phillip G. Bradfordy, Gregory J. E. Rawlinsz, Gregory E.
does not carry out any matrix multiplication rather it shows Shannonx, “Efficient Matrix Chain Ordering in Polylog
Time”, Journal of Computer, SIAM, Vol. 27, No. 2, Pp. 466 -
order of sequence how to multiply. The main contribution to 490, April 1998.
this work is the formalization of the proposed algorithm [7] Heejo Lee, Jong Kim, Sung Je Hong, Sunggu Lee,
which significantly enhances performances improvement “Processor Allocation And Task Scheduling Of Matrix Chain
reducing handful arithmetic operations to the number of Products On Parallel Systems”, IEEE Transactions on Parallel
matrices and the sequence of dimensions. At the same time and Distributed Systems, Vol. 14, No. 4, April 2003.
the practice might turn out to be very straightforward for the [8] Marco Bodrato, “A Strassen-Like Matrix Multiplication
mathematic enthusiasts. Suited for Squaring And Highest Power Computation”,
CIVV, 2008.
ACKNOWLEDGEMENT [9] Muhammad Hafeez, Muhammad Younus, “An Effective
Solution for Matrix Parenthesization Problem through
Parallelization”, International Journal of Computers, Issue 1,
The author would like to convey his heartily gratitude to Vol. 1, 2007.
his beloved Madhurima Mondal, Amei Rai, and their friends, [10] T. C. Hu, M. T. Shirig, “Computation of Matrix Chain
B.Tech, 3rd Year Students, Department of Computer Science Products”, Stanford University, September 1981.
& Engineering, Bengal College of Engineering and [11] Cary Cherng, Richard E. Ladner, “Cache Efficient Simple
Technology, Durgapur for providing notes, existing solution Dynamic Programming”, ICAA, France, 2005.
illustrated in Section IV, valuable suggestions, graceful [12] Biswajit Bhowmik, “Design and Analysis of Algorithms”, S.
inspiration towards completion of this draft in time. K. Kataria and Sons, 1st Edition, 2011.
[13] Ting Wang, “Algorithms for Parallel and Sequential
Matrix-Chain Product Problem”, Ohio University, November,
REFERENCES 1997.
[14] Biswajit Bhowmik, “Dynamic Programming – Its Principles,
[1] Nai-Kuan Tsao, “Error Complexity Analysis of Algorithms
Applications, Strengths, and Limitations”, International
for Matrix Multiplication and Matrix Chain Product”, IEEE
Journal of Engineering Science and Technology, Vol. 2(9),
Transactions on Computers, Vol. C-30, No. 10, October 1981.
2010, Pages: 4822–4826.
[2] Thomas H. Coremen, Charles E. Leiserson, Ronald L. Rivest,
[15] Yanhong A. Liu, Scott D. Stoller, “Dynamic
Clifford Stein, “Introduction to Algorithms”, PHI, 2nd Edn,
Programming via Static Incrementalization”, ESOP, 1999.
2008.
[16] A. R. Vasistha, “Matrices”, Krishna Publications.
[3] Ellis Horowitz, Sartaj Sahani, Sanguthevar Rajasekaran,
[17] D. Samanta, “Classic Data Structures”, PHI, 18th printing,
“Computer Algorithms”, University Press, 2nd Edn, 2008.
2010.
[4] M.H. Alsuwaiyel , “Algorithms Design Techniques and
Analysis”, PHEI, 2002.

Biswajit Bhowmik is currently doing research work as a Research Scholar in the Department of Computer Science & Engineering of
Indian Institute of Technology Guwahati, India. Before assuming his present designation he was very renowned faculty member with Bengal
College of Engineering and Technology, Durgapur, India as Assistant Professor in the Department of Computer Science & Engineering last
seven years. He is a member of different professional bodies such as IEEE, IACSIT, IAENG, IAS, IAOE, ISOC, PASS, UACEE etc. He is
also a member of some leading professional societies such as IEEE Computer Society, IEEE Communications Societies, IAENG Society of
Computer Science, IAENG Society of Wireless Networks, IAENG Society of Software Engineering, and IAENG Society of Artificial
Intelligence. He is reviewer of several international journals such as ETASR, IJCSIC, IJCSIS, JACSM, IJoAT, JWMC etc in the area of
computer science. He has authored a book titled Design and Analysis of Algorithms. He has many publications in international journals
including international conference proceedings on the subjects ranging from Algorithms Analysis, Graph Theory, Compiler Design, and
Mobile Computing. In addition his area of interests includes Data Structures & Algorithms, Software Engineering, Computational Geometry,
and Green Computing. He has guided several projects at under graduate level. This paper is based on Dynamic Programming approach as
easy solution tool towards optimal solution of Chain Matrix Multiplication Problem and is for all the readers who feel difficulty in
understanding classical solution procedure of the problem.

  14