Dynamic Programming

Matrix-Chain Multiplication Problem

AKC Notes
Matrix-Chain Multiplication Problem
Given a chain < A1 , A2 , . . . , An > of n matrices, where for i = 1, 2, . . . , n, matrix Ai has dimension pi−1 × pi ,
fully parenthesize the product A1 A2 . . . An in a way that minimizes the number of scalar multiplications.
First we show that exhaustively checking all possible parenthesizations leads to exponential growth of
computation. Denote the number of alternative parenthesizations of a sequence of n matrices by P (n). When
n = 1, we have just one way to fully parenthesize one matrix. When n ≥ 2, a fully parenthesized matrix
product may be splited into two fully parenthesized matrix subproducts, between the kth and (k + 1)st

s
matrices for some k = 1, 2, . . . , n − 1. Thus, we obtain the recurrence
(
1 if n = 1
P (n) = Pn−1

te
k=1 P (k)P (n − k) if n ≥ 2
We show that P (n) ≥ 2n−2 = Ω(2n ). Clearly, the claim is true for n = 1, 2. Assume that the claim is true
for each k < n.
n−1
X
P (n) = P (k)P (n − k)
No
k=1
≥ P (1)P (n − 1) + P (n − 1)P (1)
= 2P (1)P (n − 1)
= 2P (n − 1)
≥ 2 · 2n−1−2
= 2n−2
n
In fact, P (n) = Ω( n43/2 ).
C

Methodology
AK

(1) Characterize the Structure of an Optimal Solution.

Claim 1 Suppose that in and optimal way to parenthesize Ai Ai+1 . . . Aj , we split the product between Ak and
Ak+1 . Then the way we parenthesize the prefix subchain Ai Ai+1 . . . Ak within this optimal parenthesization
of Ai Ai+1 . . . Aj must be an optimal parenthesization of Ai Ai+1 . . . Ak . And the way we parenthesize the
suffix subchain Ak+1 Ak+2 . . . Aj within this optimal parenthesization of Ai Ai+1 . . . Aj must be an optimal
parenthesization of Ak+1 Ak+2 . . . Aj .
prefix subchain suffix subchain
((Ai Ai+1 . . . Ak )(Ak+1 Ak+2 . . . Aj ))

Proof: By contradiction, suppose there were a less costly way to parenthesize Ai Ai+1 . . . Ak , then the
optimal parenthesization of Ai Ai+1 . . . Aj could be replaced with this parenthesization, yielding another
way to parenthesize Ai Ai+1 . . . Aj whose cost was lower than the optimum: a contradiction. An identical
argument applies to the subchain Ak+1 Ak+2 . . . Aj in the optimal parenthesization of Ai Ai+1 . . . Aj . ✷

1 AKC Notes
(2) Recursively Define the Value of the Optimal Solution. First, we define in English the quan-
tity we shall later define recursively. Let Ai..j be the matrix that results from evaluating the product
Ai Ai+1 . . . Aj , where i ≤ j. Let C[i, j] be the minimum number of scalar multiplications needed to compute
the matrix Ai...j . If i = j, no scalar multiplications are needed. Thus, C[i, i] = 0 for i = 1, 2, . . . , n. When
i < j, we assume that in an optimal parenthesization, the product Ai Ai+1 . . . Aj is split between Ak and
Ak+1 , where i ≤ k < j. Then C[i, j] = C[i, k] + C[k + 1, j] + pi−1 pk pj . We don’t know the value of k in the
optimal parenthesization, but there are only j − i possible values for k, namely k = i, i + 1, . . . , j + 1. So we
should check all these values to find the best. We thus have the following recurrence.
Claim 2 (
0 if i = j,
C[i, j] =
mini≤k<j {C[i, k] + C[k + 1, j] + pi−1 pk pj } if i < j.

Proof: The correctness of this recursive definition is embodied in the paragraph which precedes it. ✷

s
(3) Compute the Value of the Optimal Solution Bottom-up. Consider the following piece os pseu-
docode, where p =< p0 , p1 , . . . , pn >, with p.length = n + 1, table C[1..n, 1..n] stores the costs and table

te
S[1..n − 1, 2..n] stores which index of k achieved the optimal cost in computing C[i, j].
The way we fill in the tables C and S here is more tricky than that in the knapsack problem and
checkboard problem. In knapsack problem and checkboard problem, we fill in the tables by rows. Here, the
recurrence formula shows that the cost C[i, j] of computing a matrix-chain product of j − i + 1 matrices
depends only on the costs of computing matrix-chain products of fewer than j − i + 1 matrices. Thus, the
tables C and S should be filled in by increasing lengths of the matrix chains. This corresponds to filling in
No
the tables diagonally.

matrix-chain-order(p)
1 n = p.length − 1
2 let C[1..n, 1..n] and S[1..n − 1, 2..n] be new tables
3 for i = 1 to n
4 C[i, i] = 0
5 for l = 2 to n //l is the chain length
6 for i = 1 to n − l + 1
C

7 j =i+l−1
8 C[i, j] = 0
9 for k = i to j − 1
AK

10 q = C[i, k] + C[k + 1, j] + pi−1 pk pj

11 if q < C[i, j]
12 C[i, j] = q
13 S[i, j] = k
14 return C and S

cations needed to compute the matrix Ai...j , and S[i, j] will contain the index of k achieved the optimal cost
in computing C[i, j].

Proof:
definition given above. The base case is properly handled in Lines 3-4, and the recursive case is properly
handled in Lines 5-13. At each step, the C[i, j] cost computed in lines 10-13 depends only on table entries
C[i, k] and C[k + 1, j] already computed. Lines 8-12 correctly compute mini≤k<j {C[i, k] + C[k + 1, j] +
pi−1 pk pj }, and C[i, j] is set to this value in Line 12. Lines 8-13 correctly compute arg mini≤k<j {C[i, k] +

2 AKC Notes
C[k + 1, j] + pi−1 pk pj }, and S[i, j] is set to this value in Line 13. ✷

Figure 1 illustrates this procedure on a chain of n = 6 matrices.

(4) Construct the Optimal Solution from the Computed Information. Consider the following
piece of pseudocode, where S is the table computed by MATRIX-CHAIN-ORDER.

Print-Optimal-Parens(S, i, j)
1 if i == j
2 print “A”i
3 else print “(”
4 Print-Optimal-Parens(S, i, S[i, j])
5 Print-Optimal-Parens(S, S[i, j] + 1, j)
6 print “)”

s
Claim 4 The above procedure correctly prints an optimal parenthesization of < Ai , Ai+1 , . . . , Aj >.

te
Proof: S[i, j] indicates the value of k such that an optimal parenthesization of Ai Ai+1 . . . Aj splits the
product between Ak and Ak+1 . The above procedure just recursively splits the parenthesization of a chain
into the parenthesization of its prefix chain and the parenthesization of its suffix chain. ✷

(5) Running Time and Space Requirements. The procedure MATRIX-CHAIN-ORDER runs in
No
O(n3 ) due to the nested loop defined in Lines 5, 6 and 9. We can also show that the running time of this
algorithm is in fact also Ω(n3 ) (exercise). The algorithm requires Θ(n2 ) space to store the C and S tables.
The procedure PRINT-OPTIMAL-PARENS runs in Θ(n) time and uses no additional space. The overall
running time is Θ(n3 ) and the space requirement is Θ(n2 ).
C
AK

3 AKC Notes