Matrix Chain Mul
Matrix Chain Mul
Problem Statement:
Given a chain of n matrices <A1, A2, . . . , An>,
Ai has dimension pi-1 x pi , (i = 1, 2, . . . , n)
Fully Parenthesize the product A1 A2 . . . An in a way that Minimizes the number of
Scalar Multiplications.
Fully Parenthesize:
A product of matrices is Fully Parenthesized if it is either
• A Single Matrix or
• Product of Two Fully Parenthesized Matrix Products, surrounded by
parentheses.
Introduction…
Eg.:
The product A1A2A3A4 of the matrices <A1, A2, A3, A4>, can be Fully
Parenthesized in Five distinct ways:
• (A1(A2 (A3A4)))
• (A1((A2A3) A4))
• ((A1A2)(A3A4))
• ((A1(A2A3))A4)
• (((A1A2)A3)A4)
Multiplication of Two Matrices
MATRIX-MULTIPLY (A, B)
If (A.columns B.rows)
Error “Incompatible Dimensions” Order of A: i x k
Order of B: k x j
Else
Order of C: i x j
Let C be a new A.rows x B.columns Matrix
For (i = 1 to A.rows) Number of Scalar Multiplications:
For (j = 1 to B.columns) k multiplications for each of the
(i * j) entries of C.
cij = 0
=i*k*j
For (k = 1 to A.columns) =ikj
cij = cij + aik . bkj
return C
Cost of Matrix Multiplication
Parenthesizing a chain of matrices can have a dramatic impact on the cost of product
evaluation.
((A1A2) A3)
A1A2 - 10 * 100 * 5 = 5000
((A1A2) A3) – 10 * 5 * 50 = 2500
Eg.:
Total 5000 + 2500
<A1, A2, A3> = 7500 Scalar Multipliations
A1 - 10 x 100
A2 - 100 x 5 (A1 (A2A3))
A3 - 5 x 50 A2A3 – 100 * 5 * 50 = 25000
(A1 (A2A3) – 10 * 100 * 50 = 50000
Total 25000 + 50000
= 75000 Scalar Multiplications
Goal:
To determine an
Order for Multiplying Matrices
that has the
Lowest Cost
Counting the Number of Parenthesizations
Let P(n) - Number of Alternative Parenthesizations of a sequence of n matrices.
• n = 1, Only one matrix:
• Only one way to fully parenthesize the matrix product.
• n > = 2:
• A Fully Parenthesized matrix product is the Product of Two Fully
Parenthesized Matrix Subproducts.
• Split between the two subproducts may occur between the kth and (k + 1)st
matrices for any k = 1, 2, . . . , n - 1.
Optimal Parenthesization of Matrix-Chain
Dynamic-programming method is used to determine how to optimally parenthesize
a matrix chain:
• If (i < j)
Parenthesize Ai Ai+1 . . . Aj by splitting the product between Ak and Ak+1
for i k < j.
→ Compute matrices Ai . . . k and Ak + 1 . . . j and then multiply them together to
produce the final product Ai . . . j.
Table m
• Only the lower half of the
table upto (i, i) gets filled 3 0
i
because 1 i j n 4 0
5 0
6 0
Example…
• l = 2 → 2 Matrices
• Matrices: A1 A2, A2 A3, A3 A4, A4 A5, A5 A6
• No. of positions where split can occur = 1. Therefore, k = 1, 2, 3, 4, 5
respectively.
A2 A 3 A4 :
m [2, 4] = min m [2, 2] + m [3, 4] + p1 p2 p4 = 0 + 750 + 30 x 15 x 10 = 6,000 = 4,375
m [2, 3] + m [4, 4] + p1 p3 p4 = 2625 + 0 + 35 x 05 x 10 = 4,375
s [2, 4] = 3
Example…
A3 A 4 A5 :
m [3, 3] + m [4, 5] + p2 p3 p5 = 0 + 1000 + 15 x 05 x 20 = 2,500
m [3, 5] = min = 2,500
m [3, 4] + m [5, 5] + p2 p4 p5 = 750 + 0 + 15 x 10 x 20 = 3,750
s [3, 5] = 3
A4 A 5 A6 :
m [4, 4] + m [5, 6] + p3 p4 p6 = 0 + 5000 + 05 x 10 x 25 = 6,250
m [4, 6] = min = 3,500
m [4, 5] + m [6, 6] + p3 p5 p6 = 1000 + 0 + 05 x 20 x 25 = 3,500
s [4, 6] = 5
Example…
n = 6; pi = 30, 35, 15, 5, 10, 20, 25; l=3 i = 1 to n - l + 1; j = i + l – 1; k = i to j – 1
q = m [i, k] + m [k + 1, j] + pi-1 pk pj
i=1 i=2
j = 1+3-1 = 3 j = 2+3-1 = 4
k=1 k=2 k=2 k=3
q=m[1,1]+m[2,3]+ p0 p1 p3 q=m[1,2]+m[3,3]+ p0 p2 p3 q=m[2,2]+m[3,4]+ p1 p2 p4 q=m[2,3]+m[4,4]+ p1 p3 p4
=0+2625+30x35x5 =15750+0+30x15x5 =0+750+35x15x10 =2625+0+35x5x10
q = 7,875 q = 18,000 q = 6,000 q = 4,375
m [1, 3] = min (7875, 18000) = 7,875 m [2, 4] = min (6000, 4375) = 4,375
s [1, 3] = 1 s [2,4] = 3
i=3 i=4
j = 3+3-1 = 5 j = 4+3-1 = 6
k=3 k=4 k=4 k=5
q=m[3,3]+m[4,5]+ p2 p3 p5 q=m[3,4]+m[5,5]+ p2 p4 p5 q=m[4,4]+m[5,6]+ p3 p4 p6 q=m[4,5]+m[6,6]+ p3 p5 p6
=0+1000+15x5x20 =750+0+15x10x20 =0+5000+5x10x25 =1000+0+5x20x25
q = 2,500 q = 3,750 q = 6,250 q = 3,500
m [3, 5] = min (2500, 3750) = 2,500 m [4, 6] = min (6250, 3500) = 3,500
s [3, 5] = 3 s [4, 6] = 5
j Example…
1 2 3 4 5 6
1 0 15,750 7,875
2 0 2,625 4,375
i 3 0 750 2,500
4 0 1,000 3,500
5 0 5,000
6 0 j
Table m 1 2 3 4 5 6
1 1 1
2 2 3
i
3 3 3
4 4 5
5 5
Table s
Example…
• l = 4 → 4 Matrices
• Matrices: A1 A2 A3 A4, A2 A3 A4 A5, A3 A4 A5 A6
• No. of positions where split can occur = 3.
Therefore, k = (1, 2, 3), (2, 3, 4) (3, 4, 5) respectively.
A 2 A 3 A4 A 5 :
m [2, 2] + m [3, 5] + p1 p2 p5 = 0 + 2500 + 35 x 15 x 20 = 13,000
m [2, 5] = min m [2, 3] + m [4, 5] + p1 p3 p5 = 2625 + 1000 + 35 x 5 x 20 = 7,125 = 7,125
m [2, 4] + m [5, 5] + p1 p4 p5 = 4375 + 0 + 30 x 10 x 20 = 11,375
s [2, 5] = 3
A3 A4 A5 A6 :
m [3, 3] + m [4, 6] + p2 p3 p6 = 0 + 3500 + 15 x 5 x 25 = 5,375
m [3, 6] = min m [3, 4] + m [5, 6] + p2 p4 p6 = 750 + 3500 + 15 x 10 x 25 = 8,000 = 5,375
m [3, 5] + m [6, 6] + p2 p5 p6 = 2500 + 0 + 15 x 20 x 25 = 10,000
s [3, 6] = 3
Example…
n = 6; pi = 30, 35, 15, 5, 10, 20, 25; l=4 i = 1 to n - l + 1; j = i + l – 1; k = i to j – 1
q = m [i, k] + m [k + 1, j] + pi-1 pk pj
i = 1; j = 1+4-1 = 4
k=1 k=2 k=3
q=m[1,1]+m[2,4]+ p0 p1 p4 q=m[1,2]+m[3,4]+ p0 p2 p4 q=m[1,3]+m[4,4]+ p0 p3 p4
=0+4375+30x35x10 = 14,875 =15750+750+30x15x10 = 21,000 =7875+0+30x5x10 = 9,375
m [1, 4] = min (14875, 21000, 9375) = 9,375; s [1, 4] = 3
i = 2; j = 2+4-1 = 5
k=2 k=3 k=4
q=m[2,2]+m[3,5]+ p1 p2 p5 q=m[2,3]+m[4,5]+ p1 p3 p5 q=m[2,4]+m[5,5]+ p1 p4 p5
=0+2500+35x15x20 = 13,000 =2625+1000+35x5x20 = 7,125 =4375+0+35x10x20 = 11,375
m [2, 5] = min (13000, 7125, 11375) = 7,125; s [2, 5] = 3
i = 3; j = 3+4-1 = 6
k=3 k=4 k=5
q=m[3,3]+m[4,6]+ p2 p3 p6 q=m[3,4]+m[5,6]+ p2 p4 p6 q=m[3,5]+m[6,6]+ p2 p5 p6
=0+3500+15x5x25 = 5,375 =750+3500+15x10x25 = 8,000 =2500+0+15x20x25 = 10,000
m [3, 6] = min (5375, 8000, 10000) = 5,375; s [3, 6] = 3
j Example…
1 2 3 4 5 6
1 0 15,750 7,875 9,375
2 0 2,625 4,375 7,125
i 3 0 750 2,500 5,375
4 0 1,000 3,500
5 0 5,000
6 0 j
Table m 1 2 3 4 5 6
1 1 1 3
2 2 3 3
i
3 3 3 3
4 4 5
5 5
Table s
Example…
• l = 5 → 5 Matrices
• Matrices: A1 A2 A3 A4 A5, A2 A3 A4 A5 A6
• No. of positions where split can occur = 4.
Therefore, k = (1, 2, 3, 4), (2, 3, 4, 5) respectively.
• Number of different parenthesizations:
A 1 A2 A 3 A4 A5 :
m [1, 1] + m [2, 5] + p0 p1 p5 = 0 + 7175 + 30 x 35 x 20 = 28,125
m [1, 2] + m [3, 5] + p0 p2 p5 = 15750 + 2500 + 30 x 15 x 20 = 27,250
m [1, 5] = min = 11,875
m [1, 3] + m [4, 5] + p0 p3 p5 = 7875 + 1000 + 30 x 5 x 20 = 11,875
m [1, 4] + m [5, 5] + p0 p4 p5 = 9375 + 0 + 35 x 10 x 20 = 16,375
s [1, 5] = 3
A2 A3 A4 A5 A6 :
m [2, 2] + m [3, 6] + p1 p2 p6 = 0 + 5375 + 35 x 15 x 25 = 18,500
m [2, 3] + m [4, 6] + p2 p3 p6 = 2625 + 3500 + 15 x 5 x 25 = 8,000
m [2, 6] = min m [2, 4] + m [5, 6] + p p p = 4375 + 5000 + 35 x 10 x 25 = 18,125 = 8,000
1 4 6
m [2, 5] + m [6, 6] + p1 p5 p6 = 7125 + 0 + 35 x 20 x 25 = 24,625
s [2, 6] = 3
Example…
n = 6; pi = 30, 35, 15, 5, 10, 20, 25; l=5 i = 1 to n - l + 1; j = i + l – 1; k = i to j – 1
q = m [i, k] + m [k + 1, j] + pi-1 pk pj
i=1
j = 1+5-1 = 5
k=1 k=2 k=3 k=4
q=m[1,1]+m[2,5]+ p0 p1 p5 q=m[1,2]+m[3,5]+ p0 p2 p5 q=m[1,3]+m[4,5]+ p0 p3 p5 q=m[1,4]+m[5,5]+ p0 p4 p5
=0+7125+30x35x20 =15750+2500+30x15x20 =7875+1000+30x5x20 =9375+0+35x10x20
q = 28,125 q = 27,250 q = 11,875 q = 16,375
m [1, 5] = min (28125, 27250, 11875, 16375) = 11,875
s [1, 5] = 3
i=2
j = 2+5-1 = 6
k=2 k=3 k=4 k=5
q=m[2,2]+m[3,6]+ p1 p2 p6 q=m[2,3]+m[4,6]+ p2 p3 p6 q=m[2,4]+m[5,6]+ p1 p4 p6 q=m[2,5]+m[6,6]+ p1 p5 p6
=0+5375+35x15x25 =2625+3500+15x5x25 =4375+5000+35x10x25 =7125+0+35x20x25
q = 18,500 q = 8,000 q = 18,125 q = 24,625
m [2, 6] = min (18500, 8000, 18125, 24625) = 8,000
s [2, 6] = 3
j Example…
1 2 3 4 5 6
1 0 15,750 7,875 9,375 11,875
2 0 2,625 4,375 7,125 8,000
i 3 0 750 2,500 5,375
4 0 1,000 3,500
5 0 5,000
6 0 j
Table m 1 2 3 4 5 6
1 1 1 3 3
2 2 3 3 3
i
3 3 3 3
4 4 5
5 5
Table s
Example…
• l = 6 → 6 Matrices
• Matrices: A1 A2 A3 A4 A5 A6
• No. of positions where split can occur = 5.
Therefore, k = (1, 2, 3, 4, 5).