Matrix Chain Multiplication

Neelima Gupta
[email protected]

Neelima Gupta Matrix Chain Multiplication September 4, 2024 1/9

Matrix Chain Multiplication

Consider the problem of multiplying a chain of matrices A1 A2 A3 where

A1 : (2x3), A2 : (3x4), A3 : (4x5)
Case 1: (A1 A2 )A3 : (2x3x4) + (2x4x5) =24 + 40 = 64.
Case 2: A1 (A2 A3 ): (3x4x5) + (2x3x5) = 60 + 30 = 90

We see that changing the order in which the matrices are multiplied
changes the cost of operation (number of multiplications).

Aim
Determine the order in which the matrices should be multiplied so as
to minimize the number of multiplications.

Neelima Gupta Matrix Chain Multiplication September 4, 2024 2/9

Matrix Chain Multiplication

Consider the problem of multiplying a chain of matrices A1 A2 A3 where

A1 : (2x3), A2 : (3x4), A3 : (4x5)
Case 1: (A1 A2 )A3 : (2x3x4) + (2x4x5) =24 + 40 = 64.
Case 2: A1 (A2 A3 ): (3x4x5) + (2x3x5) = 60 + 30 = 90

We see that changing the order in which the matrices are multiplied
changes the cost of operation (number of multiplications).

Aim
Determine the order in which the matrices should be multiplied so as
to minimize the number of multiplications.

Neelima Gupta Matrix Chain Multiplication September 4, 2024 2/9

Matrix Chain Multiplication

Consider the problem of multiplying a chain of matrices A1 A2 A3 where

A1 : (2x3), A2 : (3x4), A3 : (4x5)
Case 1: (A1 A2 )A3 : (2x3x4) + (2x4x5) =24 + 40 = 64.
Case 2: A1 (A2 A3 ): (3x4x5) + (2x3x5) = 60 + 30 = 90

We see that changing the order in which the matrices are multiplied
changes the cost of operation (number of multiplications).

Aim
Determine the order in which the matrices should be multiplied so as
to minimize the number of multiplications.

Neelima Gupta Matrix Chain Multiplication September 4, 2024 2/9

Matrix Chain Multiplication: Definition

Input
We are given n matrices say, A1 , A2 , . . . , An , of order
(d1 , d2 ), (d2 , d3 ), . . . , (dn , dn+1 ) respectively.

Aim
Determine the order (parenthesization) in which the matrices should
be multiplied so as to minimize the number of multiplications.

Neelima Gupta Matrix Chain Multiplication September 4, 2024 3/9

Matrix Chain Multiplication: Definition

Input
We are given n matrices say, A1 , A2 , . . . , An , of order
(d1 , d2 ), (d2 , d3 ), . . . , (dn , dn+1 ) respectively.

Aim
Determine the order (parenthesization) in which the matrices should
be multiplied so as to minimize the number of multiplications.

Neelima Gupta Matrix Chain Multiplication September 4, 2024 3/9

Order and Choices

Order: Implicit in the problem.

Split point: Note that in any solution, we multiply two matrices in
the end to obtain the final product. These two matrices must be of
the following form
(A1 , . . . , Ak ) and (Ak +1 , . . . , An )
for some k : 1 ≤ k < n. k is called the split point.
Then optimal parenthesization of A1 . . . An consists of (optimal
parenthesization of A1 . . . Ak followed by optimal parenthesization
of Ak +1 . . . An ).

Neelima Gupta Matrix Chain Multiplication September 4, 2024 4/9

DP Solution

Let Opt(1, n) denote the minimum the number of multiplications

required to multiply the chain A1 , A2 , . . . , An .
Let k be the split point then the minimum the number of
multiplications required to multiply the chain is

(OPT (1, k ) + Opt(k + 1, n) + d1 dk +1 dn+1 )

Since we don’t know the split point k of the optimal solution, we
make a guess and take the minimum over all possible guesses.
How many choices for the guesses? Optimal has n − 1 choices
for the split point k . For example, for k = 1, we have
(A1 )(A2 , . . . , An ) whereas for k = n − 1, we have,
(A1 , . . . , An−1 )(An ).
So n − 1 possible guesses.

Neelima Gupta Matrix Chain Multiplication September 4, 2024 5/9

DP Solution

Let Opt(1, n) denote the minimum the number of multiplications

required to multiply the chain A1 , A2 , . . . , An .
Let k be the split point then the minimum the number of
multiplications required to multiply the chain is

(OPT (1, k ) + Opt(k + 1, n) + d1 dk +1 dn+1 )

Since we don’t know the split point k of the optimal solution, we
make a guess and take the minimum over all possible guesses.
How many choices for the guesses? Optimal has n − 1 choices
for the split point k . For example, for k = 1, we have
(A1 )(A2 , . . . , An ) whereas for k = n − 1, we have,
(A1 , . . . , An−1 )(An ).
So n − 1 possible guesses.

Neelima Gupta Matrix Chain Multiplication September 4, 2024 5/9

DP Solution

Thus,

n−1
OPT (1, n) = min{OPT (1, k ) + OPT (k + 1, n) + d1 dk +1 dn+1 } (1)
k =1

Generalising,
j−1
OPT (i, j) = min{OPT (i, k ) + OPT (k + 1, j) + di dk +1 dj+1 } (2)
k =i

Neelima Gupta Matrix Chain Multiplication September 4, 2024 6/9

Overlapping Subproblems

Neelima Gupta Matrix Chain Multiplication September 4, 2024 7/9

Example

Matrix dimensions
A1 : 3x5, A2 : 5x4, A3 : 4x2, A4 : 2x7, A5 : 7x3, A6 : 3x8

Neelima Gupta Matrix Chain Multiplication September 4, 2024 8/9

Example

Matrix dimensions
A1 : 3x5, A2 : 5x4, A3 : 4x2, A4 : 2x7, A5 : 7x3, A6 : 3x8

A1 A2 A3 A4 A5 A6
A1 xA2 = 3 x 5 x 4 = 60
A1 0 60
A2 0 40 A2 xA3 = 5 x 4 x 2 = 40
A3 0 56 A3 xA4 = 4 x 2 x 7 = 56
A4 0 42 A4 xA5 = 2 x 7 x 3 = 42
A5 0 168 A5 xA6 = 7 x 3 x 8 = 168
A6 0

Neelima Gupta Matrix Chain Multiplication September 4, 2024 8/9

Example

Matrix dimensions
A1 : 3x5, A2 : 5x4, A3 : 4x2, A4 : 2x7, A5 : 7x3, A6 : 3x8

A1 A2 A3 A4 A5 A6
A1 0 60 70 A1 xA2 xA3 =
A2 0 40 min{(A1 A2 )A3 , A1 (A2 A3 )} =
A3 0 56 min{60 + 3x4x2, 40 +
A4 0 42 3x5x2} = min{84, 70} = 70
A5 0 168
A6 0

Neelima Gupta Matrix Chain Multiplication September 4, 2024 8/9

Example

Matrix dimensions
A1 : 3x5, A2 : 5x4, A3 : 4x2, A4 : 2x7, A5 : 7x3, A6 : 3x8

A1 A2 A3 A4 A5 A6
A1 0 60 70 A2 xA3 xA4 =
A2 0 40 110 min{(A2 A3 )A4 , A2 (A3 A4 )} =
A3 0 56 min{40 + 5x2x7, 56 +
A4 0 42 5x4x7} = min{110, 196} =
A5 0 168 110.
A6 0

Neelima Gupta Matrix Chain Multiplication September 4, 2024 8/9

Example

Matrix dimensions
A1 : 3x5, A2 : 5x4, A3 : 4x2, A4 : 2x7, A5 : 7x3, A6 : 3x8

A1 A2 A3 A4 A5 A6
A1 0 60 70 A3 xA4 xA5 =
A2 0 40 110 min{(A3 A4 )A5 , A3 (A4 A5 )} =
A3 0 56 66 min{56 + 4x7x3, 42 +
A4 0 42 4x2x3} = min{140, 66}
A5 0 168 =66.
A6 0

Neelima Gupta Matrix Chain Multiplication September 4, 2024 8/9

Example

Matrix dimensions
A1 : 3x5, A2 : 5x4, A3 : 4x2, A4 : 2x7, A5 : 7x3, A6 : 3x8

A1 A2 A3 A4 A5 A6
A1 0 60 70 A4 xA5 xA6 =
A2 0 40 110 min{(A4 A5 )A6 , A4 (A5 A6 )} =
A3 0 56 66 min{42 + 2x3x8, 168 +
A4 0 42 90 2x7x8} = min{90, 312}
A5 0 168 =90.
A6 0

Neelima Gupta Matrix Chain Multiplication September 4, 2024 8/9

Example

Matrix dimensions
A1 : 3x5, A2 : 5x4, A3 : 4x2, A4 : 2x7, A5 : 7x3, A6 : 3x8

A1 A2 A3 A4 A5 A6
70 112 130 202
A1 0 60
(k = 1) (k = 3) (k = 3) (k=5)
110 112 218
A2 0 40
(k = 3) (k = 3) (k = 3)
66 154
A3 0 56
(k = 3) (k = 3)
90
A4 0 42
(k=5)
A5 0 168
A6 0

Neelima Gupta Matrix Chain Multiplication September 4, 2024 8/9

Running Time

O(n2 ) entries in the table.

Each entry takes O(n) time to .
Therefore, total running time is O(n3 ).

Neelima Gupta Matrix Chain Multiplication September 4, 2024 9/9