1 Basic and MatrixMultiplication

Dynamic Programming!
• Example of Dynamic programming:

• Fibonacci numbers.
• What is dynamic programming, exactly?
• Applications:
• Matrix-chain Multiplication
• Longest Common Subsequence
• 0/1 knapsack Algorithm
• Optimal Binary Search Tree
Fibonacci Numbers
• Definition:
• F(n) = F(n-1) + F(n-2), with F(1) = F(2) = 1.
• The first several are:
• 1
• 1
• 2
• 3
• 5
• 8
• 13, 21, 34, 55, 89, 144,…
• Question:
• Given n, what is F(n)?
Candidate algorithm
• def Fibonacci(n):
• if n == 0, return 0
• if n == 1, return 1
• return Fibonacci(n-1) + Fibonacci(n-2)
Running time?
• T(n) = T(n-1) + T(n-2) + O(1)
• T(n) ≥ T(n-1) + T(n-2) for n ≥ 2
• So T(n) grows at least as fast as
the Fibonacci numbers
themselves…
• This is EXPONENTIALLY QUICKLY!
IPython notebook
Why do the Fibonacci numbers grow exponentially
quickly?
• 𝑇 𝑛 =𝑇 𝑛−1 +𝑇 𝑛−2
• ≥ 2𝑇 𝑛 − 2
Trying solving this with the Back Substitution method
• 𝑇 𝑛 ≥ 2𝑇 𝑛 − 2
• ≥ 4𝑇 𝑛 − 4
• ≥ 8𝑇 𝑛 − 6
•… ≥ 2 k 𝑇 𝑛 − 2k for any k < 𝑛/2
•… ≥ 2𝑛/2𝑇 1 by plugging in k = 𝑛−1
2
• So 𝑻 𝒏 ≥ 𝟐𝒏/𝟐, which is REALLY BIG!!!
What’s going on? That’s a lot of
repeated
Consider Fib(8) computation!
6 7
4 5 5 6
2 3 3 4 3 4 4 5
0 1 1 2 1 2 2 3 1 2 2 3 2 3 3 4
1 2 1 2 etc
0 1 0 1 0 1 1 2 0 1 0 1 0 1 1 2
0 1 0 1 0 1 1
Maybe this would be better:
8 def fasterFibonacci(n):
• F = [0, 1, None, None, …, None ]
• \\ F has length n + 1
• for i = 2, …, n:
7 • F[i] = F[i-1] + F[i-2]
• return F[n]
6 Much better running time!
0
This was an example of…
What is dynamic programming?
• It is an algorithm design paradigm
• like divide-and-conquer is an algorithm design paradigm.
• Usually, it is for solving optimization problems
• eg, shortest path (Bellman Ford’s Algorithm)
• (Fibonacci numbers aren’t an optimization problem, but
they are a good example of DP anyway…)
Elements of dynamic programming
1. Optimal sub-structure:
• Big problems break up into sub-problems.
• Fibonacci: F(i) for i ≤ n
• The optimal solution to a problem can be expressed in

terms of optimal solutions to smaller sub-problems.
• Fibonacci:
F(i+1) = F(i) + F(i-1)
2. Overlapping sub-problems:
• The sub-problems overlap.
• Fibonacci:
• Both F[i+1] and F[i+2] directly use F[i].
• And lots of different F[i+x] indirectly use F[i].
• This means that we can save time by solving a sub-problem

just once and storing the answer.
• Optimal substructure.
• Optimal solutions to sub-problems can be used to find the
optimal solution to the original problem.
• Overlapping subproblems.
• The subproblems show up again and again
• Using these properties, we can design a dynamic

programming algorithm:
• Keep a table of solutions to the smaller problems.
• Use the solutions in the table to solve bigger problems.
• Ultimately, we can use the information we collected to
find the solution to the whole thing.
• Construct the optimal solution

Two ways to think about and/or
implement DP algorithms
• Top down
• Bottom up
Bottom up approach
what we just saw.
• For Fibonacci:
• Solve the small problems first
• fill in F[0],F[1]
• Then bigger problems
• fill in F[2]
•…
• Then bigger problems
• fill in F[n-1]
• Then finally solve the real problem.
• fill in F[n]
Top down approach
• Think of it like a recursive algorithm.
• To solve the big problem:
• Recurse to solve smaller problems
• Those recurse to solve smaller problems
• etc..
• The difference from divide and

conquer:
• Keep track of what small problems you’ve
already solved to prevent re-solving the
same problem twice.
• Aka, “memo-ization”
Example of top-down Fibonacci
• define a global list F = [0,1,None, None, …, None]
• if F[n] != None:
• return F[n]
• else:
• F[n] = Fibonacci(n-1) + Fibonacci(n-2)
• return F[n]
Memo-ization visualization
8
6 7
4 5 5 6
2 3 3 4 3 4 4 5
0 1 1 2 1 2 2 3 1 2 2 3 2 3 3 4
1 2 1 2 etc
0 1 0 1 0 1 1 2 0 1 0 1 0 1 1 2
0 1 0 1 0 1 1
30
9
Memo-ization Visualization 8
ctd
7
4
• define a global list F = [0,1,None, None, …, None]
3
• if F[n] != None: 2
• return F[n]
• else: 1
• F[n] = Fibonacci(n-1) + Fibonacci(n-2)
0
• return F[n] 40
What have we learned?
• Paradigm in algorithm design.

• Uses optimal substructure
• Uses overlapping subproblems
• Can be implemented bottom-up or top-down.
• It’s a fancy name for a pretty common-sense idea:
Steps for applying Dynamic Programming
• Step 1: Identify optimal substructure.
• Step 2: Find a recursive formulation for the thing

you want.
• Step 3: Use dynamic programming to find the thing

you want.
• Fill in a table, starting with the smallest sub-problems
and building up.
• Step 4: construct an optimal solution (will discuss

with applications…)
Applications of
Dynamic Programming
Matrix-chain Multiplication
Let’s discuss Matrix Multiplication First
Matrix Multiplication
Matrix: A 𝑛 × 𝑚 matrix 𝐴 = [𝑎[𝑖, 𝑗]] is a two-
dimensional array
𝑎 1,1 𝑎 1,2 ⋯ 𝑎 1, 𝑚 − 1 𝑎 1, 𝑚
𝑎 2,1 𝑎 2,2 ⋯ 𝑎 2, 𝑚 − 1 𝑎 2, 𝑚
𝐴=
⋮ ⋮ ⋯ ⋮ ⋮
𝑎 𝑛, 1 ⋯ ⋯ 𝑎 𝑛, 𝑚 − 1 𝑎 𝑛, 𝑚
Which has 𝑛 rows and 𝑚 columns
Example: The following is a 2 3 matrix:

5 2 3
−3 1 4
The product 𝑪 = 𝑨𝑩 of a 𝒑𝒒 matrix 𝐴 and a 𝒒𝒓 matrix 𝐁 is
a 𝒑𝒓 matrix given by
𝒒
𝒄[𝒊, 𝒋] = ෍ 𝒂 𝒊, 𝒌 𝒃[𝒌, 𝒋]
𝒌=𝟏
For 1 ≤ 𝑖 ≤ 𝑝 and 1 ≤ 𝑗 ≤ 𝑟
• If 𝐴𝐵 is defined, 𝐵𝐴 may not be defined
• Quite possible that 𝐴𝐵 ≠ 𝐵𝐴
• Multiplication is recursively defined by

𝐴1 𝐴2 𝐴3 … 𝐴𝑛 − 1 𝐴𝑛 = 𝐴1( 𝐴2 (𝐴3 … ( 𝐴𝑛 − 1 𝐴𝑛) ) )
• Matrix multiplication is associative, i.e.,

𝐴1 𝐴2 𝐴3 =(𝐴1 𝐴2)𝐴3 = 𝐴1(𝐴2 𝐴3)
so parenthesis does not change the result.
Given a 𝑝 × 𝑞 matrix 𝐴 and a 𝑞𝑟 matrix 𝐵, the direct way of
multiplying 𝐶 = 𝐴𝐵 is to compute a 𝑝𝑟 matrix by
𝑞
𝑐[𝑖, 𝑗] = ෍ 𝑎 𝑖, 𝑘 𝑏[𝑘, 𝑗]
𝑘=1
For 1 ≤ 𝑖 ≤ 𝑝 and 1 ≤ 𝑗 ≤ 𝑟.
Complexity of Direct Matrix Multiplication

Note that 𝐶 has 𝑝𝑟 entries and each entry takes 𝑂(𝑞) time to
compute so the total procedure takes 𝑶 𝒑𝒒𝒓 𝒕𝒊𝒎𝒆
Matrix Multiplication of 𝐴𝐵C
Given a 𝑝 × 𝑞 matrix 𝐴 and a 𝑞𝑟 matrix 𝐵 and a 𝑟𝑠 matrix C, then
𝐴𝐵𝐶 can be computed in two ways (𝐴𝐵)𝐶 and 𝑨 𝑩𝑪 :
The number of multiplications needed are:

𝑚𝑢𝑙𝑡 𝐴𝐵 𝐶 = 𝑝𝑞𝑟 + 𝑝𝑟𝑠
𝑚𝑢𝑙𝑡 𝐴(𝐵𝐶) = 𝑞𝑟𝑠 + 𝑝𝑞𝑠
When p = 5, q = 4, 𝑟 = 6 and 𝑠 = 2, then

𝑚𝑢𝑙𝑡 𝐴𝐵 𝐶 = 180
𝑚𝑢𝑙𝑡 𝐴(𝐵𝐶) = 88
A big difference!
Implication: The multiplication “sequence” (parenthesis) is important

Given a sequence 𝐴1, 𝐴2, … , 𝐴𝑛 of 𝑛 matrices to be multiplied with
the dimensions of 𝑝0, 𝑝1, …, 𝑝𝑛 respectively
𝐴𝑖, has a dimension of 𝑝𝑖 − 1 × 𝑝𝑖,
• determine the multiplication sequence that minimizes the
number of scalar multiplications in computing 𝐴1, 𝐴2, … , 𝐴𝑛
• determine how to parenthesize the multiplications to compute
the product of 𝐴1, 𝐴2, … , 𝐴𝑛 with minimum multiplications
𝐴1 𝐴2 𝐴3 𝐴4 = 𝐴1 𝐴2 (𝐴3, 𝐴4)
= 𝐴1 (𝐴2(𝐴3𝐴4)) = 𝐴1((𝐴2 𝐴3)𝐴4)
= ((𝐴1 𝐴2)𝐴3)𝐴4 = (𝐴1 (𝐴2 𝐴3))𝐴4
Exhaustive search: Exponential
DP a better approach…
• Step 2: Find a recursive formulation for the

matrix-chain multiplication
• Step 3: Use dynamic programming to find the
minimum product in matrix-chain
multiplication.
• Step 4: If needed, keep track of some additional
info so that the algorithm from Step 3 can find the
actual parenthesis multiplication.
Step 1: Optimal substructure
Decompose the problem into subproblems:
for each pair 1 ≤ 𝑖 ≤ 𝑗 ≤ 𝑛,
determine the multiplication sequence for

𝐴 𝑖 … 𝑗 = 𝐴𝑖 , 𝐴𝑖 + 1 , … , 𝐴𝑗
that minimizes the number of multiplications.
𝐴𝑖 … 𝑗 is a 𝑝𝑖 − 1 × 𝑝𝑗 matrix
Original problem: determine a sequence of

multiplication for 𝑨𝟏 … 𝒏
Split the original structure into substructures
Suppose 𝐴𝑖 , 𝐴 𝑖 + 1, …, 𝐴𝑗 is split between two substructures
around 𝑘
𝐴𝑖 , 𝐴 𝑖 + 1, …, 𝐴𝑗
Split into 2
substructures
𝑨𝒊, 𝑨 𝒊 + 𝟏, …, 𝑨𝒌 𝑨𝒌 + 𝟏, …, 𝑨𝒋
• Find the optimal solution for both substructures
• And then combine them to get the optimal solution of the
original structure
NOTE: Need to ensure the correct place (i.e., 𝒌) to split the product
around 𝑘
𝐴3 … 6 = (𝐴3(𝐴4 𝐴5))(𝐴6)
𝑘=5
𝑨𝟑 …𝟓
𝑨𝟔 … 𝟔
𝑘=3
𝑨𝟑 …𝟑
𝑨𝟒 … 𝟓
around 𝑘
• How do we decide where to split the chain (what is 𝑘)?

(Search all possible values of 𝑘)
• How do we parenthesize the substructures

𝑨𝒊 … 𝒌 and 𝑨𝒌 + 𝟏 … 𝒋 ?
(Problem has optimal substructure property that

𝑨𝒊 … 𝒌 and 𝑨𝒌 + 𝟏 … 𝒋 must be optimal so we can apply
the same procedure recursively (STEP 2))

multiplication.
Step 2: Recursive Formulation
Find the cost of the optimal solution of the original problem
recursively in terms of the optimal solution to the
subproblems
Objective: Minimum cost parenthesis

𝐴𝑖 , 𝐴 𝑖 + 1, …, 𝐴𝑗 where 1 ≤ 𝑖 ≤ 𝑗 ≤ 𝑛
Let 𝑚[𝑖, 𝑗] be the minimum number of scalar multiplication

needed to compute matrix 𝐴𝑖 … 𝑗
Objective: Minimum cost parenthesis
𝐴𝑖 , 𝐴 𝑖 + 1, …, 𝐴𝑗 where 1 ≤ 𝑖 ≤ 𝑗 ≤ 𝑛
CASE 1 (trivial case):
𝑖 == 𝑗 means only a single matrix
𝐴𝑖 … 𝑖 = 𝐴𝑖
Thus, 𝒎[𝒊, 𝒊] = 0, for 𝒊 = 𝟏, 𝟐, … , 𝒏
CASE 2: when 𝑖 < 𝑗

Split 𝐴𝑖 … 𝑗 into 𝑨𝒊 … 𝒌 and 𝑨𝒌 + 𝟏 … 𝒋 substructures
Thus
𝒎 𝒊, 𝒋 = 𝒎 𝒊, 𝒌 + 𝒎 𝒌 + 𝟏, 𝒋 + 𝒑𝒊 − 𝟏 𝒑𝒌 𝒑𝒋
for 𝒌 = 𝒊, 𝒊 + 𝟏, 𝒊 + 𝟐, … , 𝒋 − 𝟏
CASE 2: when 𝑖 < 𝑗
Split 𝐴𝑖 … 𝑗 into 𝐴𝑖 … 𝑘 and 𝐴𝑘 + 1 … 𝑗 substructures
Thus
𝑚 𝑖, 𝑗 = 𝑚 𝑖, 𝑘 + 𝑚 𝑘 + 1, 𝑗 + 𝑝𝑖 − 1 𝑝𝑘 𝑝𝑗
for 𝑘 = 𝑖, 𝑖 + 1, 𝑖 + 2, … , 𝑗 − 1
Check all possible values 𝑘 and pick the best one
Recursive Formulation:
𝟎 𝒊𝒇 𝒊 == 𝒋
𝒎 𝒊, 𝒋 = 𝒎𝒊𝒏 (𝒎 𝒊, 𝒌 + 𝒎 𝒌 + 𝟏, 𝒋 + 𝒑𝒊 − 𝟏 𝒑𝒌 𝒑𝒋)
𝒊≤𝒌<𝒋
𝒊𝒇 𝒊 < 𝒋
• Step 2: Find a recursive formulation for the matrix-
chain multiplication

multiplication.
Step 2: Dynamic Programming
(Bottom-Up) Uses two auxiliary tables
• 𝒎[𝟏 … 𝒏, 𝟏 … 𝒏] for storing
𝒎[𝒊, 𝒋]
• 𝒔 𝟏 … 𝒏, −𝟏 𝟐 … 𝒏 for storing
𝒌 index value
(Bottom-Up)
CASE 1: trivial
Single matrix
(Bottom-Up)
CASE 2: 𝒊 < 𝒋
multiplication.

actual parenthesis multiplication from the
auxiliary table 𝒔.
Example: Given a sequence of four matrices 𝐴1, 𝐴2, 𝐴3, 𝐴4
with 𝑝0 = 5, 𝑝1 = 4, 𝑝2 = 6, 𝑝3 = 2 and 𝑝4 = 7.
with 𝑝0 = 5, 𝑝1 = 4, 𝑝2 = 6, 𝑝3 = 2 and 𝑝4 = 7.
The Optimal Solution 𝑨𝟏 𝑨𝟐 𝑨𝟑 𝑨𝟒 :
𝐴2 𝐴3 = 4 × 6 × 2 = 48
𝐴1)( 𝐴2 𝐴3 = 5 × 4 × 2 = 40
𝐴1 𝐴2 𝐴3 𝐴4 = 5 × 2 × 7 = 70
Total Multiplication 𝐴1 𝐴2 𝐴3 𝐴4 = 48 + 40 + 70
= 158
with 𝑝0 = 5, 𝑝1 = 4, 𝑝2 = 6, 𝑝3 = 2 and 𝑝4 = 7.
𝑖 𝑖
1 2 3 4 1 2 3
4 0 4
3 0 𝑗 3
𝑗
2 0 2
1 0
𝒎 𝒔
𝑝0 = 5, 𝑝1 = 4, 𝑝2 = 6, 𝑝3 = 2 and 𝑝4 = 7
𝑚[1,2] = 𝑚𝑖𝑛 (𝑚 1, 𝑘 + 𝑚 𝑘 + 1, 2 + 𝑝0 𝑝𝑘 𝑝2)

1≤𝑘<2
= 𝑚 1,1 + 𝑚 2, 2 + 𝑝0 𝑝1 𝑝2 = 120
𝑖 𝑖
1 2 3 4 1 2 3
4 0 4
3 0 𝑗 3
𝑗
2 120 0 2 1
1 0
𝒎 𝒔
𝑝0 = 5, 𝑝1 = 4, 𝑝2 = 6, 𝑝3 = 2 and 𝑝4 = 7.
𝑚[2, 3] = 𝑚𝑖𝑛 (𝑚 2, 𝑘 + 𝑚 𝑘 + 1, 3 + 𝑝1 𝑝𝑘 𝑝3)

2≤𝑘<3
= 𝑚 2,2 + 𝑚 3, 3 + 𝑝1 𝑝2 𝑝3 = 48
𝑖 𝑖
1 2 3 4 1 2 3
4 0 4
3 48 0 𝑗 3 2
𝑗
2 120 0 2 1
1 0
𝒎 𝒔
𝑝0 = 5, 𝑝1 = 4, 𝑝2 = 6, 𝑝3 = 2 and 𝑝4 = 7.
𝑚[3, 4] = 𝑚𝑖𝑛 (𝑚 3, 𝑘 + 𝑚 𝑘 + 1, 4 + 𝑝2 𝑝𝑘 𝑝4)

3≤𝑘<4
= 𝑚 3,3 + 𝑚 4 4 + 𝑝2 𝑝3 𝑝4 = 84
𝑖 𝑖
1 2 3 4 1 2 3
4 84 0 4 3
3 48 0 𝑗 3 2
𝑗
2 120 0 2 1
1 0
𝒎 𝒔
𝑝 = 5, 𝑝 = 4, 𝑝 = 6, 𝑝 = 2 and 𝑝 = 7.
0 1 2 3 4
𝑚[1, 3] = 𝑚𝑖𝑛 (𝑚 1, 𝑘 + 𝑚 𝑘 + 1, 3 + 𝑝0 𝑝𝑘 𝑝3)
1≤𝑘<3
𝑚 1,1 + 𝑚 2, 3 + 𝑝0 𝑝1 𝑝3 = 88
= 𝑚𝑖𝑛 ቊ
𝑚 1,2 + 𝑚 3, 3 + 𝑝0 𝑝2 𝑝3 = 180
= 88
𝑖 𝑖
1 2 3 4 1 2 3
4 84 0 4 3
3 88 48 0 𝑗 3 1 2
𝑗
2 120 0 2 1
1 0
𝒎 𝒔
𝑝 = 5, 𝑝 = 4, 𝑝 = 6, 𝑝 = 2 and 𝑝 = 7.
0 1 2 3 4
𝑚[2, 4] = 𝑚𝑖𝑛 (𝑚 2, 𝑘 + 𝑚 𝑘 + 1, 4 + 𝑝1 𝑝𝑘 𝑝4)
2≤𝑘<4
𝑚 2,2 + 𝑚 3 4 + 𝑝1 𝑝2 𝑝4 = 252
= 𝑚𝑖𝑛 ቊ
𝑚 2,3 + 𝑚 4, 4 + 𝑝1 𝑝3 𝑝4 = 104
= 104
𝑖 𝑖
1 2 3 4 1 2 3
4 104 84 0 4 3 3
3 88 48 0 𝑗 3 1 2
𝑗
2 120 0 2 1
1 0
𝒎 𝒔
𝑝 = 5, 𝑝 = 4, 𝑝 = 6, 𝑝 = 2 and 𝑝 = 7.
0 1 2 3 4
𝑚[1, 4] = 𝑚𝑖𝑛 (𝑚 1, 𝑘 + 𝑚 𝑘 + 1, 4 + 𝑝0 𝑝𝑘 𝑝4)
1≤𝑘<4
𝑚 1,1 + 𝑚 2, 4 + 𝑝0 𝑝1 𝑝4 = 244
= 𝑚𝑖𝑛 ൞ 𝑚 1,2 + 𝑚 3, 4 + 𝑝0 𝑝2 𝑝4 = 294 = 158
𝑚 1,3 + 𝑚 4, 4 + 𝑝0 𝑝3 𝑝4 = 158
1 2 3 4 1 2 3
4 158 104 84 0 4 3 3 3
3 88 48 0 𝑗 3 1 2
𝑗
2 120 0 2 1
1 0
𝒎 𝒔
Step 4: If needed, keep track of some additional info so that
the algorithm from Step 3 can find the actual parenthesis
multiplication from the auxiliary table 𝒔.
1 2 3
𝒔 𝟏, 𝟒 = 3 𝑨𝟏𝑨𝟐 𝑨𝟑 (𝑨𝟒) 4 3 3 3
𝑗 3 1 2
𝒔 𝟏, 𝟑 = 1 ( 𝑨𝟏)(𝑨𝟐 𝑨𝟑 )(𝑨𝟒)
2 1
Solution: 𝒔
Total Multiplication = 𝒎 𝟏, 𝟒 = 158
𝑨𝟏 𝑨𝟐 𝑨𝟑 𝑨𝟒
1 2 3
𝒔 𝟏, 𝟒 = 3 𝑨𝟏𝑨𝟐 𝑨𝟑 (𝑨𝟒) 4 3 2 3
𝑗 3 1 2
𝒔 𝟏, 𝟑 = 1 ( 𝑨𝟏)(𝑨𝟐 𝑨𝟑 )(𝑨𝟒)
2 1
𝑝0 = 5, 𝑝1 = 4, 𝑝2 = 6, 𝑝3 = 2 and 𝑝4 = 7 𝒔
𝑨𝟐 𝑨𝟑 = 𝟒 × 𝟔 × 𝟐 = 𝟒𝟖
𝑨𝟏)( 𝑨𝟐 𝑨𝟑 = 𝟓 × 𝟒 × 𝟐 = 𝟒𝟎
𝑨𝟏 𝑨𝟐 𝑨𝟑 𝑨𝟒 = 𝟓 × 𝟐 × 𝟕 = 𝟕𝟎
Total Multiplication = 48 + 40 + 70 = 158
Algorithm (refer Cormen book for detail)
PRINT-OPTIMAL-PARENS(𝑠, 𝑖, 𝑗)
1 if 𝑖 == j
2 print “𝐴” 𝑖
3 else print “(”
4 PRINT-OPTIMAL-PARENS(𝑠, 𝑖, 𝑠[𝑖, 𝑗])
5 PRINT-OPTIMAL-PARENS (𝑠, 𝑠 𝑖, 𝑗 + 1, 𝑗)
6 print “)”

1 Basic and MatrixMultiplication

Uploaded by

Copyright:

Available Formats

1 Basic and MatrixMultiplication

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

1 Basic and MatrixMultiplication

Uploaded by

Copyright:

Available Formats

Dynamic Programming!

• Example of Dynamic programming:

6 Much better running time!

• The optimal solution to a problem can be expressed in

• This means that we can save time by solving a sub-problem

• Using these properties, we can design a dynamic

• Construct the optimal solution

• The difference from divide and

• Paradigm in algorithm design.

• Step 2: Find a recursive formulation for the thing

• Step 3: Use dynamic programming to find the thing

• Step 4: construct an optimal solution (will discuss

Example: The following is a 2 3 matrix:

• Quite possible that 𝐴𝐵 ≠ 𝐵𝐴

• Multiplication is recursively defined by

• Matrix multiplication is associative, i.e.,

Complexity of Direct Matrix Multiplication

The number of multiplications needed are:

When p = 5, q = 4, 𝑟 = 6 and 𝑠 = 2, then

Implication: The multiplication “sequence” (parenthesis) is important

• Step 2: Find a recursive formulation for the

determine the multiplication sequence for

that minimizes the number of multiplications.

Original problem: determine a sequence of

• How do we decide where to split the chain (what is 𝑘)?

• How do we parenthesize the substructures

(Problem has optimal substructure property that

• Step 2: Find a recursive formulation for the

Objective: Minimum cost parenthesis

Let 𝑚[𝑖, 𝑗] be the minimum number of scalar multiplication

Thus, 𝒎[𝒊, 𝒊] = 0, for 𝒊 = 𝟏, 𝟐, … , 𝒏

CASE 2: when 𝑖 < 𝑗

Check all possible values 𝑘 and pick the best one

• Step 3: Use dynamic programming to find the

• Step 4: If needed, keep track of some additional

The Optimal Solution 𝑨𝟏 𝑨𝟐 𝑨𝟑 𝑨𝟒 :

𝑚[1,2] = 𝑚𝑖𝑛 (𝑚 1, 𝑘 + 𝑚 𝑘 + 1, 2 + 𝑝0 𝑝𝑘 𝑝2)

𝑚[2, 3] = 𝑚𝑖𝑛 (𝑚 2, 𝑘 + 𝑚 𝑘 + 1, 3 + 𝑝1 𝑝𝑘 𝑝3)

𝑚[3, 4] = 𝑚𝑖𝑛 (𝑚 3, 𝑘 + 𝑚 𝑘 + 1, 4 + 𝑝2 𝑝𝑘 𝑝4)

Algorithm (refer Cormen book for detail)

You might also like