Scribd
Upload
990 views

Optimal Binary Search Tree

The document discusses the optimal binary search tree problem and its solution using dynamic programming. It explains that dynamic programming solves this problem by breaking it down into solving smaller subproblems and storing the results. It describes building a cost matrix to store the costs of the best trees for different subsets of keys, and using this to iteratively determine the optimal tree for sets with increasing numbers of keys, reaching the overall optimal solution in O(n3) time. The optimal binary search tree places the keys in a tree structure to minimize the expected search time if all keys have equal access probabilities.

Uploaded by

TahaRazvi
Copyright
© © All Rights Reserved
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
990 views

Optimal Binary Search Tree

The document discusses the optimal binary search tree problem and its solution using dynamic programming. It explains that dynamic programming solves this problem by breaking it down into solving smaller subproblems and storing the results. It describes building a cost matrix to store the costs of the best trees for different subsets of keys, and using this to iteratively determine the optimal tree for sets with increasing numbers of keys, reaching the overall optimal solution in O(n3) time. The optimal binary search tree places the keys in a tree structure to minimize the expected search time if all keys have equal access probabilities.

Uploaded by

TahaRazvi
Copyright
© © All Rights Reserved
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 21

Design and Analysis of Algorithms

Example of Dynamic Programming


Algorithms
Optimal Binary Search Tree

Key Points
Dynamic Algorithms

Solve small problems


Store answers to these small problems
Use the small problem results to answer larger problems
Use space to obtain speed-up

Optimal Binary Search Trees


Balanced trees
Always the most efficient search trees?

Optimal Binary Search Trees


Balanced trees
Always the most efficient search trees?
Yes, if every key is equally probable

Spelling check dictionary

Entry at root of a balanced tree ... miasma?


Occurrence in ordinary text ... 0.01%, 0.0001%, .. ?
99.99+% of searches waste at least one comparison!
Common words (a, and, the, ...) in leaves?
If key, k, has relative frequency, rk ,
then in an optimal tree, we minimise

dkrk

dk

is the depth of key k

Optimal Binary Search Trees


Finding the optimal tree
Try each candidate key as the root
Divides the keys into left and right groups
Try each key in the left group as root of the left sub-tree
...
Number of candidate keys:

O(n)

Number of candidates for roots of sub-trees:

O(nn) algorithm

2O(n)

Optimal Binary Search Trees


Lemma
Sub-trees of optimal trees are themselves optimal trees

Proof
If a sub-tree of an optimal tree is not optimal,
then a better search tree will be produced if the sub-tree
is replaced by an optimal tree.

Optimal Binary Search Trees


Key Table
Keys (in order) + frequency

A B C D E F G H I J K L M N O P ..
23 10 8 12 30 5 14 18 20 2

4 11 7 22 22 10 ..

Key Problem
Which key should be placed at the root?
If we can determine this, we can ask the same
question for the left and right subtrees.

Optimal Binary Search Tree


Divide and conquer?
n choices
Repeat the process for the sub-trees 2 O(n)
O(nn)
Smaller problems are not small enough!
One k, one n-k-1
Choose a key for the root

Optimal Binary Search Tree


Start with the small problems
A B C D E F G H I J K L M N O P ..
23 10 8 12 30 5 14 18 20 2

4 11 7 22 22 10 ..

Look at pairs of adjacent keys


Two possible
C
arrangements
D
Cost

8x1 + 12x2 = 32

D
C

Min

8x2 + 12x1 = 28

Optimal Binary Search Tree - Cost matrix


Initialise
Diagonal
C[j,j]

Cjj

Costs of
one-element trees

Below diagonal
C[j,k]
Costs of best tree

j to k
Cost of best tree
C-G

Zero
x

Optimal Binary Search Tree - Cost matrix


Store the costs of the best two element trees
Diagonal
Cj-1,j
C[j,j]
Costs of
one-element trees

Below diagonal
C[j-1,j]
Costs of best
2-element trees

j-1 to j

Optimal Binary Search Tree - Root matrix


Store the roots of the best two element trees
Diagonal
Roots of 1-element trees

Below diagonal
best[j-1,j]
Root of best
2-element trees

j-1 to j

Optimal Binary Search Tree - 3-element trees


Now examine the 3-element trees
A B C D E F G H I J K L M N O P ..
23 10 8 12 30 5 14 18 20 2

4 11 7 22 22 10 ..

Choose each in turn as the root


B with (C,D) to the right
C with B and D as children
D with (B,C) to the left

Find best, store cost in C[B,D]


Store root in best[B,D]

Next slide

Optimal Binary Search Tree - 3-element trees


3-element trees
Root = B
B
D
C

We already
know this is
best for C,D
and stored
its cost

Root = C

Root = D
D

C
B

B
C
Best B,C

Find best, store cost in C[B,D]


Store root in best[B,D]

Optimal Binary Search Tree - 3-element trees


Similarly, update all C[j-2,j] and best[j-2,j]

Costs

Roots

Optimal Binary Search Trees - 4-trees


Now the 4-element trees
eg A-D
Choose A as root

Use 0 for left


Best B-D is known

Choose B as root

A-A is in C[0,0]
Best C-D is known

Choose C as root

A-B is in C[0,1]
D is in C[3,3]

Choose D as root

A-C is in C[0,2]
Use 0 in C[4,3] for right

Optimal Binary Search Trees


Final cost will be in C[0,n-1]

Final
cost

Optimal Binary Search Trees


Construct the search tree
Root will be in

best[0,n-1]
If r0 = best[0,n-1],
Left subtree root is

best[0,r0-1],

Right subtree root is

best[r0+1,n-1]
Root = E

Optimal Binary Search Trees


Construct the search tree
E
B
A

H
D

G
F

I
J

Optimal Binary Search Trees - Analysis

k -element trees require k operations


One for each candidate root

There are k of them


There are n levels

O(k2)
n

k2 = O(n3)

k =1

Constructing the tree is O(n)


Average ~ logn
Total
O(n3)

Optimal Binary Search Trees - Notes


A good example of a dynamic algorithm
Solves all the small problems
Builds solutions to larger problems from them
Requires space to store small problem results

You might also like