Ch11

Dynamic Programming-
2
Eng. Jibril Hassan (Msc IT)

Design and Analysis of Algorithms - Chapter 1

11
 Objectives

• After studying this chapter you should be

able to:
• Define ‘Principle of Optimality’
• Analyze optimal binary search trees with
an example
• Describe Knapsack problem with an
example
• Explain memory functions with an example

Design and Analysis of Algorithms - Chapter 2

11
 Principle of Optimality
• Principle of Optimality is defined as a basic
dynamic programming principle which helps us to
view problems as a sequence of sub problems.
• We might face a difficulty in converting the
Principle of Optimality into an algorithm, as it is not
very easy to identify the sub problems that are
relevant to the problem under consideration.
• Bellman developed an equation for the Principle of
Optimality.
• We can study this equation only with the help of
dynamic programming concepts.

Design and Analysis of Algorithms - Chapter 3

11
 Principle of Optimality
• Bellman formulated an equation for the
Principle of Optimality during the early stages
in technology development.
• The computers used during that stage were
not as powerful as we use now.
• This principle may help in advanced dynamic
programming applications which support
larger dimensions than that used today. The
Bellman equation for Principle of Optimality is
given as:

Design and Analysis of Algorithms - Chapter 4

11
 Optimal Binary Search
Trees
• Binary search tree – Binary search tree
is defined as a binary tree with the
following properties:
• The values of elements in the left sub-tree
of a node are lesser than the node’s value.
• The values of elements in the right sub-
tree of a node are greater than the node’s
value.
• The right and left sub-trees of a node are
also binary search trees.
Design and Analysis of Algorithms - Chapter 5
11
 Optimal Binary Search
Trees
• Binary search trees are node based data
structures used in many system programming
applications for managing dynamic sets.

Design and Analysis of Algorithms - Chapter 6

11
Psuado code for Binary Search

Design and Analysis of Algorithms - Chapter 7

11
Design and Analysis of Algorithms - Chapter 8
11
Design and Analysis of Algorithms - Chapter 9
11
Continues’

Design and Analysis of Algorithms - Chapter 10

11
 Traversal
• Traversal – Traversing a binary search tree
involves visiting all the nodes in the tree.
• First we visit the root node. Then we visit its
left and right sub-trees.
• We can visit a node only once.
• We can traverse a binary tree recursively using
the following pseudocode:

Design and Analysis of Algorithms - Chapter 11

11
Traversing a Binary Search Tree

Design and Analysis of Algorithms - Chapter 12

11
Binary Search Tree

Design and Analysis of Algorithms - Chapter 13

11
 Binary Search Tree
• For the trees in the figure 11.2, we can find the
average number comparisons from the equation
Eq: 11.2.
• Average number comparisons for a tree
• For the tree of figure 11.2, the average number
of comparisons is given as

Design and Analysis of Algorithms - Chapter 14

11
 Binary Search Tree
• Here we can see that the tree is not optimized.
• The total number of binary search trees with n
elements is equal to the nth Catalan number,
c(n), given in Eq 11.3.

Design and Analysis of Algorithms - Chapter 15

11
Design and Analysis of Algorithms - Chapter 16
11
Continued’

Design and Analysis of Algorithms - Chapter 17

11
Design and Analysis of Algorithms - Chapter 18
11
 Solving binary search trees
using dynamic programming
• Let us illustrate the above mentioned
algorithm using the four keys that we used in
the previous section.
• The keys and the probabilities are given in
table 11.1.

Design and Analysis of Algorithms - Chapter 19

11
Table of Keys and
Probabilities

Design and Analysis of Algorithms - Chapter 20

11
Optimal Binary Search
Tree

Design and Analysis of Algorithms - Chapter 21

11
 Knapsack Problem
• If a set of items are given, each with a weight
and a value, determine the number of items
that minimizes the total weight and maximizes
the total value.

Design and Analysis of Algorithms - Chapter 22

11
 Knapsack Problem
• Consider a situation where a thief breaks into a
store and tries to fill his knapsack with as much
valuable goods as possible.
• above shows the number of goods with its value
and weight. There are 3 items given with
weights 10 kg, 20 kg and 30 kg with values
Rs.60, Rs.100 and Rs. 120 respectively.
• The capacity of the knapsack is given as 50 kg.
We have to fill the knapsack with the items
appropriately to get the maximum value, but
not exceeding the weight 50Kg
Design and Analysis of Algorithms - Chapter 23
11
 Knapsack Problem
• Let us try to fill the knapsack using different
items as shown in the figure 11.6.

Design and Analysis of Algorithms - Chapter 24

11
 Knapsack Problem
• Now let us see the best possible solution for
this problem from the figure 11.7.

Design and Analysis of Algorithms - Chapter 25

11
 Knapsack Problem
• Here we take items 1 and 2 as such and
we take the 20/30 part of item 3.
• Thus the values add up to Rs 240, which
is the maximum value.

Design and Analysis of Algorithms - Chapter 26

11
Algorithm for Knapsack

Design and Analysis of Algorithms - Chapter 27

11
Algorithm Tracing for
Knapsack

Design and Analysis of Algorithms - Chapter 28

11
 Types of Knapsack
• The different types of Knapsack problems are:
• Fractional Knapsack problem – If we have
materials of different values per unit volume
and maximum amounts, the Fractional
Knapsack problem finds the most valuable mix
of materials which fit in a knapsack of fixed
volume.
• We have to take as much as possible material
that is most valuable per unit volume.
• Continue this process until the knapsack is full.

Design and Analysis of Algorithms - Chapter 29

11
 Types of Knapsack
• Bounded Knapsack problem – If we
have the types of items of different
values and volumes, find the most
valuable set of items that fit in a
knapsack of fixed volume.
• Here the number of items of each type
is unbounded.
• This is an NP-hard optimization problem.

Design and Analysis of Algorithms - Chapter 30

11
 Solving Knapsack problem
using dynamic programming
• Let us solve an instance of Knapsack problem
using dynamic programming.
• Consider the following data given in table
11.5:

Design and Analysis of Algorithms - Chapter 31

11
 Example Table for
Knapsack Problem
• If we apply the recurrence formulas to
this set of data, then we will get the
following table 11.6.

Design and Analysis of Algorithms - Chapter 32

11
 Memory Functions
• The goal of using memory functions is to solve
only the sub problems which are necessary.
• Memory functions use a dynamic programming
technique called memoization in order to
reduce the inefficiency of recursion that might
occur.
• We use memoization for finding solution to sub
problems, so as to reduce recalculation.

Design and Analysis of Algorithms - Chapter 33

11
 Memory Functions
• We use it in algorithms which have lots
of recursive calls to the sub problems.
• Memory functions method solves
problems using top down approach, but
maintains a table which is used for the
bottom up dynamic programming
algorithms.

Design and Analysis of Algorithms - Chapter 34

11
 Memory Functions
• We can initialize the table values to a
‘null’ symbol.
• When we have to compute a new value:
• The method checks the corresponding
entry in the table
• If this entry is not ‘null’, it is retrieved
• If this entry is ‘null’, then the value is
computed using recursive calls and the
results are entered in the table.

Design and Analysis of Algorithms - Chapter 35

11
Algorithm for Memory
Functions

Design and Analysis of Algorithms - Chapter 36

11
Algorithm for Memory
Functions

Design and Analysis of Algorithms - Chapter 37

11
 Solving Knapsack problem
using memory functions
• Now, let us solve the same instance given in
the previous section by using memory
functions.
• The table 11.7 gives the result. We can see
that, here only 13 out of 40 non trivial values
are computed.

Design and Analysis of Algorithms - Chapter 38

11
 Terminal Questions
• 1. What is the basic Principle of Optimality?
• 2. What are the properties followed by a binary
search tree?
• 3. Explain the steps for inserting an element in
a binary search tree and give its pseudocode.
• 4. Explain the dynamic programming algorithm
for solving a binary search tree.
• 5. Explain the algorithm to solve the Knapsack
problem using the dynamic programming
method

Design and Analysis of Algorithms - Chapter 39

11
 Self Assessment
Questions
• 1. Principle of Optimality is defined as a basic
dynamic programming principle which helps us
to view problems as a sequence of sub
problems.
• 2. Richard Ernest Bellman , a mathematician,
invented the Principle of Optimality.
• 3. All optimization problems tend to
minimizing cost, time and maximizing Profits
• 4. Binary search trees are node based data
structures used in many system programming
applications for managing dynamic sets.

Design and Analysis of Algorithms - Chapter 40

11
 Self Assessment
Questions
• 5. The Insertion, deletion and search
operations of a binary search tree has an
average case complexity of O(log n)
• 6. The time taken to perform operations on a
binary search tree is directly proportional to
the Height of the tree.
• 7. The Recurrence relation expresses the
problem using its sub-instances.
• 8. Bounded Knapsack problem is an NP-hard
optimization problem.
• 9. The Knapsack problem minimizes the total
Weight and maximizes the total value.
Design and Analysis of Algorithms - Chapter 41
11
 Self Assessment
Questions
• 10. The goal of using Memory functions
is to solve only the sub problems which
are necessary.
• 11. Memory functions use a dynamic
programming technique called
Memoization in order to reduce the
inefficiency of recursion that might occur.
• 12. Memory functions method solves the
problem using Top down approach.

Design and Analysis of Algorithms - Chapter 42

11
 Summary
• Principle of Optimality is defined as a basic
dynamic programming principle which helps us
to view problems as a sequence of sub
problems.
• the binary search tree and explained the various
operations performed on the tree.
• The problem is defined for a set of items where
each item has a weight and a value, and it
determines the number of items that minimizes
the total weight and maximizes the total value.
• the Knapsack problem using the dynamic
programming technique.
Design and Analysis of Algorithms - Chapter 43
11
END

Design and Analysis of Algorithms - Chapter 44

11