Scribd
Upload
38 views29 pages

Algorithms and Data Structures: Simonas Šaltenis

This document discusses analyzing the running time of recursive algorithms using recurrences. It introduces recurrences as equations that describe a function's value based on smaller inputs. Various methods are presented for solving recurrences, including repeated substitution, recursion trees, and the master method. The master method provides templates to classify recurrences into cases based on how their runtime is distributed.

Uploaded by

shruti5488
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
38 views29 pages

Algorithms and Data Structures: Simonas Šaltenis

This document discusses analyzing the running time of recursive algorithms using recurrences. It introduces recurrences as equations that describe a function's value based on smaller inputs. Various methods are presented for solving recurrences, including repeated substitution, recursion trees, and the master method. The master method provides templates to classify recurrences into cases based on how their runtime is distributed.

Uploaded by

shruti5488
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 29

September 25, 2002 1

Algorithms and Data


Structures
Lecture IV
Simonas altenis
Aalborg University
[email protected]
September 25, 2002 2
This Lecture
Analyzing the running time of recursive
algorithms (such as divide-and-conquer)
Writing and solving recurrences
September 25, 2002 3
Recurrences
Running times of algorithms with Recursive
calls can be described using recurrences
A recurrence is an equation or inequality that
describes a function in terms of its value on
smaller inputs.
For divide-and-conquer algorithms:


Example: Merge Sort
(1) if 1
( )
2 ( / 2) ( ) if 1
n
T n
T n n n
O =

+O >

solving_trivial_problem if 1
( )
num_pieces ( / subproblem_size_factor) dividing combining if 1
n
T n
T n n
=

+ + >

September 25, 2002 4


Solving Recurrences
Repeated (backward) substitution method
Expanding the recurrence by substitution and noticing
a pattern
Substitution method
guessing the solutions
verifying the solution by the mathematical induction
Recursion-trees
Master method
templates for different classes of recurrences
September 25, 2002 5
Repeated Substitution
Lets find the running time of the merge sort
(lets assume that n=2
b
, for some b).

1 if 1
( )
2 ( / 2) if 1
n
T n
T n n n
=

+ >

( )
( ) ( )
2
2
3
lg
( ) 2 / 2 substitute
2 2 / 4 / 2 expand
2 ( / 4) 2 substitute
2 (2 ( / 8) / 4) 2 expand
2 ( / 8) 3 observe the pattern
( ) 2 ( / 2 )
2 ( / ) lg lg
i i
n
T n T n n
T n n n
T n n
T n n n
T n n
T n T n in
T n n n n n n n
= +
= + +
= +
= + +
= +
= +
= + = +
September 25, 2002 6
Repeated Substitution Method
The procedure is straightforward:
Substitute
Expand
Substitute
Expand

Observe a pattern and write how your expression
looks after the i-th substitution
Find out what the value of i (e.g., lgn) should be to
get the base case of the recurrence (say T(1))
Insert the value of T(1) and the expression of i into
your expression

September 25, 2002 7
Repeated Substitution (Ex. 2)
Lets find a more exact running time of merge
sort (lets assume that n=2
b
, for some b).

2 if 1
( )
2 ( / 2) 2 3 if 1
n
T n
T n n n
=

+ + >

( )
( ) ( )
( )
( ) ( )
( ) ( )
2
2
3 3 2 1 0
( ) 2 / 2 2 3 substitute
2 2 / 4 3 2 3 expand
2 / 4 4 2 3 3 substitute
2 2 / 8 / 2 3 4 2 3 3 expand
2 / 2 2 3 2 +2 +2 3 observe the pattern
( ) 2 ( / 2 ) 2 3 2
i i j
j
T n T n n
T n n n
T n n
T n n n
T n n
T n T n in
=
= + +
= + + + +
= + + +
= + + + + +
= + +
= + +
1
0
lg lg
2 ( / ) 2 lg 3(2 1) 5 2 lg 3
i
n n
T n n n n n n n

= + + = +

September 25, 2002 8


Repeated Substitution (Ex. 3)
Lets find the running time of the tromino tiling
algorithm for a 2
n
x2
n
board.

1 if 1
( )
4 ( 1) 1 if 1
n
T n
T n n
=

+ >

( )
( ) ( )
( )
( ) ( )
( ) ( )
2
2
3 2 1 0
1
0
1
1
( ) 4 1 1 substitute
4 4 2 1 1 expand
4 2 4 1 substitute
4 4 3 1 4 1 expand
4 3 4 +4 +4 observe the pattern
( ) 4 ( ) 4
4 1 4 1
4 (1)
4 1 3
i
i j
j
n n
n
T n T n
T n
T n
T n
T n
T n T n i
T

= +
= + +
= + +
= + + +
= +
= +

= + =

September 25, 2002 9


Substitution method
3 3
3
3
3
3
Solve ( ) 4 ( / 2)
1) Guess that ( ) ( ), i.e., that of the form
2) Assume ( ) for / 2 and
3) Prove ( ) by induction
( ) 4 ( / 2) (recurrence)
4c(n/2) (ind. hypoth.)
(si
2
T n T n n
T n O n T cn
T k ck k n
T n cn
T n T n n
n
c
n n
= +
=
s s
s
= +
s +
= +
3 3
3
3
mplify)
(rearrange)
2
if 2 and 1 (satisfy)
Thus ( ) ( )!
Subtlety: Must also check the base case (not necessarily (1))
c
cn n n
cn c n
T n O n
T
| |
=
|
\ .
s > >
=
September 25, 2002 10
Substitution Method
Achieving tighter bounds
2
2
2
2
2
Try to show ( ) ( )
Assume ( )
( ) 4 ( / 2)
4 ( / 2)
for no choice of 0.
T n O n
T k ck
T n T n n
c n n
cn n
cn c
=
s
= +
s +
= +
s >
September 25, 2002 11
Substitution Method (2)
The problem? We could not rewrite the equality

as:

in order to show the inequality we wanted
Sometimes to prove inductive step, try to
strengthen your hypothesis
T(n) s (answer you want) - (something > 0)

2
( ) + (something positive) T n cn =
2
( ) T n cn s
September 25, 2002 12
Substitution Method (3)
Corrected proof: the idea is to strengthen
the inductive hypothesis by subtracting
lower-order terms!

2
1 2
2
1 2
2
1 2
2
1 2 2
2
1 2 2
Assume ( ) for
( ) 4 ( / 2)
4( ( / 2) ( / 2))
2
( )
if 1
T k c k c k k n
T n T n n
c n c n n
c n c n n
c n c n c n n
c n c n c
s <
= +
s +
= +
=
s >
September 25, 2002 13
Recursion Tree
A recursion tree is a convenient way to visualize
what happens when a recurrence is iterated
Good for guessing asymtotic solutions to recurrences
2
( ) ( / 4) ( / 2) T n T n T n n = + +
September 25, 2002 14
Recursion Tree (2)
September 25, 2002 15
Master Method
The idea is to solve a class of recurrences that
have the form

a > 1 and b > 1, and f is asymptotically positive.
Abstractly speaking, T(n) is the runtime for an
algorithm and we know that
a subproblems of size n/b are solved recursively, each
in time T(n/b)
f(n) is the cost of dividing the problem and combining
the results. In merge-sort
( ) ( / ) ( ) T n aT n b f n = +
( ) 2 ( / 2) ( ) T n T n n = + O
September 25, 2002 16
Master Method (2)
Split problem into a parts at log
b
n
levels. There are leaves
log log
b b
n a
a n =
September 25, 2002 17
Master Method (3)
Number of leaves:
Iterating the recurrence, expanding the tree
yields





The first term is a division/recombination cost (totaled
across all levels of the tree)
The second term is the cost of doing all subproblems
of size 1 (total of all work pushed to leaves)
log log
b b
n a
a n =
2 2
2 2
log 1 log 1 log
log 1
log
0
( ) ( ) ( / )
( ) ( / ) ( / )
( ) ( / ) ( / ) ...
( / ) (1)
Thus,
( ) ( / ) ( )
b b b
b
b
n n n
n
a j j
j
T n f n aT n b
f n af n b a T n b
f n af n b a T n b
a f n b a T
T n a f n b n

=
= +
= + +
= + + +
+ +
= + O

log
b
a
n
September 25, 2002 18
MM Intuition
Three common cases:
Running time dominated by cost at leaves
Running time evenly distributed throughout
the tree
Running time dominated by cost at the root
Consequently, to solve the recurrence, we
need only to characterize the dominant
term
In each case compare with

( ) f n
log
( )
b
a
O n
September 25, 2002 19
MM Case 1
for some constant
f(n) grows polynomially (by factor ) slower
than
The work at the leaf level dominates
Summation of recursion-tree levels
Cost of all the leaves
Thus, the overall cost

log
( ) ( )
b
a
f n O n
c
=
log
b
a
n
0 c >
n
c
log
( )
b
a
O n
log
( )
b
a
n O
log
( )
b
a
n O
September 25, 2002 20
MM Case 2

and are asymptotically the same
The work is distributed equally
throughout the tree
(level cost) (number of levels)
log
( ) ( )
b
a
f n n =O
( ) f n
log
( ) ( lg )
b
a
T n n n =O
log
b
a
n
September 25, 2002 21
MM Case 3
for some constant
Inverse of Case 1
f(n) grows polynomially faster than
Also need a regularity condition

The work at the root dominates
log
( ) ( )
b
a
f n n
c +
=O
log
b
a
n
0 0
1 and 0 such that ( / ) ( ) c n af n b cf n n n - < > s >
( ) ( ( )) T n f n = O
0 c >
September 25, 2002 22
Master Theorem Summarized
Given a recurrence of the form







The master method cannot solve every recurrence of this
form; there is a gap between cases 1 and 2, as well as
cases 2 and 3
( ) ( / ) ( ) T n aT n b f n = +
( )
( )
( )
( )
( )
( )
log
log
log
log
log
0
1. ( )
( )
2. ( )
( ) lg
3. ( ) and ( / ) ( ), for some 1,
( ) ( )
b
b
b
b
b
a
a
a
a
a
f n O n
T n n
f n n
T n n n
f n n af n b cf n c n n
T n f n
c
+c
=
= O
= O
= O
= O s < >
= O
September 25, 2002 23
Strategy
Extract a, b, and f(n) from a given recurrence
Determine
Compare f(n) and asymptotically
Determine appropriate MT case, and apply
Example merge sort
log
b
a
n
( )
( )
2
log log 2
log
( ) 2 ( / 2) ( )
2, 2; ( )
Also ( ) ( )
Case 2 ( ) lg lg :
b
b
a
a
T n T n n
a b n n n n
f n n
T n n n n n
= +O
= = = = = O
= O
= O = O
log
b
a
n
September 25, 2002 24
Examples of MM
( )
2
3
log 1
log 9
2
( ) ( / 2) 1
1, 2; 1
also ( ) 1, ( ) (1)
( ) (lg )
(
Case 2:
Cas
) 9 ( / 3)
9, 3;
( ) , ( ) ( ) with 1
( ) e 1:
T n T n
a b n
f n f n
T n n
T n T n n
a b
f n n f n O n
T n n
c
= +
= = =
= = O
= O
= +
= =
= = c =
= O
Binary-search(A, p, r, s):
q(p+r)/2
if A[q]=s then return q
else if A[q]>s then
Binary-search(A, p, q-1, s)
else Binary-search(A, q+1, r, s)
September 25, 2002 25
Examples (2)
4
4
2
log 3 0.793
log 3
log 2 1
( ) 3 ( / 4) lg
3, 4;
( ) lg , ( ) ( ) with 0.2
Regularity condition
( / ) 3( / 4) l
Case
g( / 4) (3/ 4) lg ( ) for 3/ 4
( ) ( lg )
( ) 2 ( / 2) lg
2, 2;
3:
T n T n n n
a b n n
f n n n f n n
af n b n n n n cf n c
T n n n
T n T n n n
a b n n
f
+c
= +
= = =
= = O c ~

= s = =
= O
= +
= = =
1
1
( ) lg , ( ) ( ) with ?
also l
neither Case 3 nor Case 2!
g / lg
n n n f n n
n n n n
+c
= = O c
=

September 25, 2002 26


Examples: Multiplication (I)
Multiplying two n-digit (or n-bit) numbers
costs n
2
digit multiplications using a
classical procedure.
Observation:
23*14 = (210
1
+3)*(110
1
+4) =
(2*1)10
2
+ (3*1 + 2*4)10
1
+ (3*4)
To save one multiplication we use a trick:
(3*1 + 2*4) = (2+3)*(1+4) - (2*1) - (3*4)
September 25, 2002 27
Examples: Multiplication (II)
To multiply a and b, which are n-digit numbers,
we use a divide and conquer algorithm. We split
them in half:
a = a
1
10
n/2
+ a
0
and b = b
1
10
n/2
+ b
0

Then:
a *b = (a
1
*b
1
)10
n
+ (a
1
*b
0
+ a
0
*b
1
)10
n/2
+ (a
0
*b
0
)
Use a trick to save a multiplication:
(a
1
*b
0
+ a
0
*b
1
) = (a
1
+a
0
)*(b
1
+b
0
) - (a
1
*b
1
) - (a
0
*b
0
)
September 25, 2002 28
Examples: Multiplication (III)
The number of single-digit multiplications
performed by the algorithm can be described by
a recurrence:
1 if 1
( )
3 ( / 2) if 1
n
T n
T n n
=

>

Solution:
2
log 3 1.585
( ) T n n n = =
September 25, 2002 29
Next Week
Sorting
QuickSort
HeapSort

You might also like