Scribd
Upload
62 views4 pages

Complexity Analysis - Difficult Recurrences: Example 1: The Fibonacci Recurrence

The document discusses analyzing the complexity of difficult recurrence relations by finding upper and lower bounds rather than exact solutions. It provides examples of finding bounds for the Fibonacci recurrence (upper bound O(2^n), lower bound Ω(2^n/2)) and a recurrence T(n) = 2*T(n/3) + T(n-1) + c (lower bound Ω(n), upper bound O(3^n)). It warns against incorrect approaches like assuming independent contributions from each term. Finding tighter bounds is desirable through techniques like guessing and proving a bound holds by induction.

Uploaded by

Koutheir EB
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
62 views4 pages

Complexity Analysis - Difficult Recurrences: Example 1: The Fibonacci Recurrence

The document discusses analyzing the complexity of difficult recurrence relations by finding upper and lower bounds rather than exact solutions. It provides examples of finding bounds for the Fibonacci recurrence (upper bound O(2^n), lower bound Ω(2^n/2)) and a recurrence T(n) = 2*T(n/3) + T(n-1) + c (lower bound Ω(n), upper bound O(3^n)). It warns against incorrect approaches like assuming independent contributions from each term. Finding tighter bounds is desirable through techniques like guessing and proving a bound holds by induction.

Uploaded by

Koutheir EB
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 4

Complexity analysis - difficult recurrences

Some recurrences can be difficult to solve mathematically, thus we cannot directly


determine a tight bound (Theta) for their running times.

In this case, we try to determine a lower bound (Omega) and an upper bound (O).

We always should try to do our best:

 find a lower bound which is the highest lower bound that we can prove, and
 find an upper bound which is the lowest upper bound that we can prove.

If we can prove the same function both for lower bound and upper bound, then we
even managed to find the tight bound (Theta).

Example 1: The Fibonacci recurrence


T(n) = c1, if n<=2
T(n) = T(n-1)+T(n-2)+c2, if n>2

The substitution method and recursion tree method lead to something complicated
to solve. We try lower bound and upper bound.

Upper bound for Fibonacci


T(n) is in O(f(n)), if there exist a>0, n0>0, such that T(n)<=a*f(n), for
all n>=n0.
For any algorithm, we have:
T(n-2)<=T(n-1), T(n-3)<=T(n-2), ... T(n-k)<=T(n-k+1)
Taking this into account (replacing T(n-2) with the bigger T(n-1)), the fibonacci
recurrence leads to:
T(n)=T(n-1) + T(n-2)+c2 <= 2*T(n-1) + c2
By substitution,
T(n)<< 2^k * T(n-k) + c2 (2^(k-1) + 2^(k-2) + .... + 2^2 +2 +1)
Substitution stops when n-k=1, k=n-1 Results that
T(n)<= c* 2^n

T(n) is in O(2^n)

Lower bound for Fibonacci


T(n) is in Omega(f(n)), if there exist a>0, n0>0, such that T(n)>=a*f(n),
for all n>=n0.
For any algorithm, we have:
T(n-2)<=T(n-1), T(n-3)<=T(n-2), ... T(n-k)<=T(n-k+1)
Taking this into account, replacing T(n-1) by the smaller T(n-2), the fibonacci
recurrence leads to:
T(n)=T(n-1)+T(n-2)+c2 >= 2*T(n-2) + c2
By substitution,
T(n)>= 2^k *T(n-2*k) + c2 ( 2^(k-1)) + .... + 2^2 +2 +1)
Substitution stops when k=n/2 Results that:
T(n)>= a* 2^(n/2)

T(n) is in Omega(2^n/2)

Counterexamples: how NOT to solve recurrences


The fibonacci relationship contains two recurrences. It is an often mistake to try
to isolate the contribution of each recursive term if these terms are not
independent, if they are the same function.

Bad solution 1

We know that recurrences such as


T(n)=T(n-1)+c or T(n)=T(n-2)+c
both lead to linear functions which are in Theta(n). BUT we cannot say that in the
fibonacci recurrence relationship each term has a linear contribution, thus
Fibonacci should be of linear complexity, which would be obviously WRONG !

Bad solution 2

In case of the Fibonacci recurrence, try to single out the "contribution" of one
recursive term, and use the result in the general recurrence.
T(n)=T(n-1)+T(n-2)+c
is rewritten as
T(n)=T(n-1)+f(n)+c
where f(n) results from solving the recurrence
T(n)=T(n-2) + c
It results that f(n) is in Theta(n), which we replace back into the general recurrence,
which becomes:
T(n)=T(n-1)+n+c
This can be easily solved by substitution and found to be in Theta (n*n), which is
also WRONG. The problem with this "solution" is that it starts with the wrong
assumption that f(n) can be calculated independently from T(n).

Example 2:
We have the following recurrence:
T(n) = 2 *T(n/3)+T(n-1)+c

Example 2 - What NOT to do

First of all, this is not a case where we can apply the Master theorem. The Master
theorem provides results for recurrences of the form
T(n) = a * T(n/b) + f(n)
But an important assumption of the Master theorem is that the function f(n)
is independent from the function T(n) ! Thus we cannot say that f(n)=T(n-1) in
our case !

Also, similarly to the Bad Solution 2 of Fibonacci, we can not try to rewrite the
initial recurrence as
T(n) = 2 * T(n/3) + f(n) + c, where
f(n)=f(n-1) + c
This approach is NOT correct and it will not lead to a correct solution, as we have
shown abowe in the counter example Bad Solution 2 of Fibonacci.

The substitution method and recursion tree method lead to something complicated
to solve. (The recursion tree has an irregular form, with heights ranging from
log_3(n) to n, and each level contains nodes which are not homogenous from the
point of view of their workload).

So, what CAN we do ?

Example 2 - Lower and upper bounds

We try to find lower and upper bounds, proceeding in a way which is similar with
what we did with Fibonacci. We have:
T(n/3)<=T(n-1)
If we replace T(n/3) by T(n-1) in the initial recurrence, we can determine an upper
bound:
T(n) = 2 *T(n/3)+T(n-1)+c <=3 * T(n-1) + c
By substitution,
T(n) <= 3^k *T(n-k) + c *(3^(k-1) + ... + 3^2 + 3 + 1)
Results
T(n)<=c*3^n
T(n) is in O(3^n)
If we replace T(n-1) by T(n/3) in the initial recurrence, we can determine a lower
bound:
T(n) = 2 *T(n/3)+T(n-1)+c >= 3 * T(n/3) + c
By substitution (or the master method),
T(n) is in Omega(n)
We have found a lower bound Omega(n) and an upper bound O(3^n) for
Example2.

These two bounds are quite far away, we should try to reduce the distance between
them - either finding a lower bound which is higher, or a higher bound which is
smaller.

Another technique for finding bounds is based on (intelligent) guess and prove: we
assume that a function is a bound and try to prove this by induction.
Computing bounds by Guess and Prove

Is it polynomial ?

We make the hypothesis that the lower bound is a polinomial n^k. Thus T(n) is in
Omega(n^k) if there are a>0, n0>0, such that
T(n)>=a* n^k, for all n>=n0
We assume that this holds for n-1 and n/3:
T(n/3) >= a* (n/3)^k

T(n-1) >= a* (n-1)^k


We try to prove that this implies that
T(n) >= a * n^k
T(n)=2*T(n/3)+T(n-1)+c >=2*a*(n/3)^k + a * (n-1)^k +c >= a * n^k, true for
all n>=nn0
T(n) is in Omega(n^k), for any k>=1 (the complexity of T(n) is bigger than any
polynomial)

Is it exponential ?

We already have that 3^n is an upper bound. We make the hypothesis that the
upper bound is an exponential b^n, b>1. Thus T(n) is in O(b^n) if there are a>0,
n0>0, such that
T(n)<=a* b^n, for all n>=n0
We assume that this holds for n-1 and n/3:
T(n/3) <= a* b^(n/3)
T(n-1) <= a* b^(n-1)
We try to prove that this implies that
T(n) <= a * b^n
T(n)=2*T(n/3)+T(n-1)+c <=2*a*b^(n/3) + a * b^(n-1) +c <= a * b^n, true for
all n>=nn0
T(n) is in O(b^n), for any b>1. The upper bound is exponential.

We can not prove that the lower bound is also exponential.

Thus for Example 2, we could only prove that its complexity is bigger than any
polynomial but smaller than an exponential. Further work on refining these bounds
is welcome ;-)

You might also like