Scribd
Upload
638 views3 pages

Lower Bound Theory

This lecture discusses techniques for establishing lower bounds on algorithms. A lower bound is an estimate of the minimum number of operations needed to solve a problem. There are several techniques for finding lower bounds, including trivial counting of inputs/outputs, information theory, decision trees, oracles/adversaries, and problem reduction. Problem reduction works by showing that if a faster algorithm existed for one problem, it could be used to solve another problem faster than its known lower bound.

Uploaded by

tay
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
638 views3 pages

Lower Bound Theory

This lecture discusses techniques for establishing lower bounds on algorithms. A lower bound is an estimate of the minimum number of operations needed to solve a problem. There are several techniques for finding lower bounds, including trivial counting of inputs/outputs, information theory, decision trees, oracles/adversaries, and problem reduction. Problem reduction works by showing that if a faster algorithm existed for one problem, it could be used to solve another problem faster than its known lower bound.

Uploaded by

tay
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 3

Lecture #1: Lower Bound Theory

Text

 Lower bound: an estimate of a number of operations needed to solve a given


problem.
 Tight Lower Bound: There exists an algorithm with the same efficiency as the
lower bound.

If two algorithms for solving the same problem were discovered and their times


differed by an order of magnitude, then the one with the smaller order was generally
regarded as superior. But still we are left with the question: is there a faster method?
The purpose of this lecture is to expose you to some techniques that have been used
to establish that a given algorithm is the most efficient possible. The way this is done
is by discovering a function g(n) that is a lower bound on the time that any
algorithm must take to solve the given problem. If we have
an algorithm whose computing time is the same order as g(n), then we
know that asymptotically we can do no better.  If f(n) is the time for some algorithm, then
we write f(n) = Ω(g(n)) to mean that g(n) is a lower bound for f(n).
Formally, this equation can be written if there exist positive constants c and no such
that |f(n)| ≥ c |g(n)| for all n > no. In addition to developing lower bounds to within a
constant factor, we are also concerned with determining more exact bounds whenever
this is possible. Deriving good lower bounds is often more difficult
than devising efficient algorithms. Perhaps this is because a lower bound states a fact
about all possible algorithms for solving a problem. Usually we cannot enumerate
and analyze all these algorithms, so lower bound proofs are often hard to
obtain. However, for many problems it is possible to easily observe that a lower bound
identical to n exists, where n is the number of inputs (or possibly outputs) to the
problem. For example, consider all algorithms that find the maximum of an unordered
set of n integers. Clearly every integer must be examined at least once, so Ω(n) is a
lower bound for any algorithm that solves this problem.

Techniques for finding lower bounds:

1. Trivial Lower Bounds:


The method consists of simply counting the number of inputs that must be
examined and the number of outputs that must be produced.

Example-1: Multiplication of a pair of nxn matrices

 requires that 2n2 inputs be examined and


 n2 outputs be computed, and
 the lower bound for this matrix multiplication problem is therefore Ω (n 2).

2. Information Theory:

The information theory method establishing lower bounds by computing


the limitations on information gained by a basic operation and then showing how
much information is required before a given problem is solved.

This is used to show that any possible algorithm for solving a problem must do
some minimal amount of work.

3. Decision tree:

This method can model the execution of any comparison based problem.

 The tree contains the comparisons along all possible instruction traces.
 The running time of the algorithm = the length of the path taken.
 Worst-case running time = the height of tree.

4. Oracles and Adversary Arguments:

Another technique for obtaining lower bounds consists of making use of an


“oracle.”

 Given some model of computation such as comparison trees, the oracle


tells us the outcome of each comparison.
 In order to derive a good lower bound, the oracle tries its best to cause the
algorithm to work as hard as it might.
 It does this by choosing as the outcome of the next test, the result which
causes the most work to be required to determine the final answer.
 And by keeping track of the work that is done, a worst-case lower bound
for the problem can be derived.

5. Problem Reduction:

Another elegant means of proving a lower bound on a problem is to show that


an algorithm for solving a problem along with a transformation on problem instances
could be used to construct an algorithm to solve another problem for which a lower
bound is known.
Lower Bound Theory through Reductions
Problem-solving framework that transforms a problem X into a simpler problem Y such
that it is easy to deduce a solution to the original problem X based on the solution for Y.

This technique calls for reducing the given problem to another problem for which a
lower bound is already known. Let P1 and P2 be any two problems. We say P1 reduces
to P2 (also written P1 α P2) in time T(n) if an instance of P1 can be converted into an
instance of P2 and a solution for P1 can be obtained from a solution for P2 in time < T(n).

Example-1: An algorithm for finding the Euclidean minimum spanning tree of n points
in the plane can be used to solve the element uniqueness problem, and must therefore
take time Ω(nlgn). The reduction is quite simple:

 Suppose we want to determine whether any two of the numbers x 1,x2,…,xn are
equal. We can solve this problem by giving any Euclidean MST algorithm
the points (x1,0), (x2,0),…,(xn,0). The two closest points are known to be joined by
one of the (n-1) spanning-tree edges, so, we can scan these edges in linear
time, determine if any edge has zero length. Such an edge exists if two of x iꞌs are
equal. Therefore, if the spanning tree algorithm could operate in less than
O(nlgn) time, then the element uniqueness problem could be solved in less than
O(nlgn) time too.

You might also like