Design And Analysis of Algorithm – Study material (B.

Sheik Md Abdullah ,MIIT)

Unit 5 LOWER BOUND THEORY

If two algorithm for solving the problem where discovered and their times differed by an
order of magnitude, the one with the smaller order was generally regarded as superior.

THE PURPOSE OF LOWER BOUND THEORY

Is to find some techniques that have been used to establish that a given alg is the most
efficient possible.

THE SOLUTION OR TECHNIQUE

It is by discovering a function g(n) that is lower bound on the time that any algorithm
must take to solve the given problem. If we have an algorithm whose computing time is
the same order as g(n) then we know that asymptotically we can do no better (i.e. this alg
is the most efficient one, no other alg can be derived more efficient then this)

TRIVAL LOWER BOUNDS

Lower bounds which are so easy to obtain is called trivial lower bounds.

E.g All alg that finds the maximum of are unordered set of n integers. Clearly every
integer must be examined at least once. So (n) is a lower bound for any alg that solves
this problem.

SOME TECHNIQUES OR COMPUTATIONAL MODEL FOR FINDING

LOWER BOUNDS.

1. Comparison trees.
2. Oracle and adversary argument
3. Lower bound through reductions.

1. Comparison Trees

a) Ordered Searching

• Only comparison – based algorithms are considered

• The process of searching algorithms can be described by a path
in a binary tree.
• Each internal node in this tree represent a comparision between
rc and A[i]
• There are three possible out come of this comparison.
X < a[I] | x = a[I] | x > a[I]
• If x = a[i] the lag terminates.
• If the alg terminates following a left or right branch, then a[I] has
been found s.t x = a[I] & alg must declare the search
unsuccessful.
Design And Analysis of Algorithm – Study material (B.Sheik Md Abdullah ,MIIT)

Theorem 10.1

Let a[1:n], n> 1 contains n distinct elements, ordered that a[1] < . . . < A[n]. Let Find
(n) be the minimum number of comparisons needed, in the worst case, by any
comparison based alg. To recognize whether X ∈ A [1:n]. The FIND (n)≥ ≥ log (n+1)

Proof

• Consider all possible comparison trees that model alg to solve the searching
problem
• FIND(n) – Lessthan the distance of the longest path from the root to a leaf node.
• There must be n internal nodes in all the these trees corresponding to the n
possible successful occurrences of x in A
• If all internal nodes of a binary tree are at levels less than or equal to K, them
there are at most 2k-1 internal nodes.
• Thus n < 2 k-1 and FIND (n) = k log (n+1). Hence Binary search is an optimal
worst – case algon for solving the searching problem.

B) Sorting

• Consider the sorting algs (comparison – based. )

Design And Analysis of Algorithm – Study material (B.Sheik Md Abdullah ,MIIT)

• Consider the case in which the n numbers a [1:n] to be sorted are distinct.
• Any comparison between a[I] and a[j] will result in one of two possibilities
A [I] < A [j] or A [j] > A [j]
• So comparison tree is a binary tree in which each internal node is labeled by the
pair I:j which represent the comparison of A[I], A[j]
•
• If A[I] is less than A[j] then the alg. Proceeds down the left branch of the tree,
otherwise it produces down the right branch.
• External nodes represent termination of the alg.
• There are n! different possible permutation of n items, and any one of there might
legitimately be the only correct answer for sorting problem.
• So comparison tree must have at least n! external nodes
E.g comparison tree for sorting three items

• Consider only the comparison based sorting algorithms.

• Let T(n) be the minimum number of comparisons that are sufficient to sort n
items in the worst case.
• Casing knowledge of Binary trees) if all internal nodes are at levels less than k,
then there are at most 2k external nodes (one more than no. of external nodes.)
• Therefore, if we let k = T(n)
N! ≤ 2T(n)
Since T(n) is an integer, we get the lower bounds T(n) ≥ log n!

By stirlings approximation it follows that

Log n! = n log n – n /ln2 + (1/2) log n + O(1)

• This formula shows that T(n) is of the order n log n.

• Hence any comparison based sorting algorithm needs (n log n) time.
Design And Analysis of Algorithm – Study material (B.Sheik Md Abdullah ,MIIT)

C ) Selection:

Let Lk(n) denote a lower bound for the number of comparisons necessary for a
compression necessary for a comparison – based algorithm to determine the largest,
second largest, kth largest out of n elements, in the worst case.

L1 (n) = n-1 as previously

Since comparison tree must contain enough external nodes to allow for any
possible permutation of the input, it follows immediately that Lk(n) ≥[log n (n-1) ……
(n-k+1)].

2. Oracle And Adversary Arguments

Given some model of computation such as comparison trees the oracle tells us the
outcome of each comparison.

It does this by choosing as the outcome of next test , the result causes the most
work to be required to determine the final answer .By keeping track of work that
is done , a worst case lower bound for the problem can be derived

e.g: Merging

Given the sets A[1:m] and B[1:n] , where the items in A & B are sorted , our problem is
to investigate lower bounds for algorithms that merges these two sets to give a single
sorted array.
Let all the m+n elements are distinct and that A[1] < A[2] … < A[m] and B[1] < B[2]
… < B[n]

Elementary combinatories tells as that there are m (m + n

)
ways that the A’s and B’s can
merge together while preserving the ordering with in A & B.

E.g 1: if m=3, n=2 ; A[1]=x, A[2] =y,A[3] =z; B[1] =u, B[2] =v.
n
m
Then there are ( 3 + 2
3 )
= 10 ways in which A & B can merge: Means n! / [ m!(n-m)!]
(u,v,x,y,z) , (u,x,v,y,z) ,(u,x,y,v,z),…. (x,y,z,u,v)

Thus if we use comparison trees on our model for merging algorithm , then there
will be (m
m
+ n
)
external nodes, and therefore at least log ( mm+ n)
comparisons are required by any comparison based algorithm.
Design And Analysis of Algorithm – Study material (B.Sheik Md Abdullah ,MIIT)

E.g 2. Largest & Second largest

Any comparison –based algorithm that computes the largest and second largest of a set of
n unsorted elements require n-2 + log n computations.

Problem given in your book- page no 468.(A tennis tournament – no need to write
theorem proof)

3. Lower Bound Through Reduction

A technique that can be used to derive lower bound of a given problem by

reducing the problem to another problem for which a lower bound is already
known.

Let P1 ,P2 are any two problems.P1 reduces to P2 in time T(n) if an instance P1
can be converted in to an instance of P2 & a solution for P1 can be obtained from
a solution for P2 in time <= T(n)

E.g 1
Let P1 be the problem of selection and P2 be the problem of sorting.Let the input have n
numbers .If the numbers are sorted , say in an array A[] , the ith smallest element of the
input can be obtained as A[i].Thus P1 reduces to P2 in O(1) time.

E.g 2 Finding the Convex Hull

Given n points in a plane the convex hull ,Problem is to identify the vertices of the Hull
in some Order (Clock wise or anti clock wise order) . We can show that any algorithm for
this problem takes Ω(n log n) time. If we call the convex hull Problem P2 and the
problem of sorting P1 , then this lower bound is obtained by showing that P1 reduces to
P2 in O(n) time. Since sorting of n Points needs Ω(n log n),the result readily follows.

E.g
Consider the problem of sorting number 2,3,1 & 4. The four points created are (2,4)
,(8,9), (1,1), (4,16).The convex hull of these points is shown in the following fig. All the
four Points are on the hull. A counter clock wise ordering of hull is (3,9), (4,16), (1,1),
(2,4) from which the sorted order of the points can be retrieved.

Therefore convex hull of n given points in the plane needs Ω(n log n) time.
Design And Analysis of Algorithm – Study material (B.Sheik Md Abdullah ,MIIT)

Chapter-11

NP-HARD AND NP-COMPLETE PROBLEMS

Def: Polynomial complexity:

An algorithm A is of polynomial complexity if there exists a polynomial p( )
such that the computing time of A is O(p(n)) for every input of size n.
Eg:

* Algorithm having complexity like O(2nn2) or O(2n/2) are nonpolynomial

* Algorithm having complexity like O(n), O(log n), (nlogn),O(m,n) are
polynomial.

Def: Non Deterministic algorithms:

The algorithms, which contain operations whose outcomes are not uniquely
defined but are limited to, specified sets of possibilities are called
NonDeterministic algorithms.

Def: Decision problem

Any problem for which the answer is either zero or one is called a decision
problems.

Def: P & NP
P is the set of all decision problems solvable by deterministic algorithms in
polynomial time. NP is a set of all decision problems solvable by
nondeterministic algorithms in polynomial time. (e.g Nondeterministic sorting
book page 498)
Design And Analysis of Algorithm – Study material (B.Sheik Md Abdullah ,MIIT)

Example for Nondeterministic Knapsack algorithm

Algorithm DKP(p,w,n,m,r,x)
{
w:=0;p:=0;
for i=1 to n do
{
x[i]=Choice(0,1)
w:= w+x[i]*w[i];
p:=p+x[i]*p[i];
}
* if((w>m) or (p<r)) then Failure();
else
Success();
}

new functions:
1. Choice(S) arbitrarily chooses one of the elements of set S.
2. Failure() signals an unsuccessful completion.
3. Success() signals a successful completion.
• Functions 2,3 cannot be used to effect a “return” statement.
• A nondeterministic algorithm terminates unsuccessfully iff there exists no
set of choices leading to a success signal.
• Computing time for Choice, Success, Failure are taken to be O(1).
• The marked line (*) in the algorithm checks to see whether this
assignment is feasible and whether the resulting profit is at least r.
• A successful termination is possible iff the answer to the decision problem
is yes. The time complexity is O(n).
• If q is the i/p length using a binary representation then time is O(q).

Relationship between P and NP

Since deterministic algorithms are just a special case of nondeterministic ones, we

conclude that P C NP. But most famous unsolved problem in computer science is
whether P=NP or P ≠ NP

NP
P
Design And Analysis of Algorithm – Study material (B.Sheik Md Abdullah ,MIIT)

Def: NP- hard problem & NP complete

• A problem L is NP-hard iff satisfiabily reduces to L (satisfiability α L).
• A problem L is NP-complete iff L is NP-hard and L Є NP.
Note:
Problem L1 reduces to L2 iff there is a way to solve L1 by a deterministic polynomial
time algorithm using a deterministic algorithm that solves L2 in polynomial time.

Relationship between NP-hard & NP-complete:

• A problem that is NP-complete has the property that it can be solved in
polynomial time iff all other NP-complete problem can also be solved in
polynomial time.
• If an NP-Hard problem can be solved in polynomial time then all NP-
complete problems can be solved in polynomial time.
• All NP-complete problems are NP-hard , but some NP-hard problems are
not known to be NP-complete.

From the figure all NP-hard problems that are not NP complete problems. Only a
decision problem may be NP-complete. Optimization problems are NP-hard.

a)If L1 is a decision problem and L2 an optimization problem it is quite possible that

L1 α L2. Example knapsack decision problem reduces to knapsack optimization problem.

a) Another conflict type is NP-hard decision problem that are not NP-
complete. Example: Halting problem for deterministic algorithms.

b) From the above discussion optimization problems cannot be NP-complete

where as decision problem can
Design And Analysis of Algorithm – Study material (B.Sheik Md Abdullah ,MIIT)

Theorem 11.1 COOK’S THEOREM

Statement:

Satisfiability is in P if and only if P=NP

Proof:

I Poly.time Algorithm Poly time

transform for transform
L2 Solution
for I

From the definition satisfiability is in NP.

Hence, if P=NP, then satisfiability is in P can be proved.

To do this, we show how to obtain form any polynomial time non-deterministic

decision algorithm A and input I a formula Q(A,I) such that Q is satisfiable iff A
has a successful termination with input I.

If the length of I is n & the time complexity of A is p(n) for some polynomial p()
then the time needed to construct Q is O(p3(n) log n)

A deterministic algorithm Z to determine the outcome of A on any input I can be

computed as follows.
o If O(q(m)) is the time needed to find whether a formula of length m is
satisfiable then the complexity of z = O(p3(n) log n + q(p3(n)(log n))
If satisfiaility is in P then q(m) is a polynomial function of m then complexity of
Z=O(r(n)) for some polynomial r().

Hence, if satisfiability is in P, then for every non-deterministic algorithm A in NP,

we can obtain a deterministic Z in P.

The above construction shows that if satisfiability is in P, then P=NP.

Design And Analysis of Algorithm – Study material (B.Sheik Md Abdullah ,MIIT)

AND/OR graph:

The complex problem can be divided into sub problems and solutions of either all
sub problems or one or more sub problems is the solution to the original problem.

The problem can be represented as directed tree in which root node represents the
problem and the decedents represents the sub problem.

Consider the problem A. It can be solved either by solving both B & C. sub
problems and D or E sub problems.

A A A A

We can also introduce dummy arguments so that their solution is obtained by

solving B & C or D or E.

The solution of B and C is solution of A’ and it is called AND node.

The solution of either D or E is the solution to A’’ and it is called OR node.

A

A’ A’’

B C D E

The node without descendents are called as Terminal nodes.

The terminal nodes represent the primitive problems. The terminal nodes, which
are solvable, or denoted by rectangle.
Design And Analysis of Algorithm – Study material (B.Sheik Md Abdullah ,MIIT)

Algorithm Solve (T) is used to determine whether the problem is solvable or not.

Algorithm: -
Algorithm Solve (T)
// T is the AND/OR tree and T is the root node.
// It will return 1 if the problem is solvable.
//Otherwise 0.
{ case : T is terminal node : if T is solvable then
Return (1);
Else Return (0);
Endif;

T is AND node : for (each child S o f T) do

If solve (S) = 0 then
Return (0);
Endif;
Repeat
Return (1);

T is OR node : For (each child S of T) do

If Solves (S) = 1 then
Return (1);
Endif;
Repeat
Return(0);

Endcase; }