LOWER BOUND

THEORY
UNIT-5
LOWER BOUND THEORY

Lower Bound Theory Concept is based upon the calculation of minimum time or
minimum number of comparisons that is required to execute an algorithm is
known as a lower bound theory or Base Bound Theory.
Concept/Aim: The main aim is to calculate a minimum number of comparisons
required to execute an algorithm

LINK (more detail): Lower bound theory and introduction to parallel

algorithm by Prof. Abhigyan / notations lower bound and upper bound
DAA Lower bound Theory - javatpoint
Lower Bound Theory uses a
number of methods/techniques
to find out the lower bound.
TECHNIQUES

1.Comparisons Trees.
2.Oracle and adversary argument
3.State Space Method
COMPARISON TREE
UNIT-5
COMPARISON TREE

A comparison tree, also known as a decision tree or binary search tree, is

a fundamental data structure used in computer science and data
analysis. It's often utilized in algorithms and data structures courses, as
well as in various applications such as sorting, searching, and decision-
making processes.
HERE'S A BASIC OVERVIEW OF HOW A COMPARISON TREE
WORKS:

1.Nodes: Each node in the tree represents an element or a decision point.

2.Edges: Edges connect the nodes and represent relationships between elements or decisions.
3.Root Node: The topmost node in the tree, from which all other nodes are descended.
4.Internal Nodes: Nodes in the tree that have child nodes.
5.Leaf Nodes: Nodes in the tree that do not have any child nodes. They represent the endpoints
of the decision-making process.
6.Binary Tree: A type of comparison tree where each internal node has at most two child nodes.
7.Comparison Operation: At each internal node, a comparison operation is performed to
determine which branch to follow based on the comparison result.
HERE'S AN EXAMPLE OF A SIMPLE
COMPARISON TREE FOR SORTING INTEGERS:

In this example, if you were to search for the

number 4, you would start at the root (5), compare
4 with 5, then traverse to the left subtree since 4 is
less than 5, then compare 4 with 3, and finally find
4 in the left subtree of 3.
COMPARISON TREES CAN BE USED FOR
VARIOUS PURPOSES, INCLUDING:
• Searching: Binary search trees are particularly efficient for searching operations.
• Sorting: Techniques like binary tree sort can be implemented using comparison
trees.
• Decision Making: Decision trees are a specific type of comparison tree used for
decision-making processes in machine learning and artificial intelligence.
• AVL Tree , Splay Tree( comparison example) leni, Balanced m-way tree
(comparison example leni ) ,Red black tree
• Overall, comparison trees are versatile data structures that are widely used in
computer science and data analysis due to their efficiency and simplicity.
DECISION MAKING

• In data structures, a "decision tree" typically refers to a tree-like

structure used for decision-making or classification purposes. It's not to
be confused with the "decision tree" used in algorithm analysis, which
was mentioned earlier in the context of lower bound theory.
• Decision Making is a FULL BINARY TREE that shows the comparisons
between elements that are executed by appropriate sorting algorithm
operating on an input of given size.
DECISION TREES
COMPARE 3 ELEMENTS
• 1. Comparison trees:
• In a comparison sort, we use only comparisons between elements to gain order information about an
input sequence (a1; a2......an).
• Given ai,aj from (a1, a2.....an)We Perform One of the Comparisons

• ai < a j less than

• ai ≤ a j less than or equal to

• ai > a j greater than

• ai ≥ a j greater than or equal to

• ai = a j equal to

• To determine their relative order, if we assume all elements are distinct, then we just need to consider
ai ≤ aj '=' is excluded &, ≥,≤,>,< are equivalent.

• Consider sorting three numbers a1, a2, and a3. There are 3! = 6 possible combinations:

1. (a1, a2, a3), (a1, a3, a2),

2. (a2, a1, a3), (a2, a3, a1)
3. (a3, a1, a2), (a3, a2, a1)
• The Comparison based algorithm defines a decision tree.
DECISION TREE:

• Decision Tree: A decision tree is a full binary tree that shows the
comparisons between elements that are executed by an appropriate
sorting algorithm operating on an input of a given size. Control, data
movement, and all other conditions of the algorithm are ignored.
COMPARISON TREE FOR BINARY SEARCH:

Example: Suppose we have a list of items according to the following Position:

What is a Full Binary Tree?

• A full binary tree is a binary tree with either zero or two child nodes for
each node.
TECHNIQUES FOR ALGEBRAIC PROBLEM
FOR MORE DETAIL

• Lower bound theory and introduction to parallel algorithm by Prof.

Abhigyan (youtube.com)

• DAA Lower bound Theory - javatpoint

ORACLE AND
ADVERSARY ARGUMENT

UNIT-5
• In summary, oracles and adversary arguments are complementary
techniques used in the analysis of algorithms. Oracles help establish
lower bounds by assuming the existence of efficient solutions, while
adversary arguments are used to derive upper bounds by considering
the worst-case behavior against a strategically chosen opponent.
Together, they provide insights into the fundamental limits and
capabilities of algorithms.
https://www.youtube.com/watch?v=hz3-YUNf3Y4(basic starting es toh
thora jha dekhna bes )
Javatpooint
Chatgpt
This topic in Ppt has done from above links
ORACLE AND ADVERSORY EXAMPLE:
(MERGING PROBLEM)
• Example: (Merging Problem) given the sets A (1: m) and B (1: n),
where the information in A and in B are sorted. Consider lower bounds
for algorithms combining these two sets to give an individual sorted
set.
• Consider that all of the m+n elements are specific and A (1) < A (2)
< ....< A (m) and B (1) < B (2) < ....< B (n).
SOLUTION ACC TO ORACLE & ADVISORY APPROACH

• Let's break down the problem of merging two sorted sets A and B into a
single sorted set using an oracle and adversary argument. We'll
consider both upper and lower bounds on the number of comparisons
required by any comparison-based merging algorithm.
• Given two sorted sets A and B, each with m and n elements
respectively, the goal is to merge them into a single sorted set.
ORACLE APPROACH (LOWER BOUND):

• Assume we have an oracle that instantly tells us the correct position to insert an
element into the merged set, given its value and the current state of the merged set.
• With this oracle, we can guarantee that we'll need at leastm+n−1 comparisons to
merge the two sets. This is because each element from A or B needs to be compared
at least once with another element from the other set.
• For example, if A has m elements and B has n elements, then there are m+n−1
positions where elements from both sets will be compared.
• This lower bound of m+n−1 comparisons is achievable only with the help of the
oracle, as it assumes we have perfect knowledge about where each element should
be placed.
EXAMPLE

LETS A={10 ,20 ,30 ,100 } , B={5 ,8, 80, 90 }

// Merge elements from A and B into mergedArray until one of the arrays is exhausted
while (i < m && j < n)
{ if (A[i] <= B[j])
{ merged[k] = A[i]; i++; }
else { merged[k] = B[j]; j++; }
k++;
}
#include <iostream>

void mergeSortedArrays(int A[], int m, int B[], int n, int merged[]) {

int i = 0, j = 0, k = 0;

// Merge elements from A and B into mergedArray until one of the arrays is exhausted
while (i < m && j < n) {
if (A[i] <= B[j]) {
merged[k] = A[i];
i++;
} else {
merged[k] = B[j];
j++;
}
k++;
}

// Append remaining elements from A, if any

while (i < m) {
merged[k] = A[i];
i++;
k++;
}

// Append remaining elements from B, if any

while (j < n) {
merged[k] = B[j];
j++;
k++;
}
}

int main() {
// Example arrays A and B (sorted)
int A[] = {1, 3, 5, 7, 9};
int m = sizeof(A) / sizeof(A[0]);
int B[] = {2, 4, 6, 8, 10};
int n = sizeof(B) / sizeof(B[0]);

// Size of merged array

int mergedSize = m + n;
int merged[mergedSize];

// Merge the arrays

mergeSortedArrays(A, m, B, n, merged);

// Print the merged array

std::cout << "Merged Array: ";
for (int i = 0; i < mergedSize; i++) {
std::cout << merged[i] << " ";
}
std::cout << std::endl;

return 0;
}
ADVERSARY ARGUMENT (UPPER
BOUND):
• Now, let's consider an adversary that strategically selects elements from sets A and B to
maximize the number of comparisons required by any merging algorithm.
• The adversary's goal is to force the merging algorithm to make as many comparisons as
possible.
• One strategy the adversary might use is to alternate between selecting elements from sets A
and B in such a way that the merging algorithm is forced to compare every element with every
other element, leading to m×n comparisons.
• However, with careful consideration, it can be shown that the upper bound on the number of
comparisons is m×n.
• This upper bound of m×n comparisons is achieved in the worst-case scenario, where every
element in A is compared with every element in B.
SUMMARY /NOTE

In summary, the lower bound provided by the oracle tells us that at least
m+n−1 comparisons are required, assuming perfect knowledge. On the
other hand, the upper bound derived from the adversary argument tells
us that in the worst-case scenario, m×n comparisons may be needed.
These bounds help us understand the range of comparisons required by
any comparison-based merging algorithm, considering both ideal and
worst-case scenarios.
LOWER BOUNDS
THROUGH REDUCTIONS
UNIT-5
LOWER BOUNDS THROUGH REDUCTIONS

In the context of Design and Analysis of Algorithms (DAA), lower bounds

through reductions involve proving that a certain problem is at least as
hard as another problem for which a lower bound is known. This
technique is commonly used to establish lower bounds on the time
complexity of algorithms.
HERE'S HOW LOWER BOUNDS THROUGH
REDUCTIONS WORK:
1.Definition: Lower bounds through reductions involve showing that
solving Problem A requires at least as much time (or resources) as
solving Problem B, for which a lower bound is already known.
2.Technique: This technique relies on reducing Problem B to Problem A,
meaning that an algorithm that solves Problem A can also be used to
solve Problem B efficiently. If Problem B has a known lower bound, then
this implies that Problem A also has at least the same lower bound.
EXAMPLES:

• If we can show that sorting a list of numbers requires at least

Ω(nlog⁡n)Ω(nlogn) comparisons, then any problem reducible to sorting
(such as finding the median or finding the kth smallest element) will
also have a lower bound of Ω(nlog⁡n)Ω(nlogn).
• Lower bounds on problems like matrix multiplication, graph
connectivity, or searching algorithms can also be established through
reductions from well-known problems with known lower bounds.
CERTAINLY! LET'S CONSIDER ANOTHER EXAMPLE WHERE WE
ESTABLISH A LOWER BOUND ON THE TIME COMPLEXITY OF A
PROBLEM BY REDUCING IT TO THE PROBLEM OF SORTING.
HERE'S HOW WE CAN PROVE THIS USING THE
SAME LOGIC AS BEFORE:
1.Reduction to Sorting:
1. We can reduce the problem of finding the maximum element to sorting by observing that once
the array is sorted in non-decreasing order, the maximum element will be at the last index of
the sorted array.
2. Therefore, if we can prove a lower bound on the number of comparisons required for sorting, it
will imply a lower bound on the number of comparisons required to find the maximum element.

2.Lower Bound on Sorting:

1. From the comparison-based sorting lower bound theorem, we know that any comparison-based
sorting algorithm requires at least Ω(nlogn) comparisons in the worst case.
2. Since finding the maximum element can be reduced to sorting, it follows that any algorithm for
finding the maximum element must also perform at least Ω(nlogn) comparisons in the worst
case.
HERE'S HOW WE CAN PROVE THIS USING THE
SAME LOGIC AS BEFORE:
1.Reduction to Sorting:
1. We can reduce the problem of finding the maximum element to sorting by observing that once
the array is sorted in non-decreasing order, the maximum element will be at the last index of
the sorted array.
2. Therefore, if we can prove a lower bound on the number of comparisons required for sorting, it
will imply a lower bound on the number of comparisons required to find the maximum element.

2.Lower Bound on Sorting:

1. From the comparison-based sorting lower bound theorem, we know that any comparison-based
sorting algorithm requires at least Ω(nlogn) comparisons in the worst case.
2. Since finding the maximum element can be reduced to sorting, it follows that any algorithm for
finding the maximum element must also perform at least Ω(nlogn) comparisons in the worst
case.
EXAMPLE2 : HERE'S HOW WE CAN PROVE
THIS USING THE SAME LOGIC AS BEFORE:
1.Reduction to Sorting:
1. We can reduce the problem of finding the maximum element to sorting by observing that once
the array is sorted in non-decreasing order, the maximum element will be at the last index of
the sorted array.
2. Therefore, if we can prove a lower bound on the number of comparisons required for sorting, it
will imply a lower bound on the number of comparisons required to find the maximum element.

2.Lower Bound on Sorting:

1. From the comparison-based sorting lower bound theorem, we know that any comparison-based
sorting algorithm requires at least Ω(n) comparisons in the worst case.
2. Since finding the maximum element can be reduced to sorting, it follows that any algorithm for
finding the maximum element must also perform at least Ω(n) comparisons in the worst case.
1.Reduction to Sorting (sorting then find largest elemnt ):
1. We can reduce the problem of finding the maximum element to sorting by observing that once the array is sorted in
non-decreasing order, the maximum element will be at the last index of the sorted array.
2. Therefore, if we can prove a lower bound on the number of comparisons required for sorting, it will imply a lower bound
on the number of comparisons required to find the maximum element.

Find largest element in array (First sort elemnts then find largest )

Let us assume : int a[6] = {10 , 5, 20 , 100 ,9 ,21 }

for(i=0;i<sizeofarray ;i++)
{

if(a[i]>a[i+1])
{
temp=a[i]
a[i]=a[i+1] Total Comparisons are : n-1 where n= total no
of elemnts
a[i+1]=temp
}
}
cout<<a[n]
1. Lower Bound on Sorting(directly find largest element without sorting ):
1. From the comparison-based sorting lower bound theorem, we know that any comparison-based sorting algorithm requires at least Ω(n)
comparisons in the worst case.
2. Since finding the maximum element can be reduced to sorting, it follows that any algorithm for finding the maximum element must also
perform at least Ω(n) comparisons in the worst case.

Let us assume : int a[6] = {10 , 5, 20 , 100 ,9 ,21 }

max=10
for(i=1;i<sizeofarray ;i++)
{
if(max<a[i])
{
max=a[i] Total Comparisons are : n-1 where n=
total no of elemnts
}
}
CONCLUSION:

• Therefore, we have established that finding the maximum element in

an unsorted array requires at least Ω(n-1) comparisons in the worst
case, based on the lower bound for sorting.
• This result implies that any problem reducible to finding the maximum
element (such as finding the minimum or the kth smallest element) will
also have a lower bound of Ω(n-1) comparisons in the worst case.
SUMMARY

In summary, by showing that the problem of finding the maximum

element can be reduced to sorting, and using the lower bound for
sorting, we establish a lower bound on the number of comparisons
required for finding the maximum element, as well as for any problem
reducible to it.
• Problem Reduction in Transform and Conquer Technique -
GeeksforGeeks
APPROXIMATE ALGORITHMS / HEURISTIC ALGORITHM

• An approximation algorithm is a type of algorithm used to find an approximate solution to an

optimization problem when finding an exact solution is computationally expensive or infeasible.
• Approximation algorithms in Design and Analysis of Algorithms (DAA) are algorithms that efficiently
find solutions to optimization problems, providing solutions that are close to optimal within a
guaranteed factor. These algorithms are particularly useful for problems where finding the exact
optimal solution is computationally infeasible, especially in cases where the problems are NP-hard
or NP-complete.(example apa kehnde haan k 6.78 ans aya means approximate aya , 6 ja 7 h pura
exact nhi aya )
• An Approximate Algorithm is a way of approach NP-COMPLETENESS for the optimization problem.
This technique does not guarantee the best solution. The goal of an approximation algorithm
is to come as close as possible to the optimum value in a reasonable amount of time which is at the
most polynomial time. Such algorithms are called approximation algorithm or heuristic algorithm.
EXAMPLE’S / VIMP

• For the traveling salesperson problem, the optimization problem is to

find the shortest cycle, and the approximation problem is to find a short
cycle.
• For the vertex cover problem, the optimization problem is to find the
vertex cover with fewest vertices, and the approximation problem is to
find the vertex cover with few vertices.
• Suppose we work on an optimization problem where every solution
carries a cost(size/time ). An Approximate Algorithm returns a legal
solution, but the cost of that legal solution may not be optimal.
HERE ARE SOME KEY/IMP ASPECTS OF
APPROXIMATION ALGORITHMS:
• Greedy Algorithms: Many approximation algorithms use greedy strategies to construct solutions iteratively.
Greedy algorithms make locally optimal choices at each step with the hope that the overall solution will be
close to optimal. While greedy algorithms do not always guarantee the best possible solution, they can be
efficient and provide good approximations for certain problems.
• Examples: Approximation algorithms are used to solve a wide range of optimization problems, including the
traveling salesman problem (TSP), the knapsack problem, vertex cover, set cover, facility location, and more.
• Trade-offs: Approximation algorithms involve trade-offs between solution quality and computational
efficiency. By sacrificing optimality for efficiency, approximation algorithms enable the solution of large-scale
optimization problems that would otherwise be intractable.
• In summary, approximation algorithms are an essential tool in algorithm design and analysis, providing
practical solutions to optimization problems with guaranteed bounds on solution quality. They play a crucial
role in addressing complex computational challenges in various fields, including computer science, operations
research, and engineering.
SUMMARY

In summary, approximation algorithms are an essential tool in algorithm

design and analysis, providing practical solutions to optimization
problems with guaranteed bounds on solution quality. They play a crucial
role in addressing complex computational challenges in various fields,
including computer science, operations research, and
engineering.
PERFORMANCE RATIOS

• Suppose we work on an optimization problem where every solution

carries a cost(size/time ). An Approximate Algorithm returns a legal
solution, but the cost of that legal solution may not be optimal.