Comparison of function

•Many of the properties of real number apply to asymptotic comparison.

Assume that f(n) and g(n) are asymptotically positive, then following
exists
•TRANSITIVITY
f(n)= Θ(g(n)) and g(n)=Θ( h(n)) imply f(n) = Θ (h(n))
f(n)= Ω(g(n)) and g(n)= Ω( h(n)) imply f(n) = Ω (h(n))
f(n)= O(g(n)) and g(n)=O ( h(n)) imply f(n) =O (h(n))

•REFLEXIVITY
f(n)= Θ(f(n))
f(n)= Ω(f(n))
f(n)= O(f(n))

Friday, June 28, 2024 Searching and Sorting 1

 Comparison of function
•SYMMETRY
f(n)= Θ(g(n)) iff g(n)= Θ(f(n))

TRANSPOSE SYMMETRY
f(n)= O(g(n)) iff g(n)= O(f(n))

•Because the property hold for asymptotic notation, one can draw analogy
between asymptotic comparison of two function f and g and then
comparison of two real number a and b
f(n)= Θ(g(n)) a=b
f(n)= Ω(g(n)) a≥b
f(n)= O(g(n)) a≤b

Friday, June 28, 2024 Searching and Sorting 2

 Comparison of function
•FLOOR and CEILING
• A function f(n) is monotonically increasing if m ≤ n implies f(m) ≤ f(n)
• A function f(n) is monotonically decreasing if m ≥ n implies f(m) ≥ f(n)
•For any real number x we denote the greatest integer less than or equal
to x by |x| (floor) or |x| (ceiling)
•Any integer n
•Floor((n/2) + (n/2)) =n

•Refer Cormen Pg.51- 56

Friday, June 28, 2024 Searching and Sorting 3

 Analysis of insertion sort
INSERTION_SORT(A) COST TIME
1. For j <-- 2 to length [A] c1 n
2. do key <-- A[j] c2 n-1
3. insert A[j] in to sorted sequence 0 n-1
4. i <-- j-1 c4 n-1
 t
n
5. While i>0 and A[i] > key c5 j 2
j


n
(tj  1)
6. do A [i+1] <-- A[i] c6 j 2


n
(tj  1)
7. I <-- i-1 c7
j 2

8. A [i+1] <-- key c8 n-1

• The runninc5g time of algorithm is sum of running time for each

statement executed. To compute T(n) the running time of this sort we
sum the product of cost and time

Friday, June 28, 2024 Searching and Sorting 4

 Analysis of insertion sort…..
• Even for inputs of given size an algorithm running time may depend
upon which input of that size is given.

•The best case occur if the array is already sorted. In that case
T(n)= c1(n)+c2(n-1)+ c4(n-1)+c5(n-1)+c8(n-1)
= (c1+c2+c4+c5+c8)n – (c2+c4+c5+c8)
= an+b

The worst case running time is expressed as an2+bn+c

Friday, June 28, 2024 Searching and Sorting 5

 Question on quick sort
•Show how Quick sort can be made to run in O(nlogn) time in worst case

•Solution:
•Quick sort is based on Divide and conquer method. Three steps are
followed for sorting an subarray A [p,……..r]

•DIVIDE: Partition the array A[p,………….r] into two subarrays

A[p……q-1] & A[q+1,……r] such that each element of A[p……q-1] ≤ A[q]
A[p……q-1] ≤ A[q] ≤ A[q+1,……r]

•CONQUER: Sort the two subarrays A[p……q-1] & A[q+1,……r] by

recursive call to quick sort.

•COMBINE: since subarrays are sorted in place we don’t need to combine

them. The entire array is sorted now.

Friday, June 28, 2024 Searching and Sorting 6

 Solution for quick sort………
•QUICKSORT(A,p,r)
1. If p<r
2. then q PARTITION(A,p,r)
3. QUICKSORT(A,p,q-1)
4. QUICKSORT(A,q+1,r)

To sort an entire array the initial call is QUICKSORT(A , l, length [A] )

Partitioning the array
PARTITION (A,p,r)
5. x A[r]
6. i p-1
7. For j  p to r-1
8. do if A[j] ≤ x
9. then i i+1
10. Exchange A[i] > A[j]
11. end for
12. Exchange A[i+1]  A[r]
13. return i+1`
Friday, June 28, 2024 Searching and Sorting 7
 Solution for quick sort………
• PARTITION always select an element x=A[r] as pivot element around
which partition of array is done.

2 8 7 1 3 5 6 4

I p,j X=4

In first step 2 is exchange with 2. Now A[j]=8 not less than 4 so do not exchange

2 8 7 1 3 5 6 4

2 1 7 8 3 5 6 4

2 1 3 8 7 5 6 4

2 1 3 8 7 5 6 4

2 1 3 4 7 5 6 8

Friday, June 28, 2024 Searching and Sorting 8

 Performance of quick sort
•The running time of quick sort depend upon the whether the
partition is balanced, unbalanced and this in turn
depends upon the pivot element that is chosen.

•If the partition is balanced then algorithm runs

asymptotically as fast as merge sort

•if the partition is unbalanced then the algorithm runs

asymptotically as slow as insertion sort

Friday, June 28, 2024 Searching and Sorting 9

 Worst case partitioning
•The worst case will occur when the partitioning routine
produce one sub problem with n -1 elements and one with 0
elements.
•Let us assume that worst case partitioning arises in each
recursive call.

•The partitioning cost Θ(n) time.

•The recursive call on array of size 0 returns T(0)= Θ(1), the
recurrence for the running time is
•T(n)= T(n-1) + T(0) + Θ(n) = T(n-1) + Θ(n)
•If sum of the cost incurred at each level we get arithmetic
series.
•If partition is maximally unbalanced at every recursive level
running time is Θ(n2)
Friday, June 28, 2024 Searching and Sorting 10
 best case partitioning
•The best case occur when PARTITION produce two sub
problems, each of size not more than n/2.

•So the size of one of them will be [n/2] and [n/2 -1].
•The recurrence for the running time will be

• T(n) ≤ 2T(n/2) + Θ(n)

•Using Master theorem T(n) = O(n lg n)

•Thus, the equal partitioning of two sides of partition at each

level of the recursion produce an asymptotically faster
algorithm.
Friday, June 28, 2024 Searching and Sorting 11
 balanced case partitioning
•The average case running of quick sort is closer to the worst
case. The partition algo produce split which seems to be
unbalance .

•The obtain recurrence is as follows

•T(n) ≤ T (9n/10) +T(n/10) +cn

•At every level cost of tree is cn represented in nxt slide

•So the total cost of quick sort is 0(n lg n)

Friday, June 28, 2024 Searching and Sorting 12

 Tree structure
•The average case running of quick sort is closer to the worst
case. The partition algo produce split which seems to be
unbalance .

•The obtain recurrence is as follows

•T(n) ≤ T (9n/10) +T(n/10) +cn

•At every level cost of tree is cn represented in nxt slide

•So the total cost of quick sort is 0(n lg n)

Friday, June 28, 2024 Searching and Sorting 13

 Kruskal’s Algorithm
Kruskal()
{
T = ;
for each v  V
MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
Friday, June 28, 2024 Searching and Sorting 14
 Kruskal’s Algorithm
Run the algorithm:
Kruskal() 2 19
{ 9
14 17
T = ; 8 25
5
for each v  V
21 13 1
MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
Friday, June 28, 2024 Searching and Sorting 15
 Kruskal’s Algorithm
Run the algorithm:
Kruskal() 2 19
{ 9
14 17
T = ; 8 25
5
for each v  V
21 13 1
MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
Friday, June 28, 2024 Searching and Sorting 16
 Kruskal’s Algorithm
Run the algorithm:
Kruskal() 2 19
{ 9
14 17
T = ; 8 25
5
for each v  V
21 13 1
MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
Friday, June 28, 2024 Searching and Sorting 17
 Kruskal’s Algorithm
Run the algorithm:
Kruskal() 2 19
{ 9
14 17
T = ; 8 25
5
for each v  V
21 13 1?
MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
Friday, June 28, 2024 Searching and Sorting 18
 Kruskal’s Algorithm
Run the algorithm:
Kruskal() 2 19
{ 9
14 17
T = ; 8 25
5
for each v  V
21 13 1
MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
Friday, June 28, 2024 Searching and Sorting 19
 Kruskal’s Algorithm
Run the algorithm:
Kruskal() 2? 19
{ 9
14 17
T = ; 8 25
5
for each v  V
21 13 1
MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
Friday, June 28, 2024 Searching and Sorting 20
 Kruskal’s Algorithm
Run the algorithm:
Kruskal() 2 19
{ 9
14 17
T = ; 8 25
5
for each v  V
21 13 1
MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
Friday, June 28, 2024 Searching and Sorting 21
 Kruskal’s Algorithm
Run the algorithm:
Kruskal() 2 19
{ 9
14 17
T = ; 8 25
5?
for each v  V
21 13 1
MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
Friday, June 28, 2024 Searching and Sorting 22
 Kruskal’s Algorithm
Run the algorithm:
Kruskal() 2 19
{ 9
14 17
T = ; 8 25
5
for each v  V
21 13 1
MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
Friday, June 28, 2024 Searching and Sorting 23
 Kruskal’s Algorithm
Run the algorithm:
Kruskal() 2 19
{ 9
14 17
T = ; 8? 25
5
for each v  V
21 13 1
MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
Friday, June 28, 2024 Searching and Sorting 24
 Kruskal’s Algorithm
Run the algorithm:
Kruskal() 2 19
{ 9
14 17
T = ; 8 25
5
for each v  V
21 13 1
MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
Searching and Sorting 25
 Kruskal’s Algorithm
Run the algorithm:
Kruskal() 2 19
{ 9?
14 17
T = ; 8 25
5
for each v  V
21 13 1
MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
Friday, June 28, 2024 Searching and Sorting 26
 Kruskal’s Algorithm
Run the algorithm:
Kruskal() 2 19
{ 9
14 17
T = ; 8 25
5
for each v  V
21 13 1
MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
Friday, June 28, 2024 Searching and Sorting 27
 Kruskal’s Algorithm
Run the algorithm:
Kruskal() 2 19
{ 9
14 17
T = ; 8 25
5
for each v  V
21 13? 1
MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
Friday, June 28, 2024 Searching and Sorting 28
 Kruskal’s Algorithm
Run the algorithm:
Kruskal() 2 19
{ 9
14 17
T = ; 8 25
5
for each v  V
21 13 1
MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
Friday, June 28, 2024 Searching and Sorting 29
 Kruskal’s Algorithm
Run the algorithm:
Kruskal() 2 19
{ 9
14? 17
T = ; 8 25
5
for each v  V
21 13 1
MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
Friday, June 28, 2024 Searching and Sorting 30
 Kruskal’s Algorithm
Run the algorithm:
Kruskal() 2 19
{ 9
14 17
T = ; 8 25
5
for each v  V
21 13 1
MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
Friday, June 28, 2024 Searching and Sorting 31
 Kruskal’s Algorithm
Run the algorithm:
Kruskal() 2 19
{ 9
14 17?
T = ; 8 25
5
for each v  V
21 13 1
MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
Friday, June 28, 2024 Searching and Sorting 32
 Kruskal’s Algorithm
Run the algorithm:
Kruskal() 2 19?
{ 9
14 17
T = ; 8 25
5
for each v  V
21 13 1
MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
Friday, June 28, 2024 Searching and Sorting 33
 Kruskal’s Algorithm
Run the algorithm:
Kruskal() 2 19
{ 9
14 17
T = ; 8 25
5
for each v  V
21? 13 1
MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
Friday, June 28, 2024 Searching and Sorting 34
 Kruskal’s Algorithm
Run the algorithm:
Kruskal() 2 19
{ 9
14 17
T = ; 8 25?
5
for each v  V
21 13 1
MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
Friday, June 28, 2024 Searching and Sorting 35
 Kruskal’s Algorithm
Run the algorithm:
Kruskal() 2 19
{ 9
14 17
T = ; 8 25
5
for each v  V
21 13 1
MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
Friday, June 28, 2024 Searching and Sorting 36
 Kruskal’s Algorithm
Run the algorithm:
Kruskal() 2 19
{ 9
14 17
T = ; 8 25
5
for each v  V
21 13 1
MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
Friday, June 28, 2024 Searching and Sorting 37
 Correctness Of Kruskal’s Algorithm
• Sketch of a proof that this algorithm produces an
MST for T:
– Assume algorithm is wrong: result is not an MST
– Then algorithm adds a wrong edge at some point
– If it adds a wrong edge, there must be a lower weight
edge (cut and paste argument)
– But algorithm chooses lowest weight edge at each
step. Contradiction
• Again, important to be comfortable with cut and
paste arguments
Friday, June 28, 2024 Searching and Sorting 38
 Kruskal’s Algorithm
Kruskal() What will affect the running time?
{
T = ;
for each v  V
MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
Friday, June 28, 2024 Searching and Sorting 39
 Kruskal’s Algorithm
Kruskal() What will affect the running time?
1 Sort
{
O(V) MakeSet() calls
T = ; O(E) FindSet() calls
for each v  V O(V) Union() calls
MakeSet(v); (Exactly how many Union()s?)
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
Friday, June 28, 2024 Searching and Sorting 40
 Kruskal’s Algorithm: Running Time
• To summarize:
– Sort edges: O(E lg E)
– O(V) MakeSet()’s
– O(E) FindSet()’s
– O(V) Union()’s
• Upshot:
– Best disjoint-set union algorithm makes above 3
operations take O(E(E,V)),  almost constant
– Overall thus O(E lg E), almost linear w/o sorting
Friday, June 28, 2024 Searching and Sorting 41