CS2004 Design and Analysis of Algorithms

Lecture 3 - Sorting Problem - Lower bounds on the

problem

Dr. V.A.Kandappan

Assistant Professor,
Department of Computer Science,
Shiv Nadar University Chennai.

Lecture Tutorial Practical Credit

3 0 0 3

Kandappan (SNU Chennai) CS2004 DAA January 6, 2025 1 / 14

1 The Sorting Problem
2 Permutation Sort
3 Selection Sort
4 Merge Sort
5 Lower bounds on the sorting problem
6 radix sort

Kandappan (SNU Chennai) CS2004 DAA January 6, 2025 2 / 14

The sorting problem

• Input: an Array (static) A with n elements and each element has a

key k
• Output: an Array (static) B with n elements which is a sorted
permutation of the array A
• SORTED - an array B is said to be sorted if B[i] ≤ B[i + 1] for all
i ∈ {1, 2, · · · , n − 1}.

??
1 For an array with n elements, How many permutations/reorders are possible?
2 What is the time complexity to establish that a given array is sorted?

Kandappan (SNU Chennai) CS2004 DAA January 6, 2025 3 / 14

Check whether an array is sorted!

Algorithm isSorted(A)
Require: An array of numbers: A sizeof(A) is n
Ensure: True if the array is sorted in ascending order, False otherwise.
1: procedure isSorted(A)
2: for i ← 0 to len(A) − 2 do ▷ Iteration costs O (n) comparisons
3: if A[i] > A[i + 1] then
4: return False
5: end if
6: end for
7: return True
8: end procedure

Kandappan (SNU Chennai) CS2004 DAA January 6, 2025 4 / 14

Permutation Sort - A naive approach

P1 : 9 7 1

P2 : 1 9 7

P3 : 9 1 7

P4 : 7 9 1

P5 : 1 7 9

P6 : 7 1 9

Kandappan (SNU Chennai) CS2004 DAA January 6, 2025 5 / 14

Permutation Sort - A naive approach

P1 : 9 7 1

P2 : 1 9 7

P3 : 9 1 7

P4 : 7 9 1

P5 : 1 7 9

P6 : 7 1 9

Kandappan (SNU Chennai) CS2004 DAA January 6, 2025 6 / 14

Permutation Sort - Version 1

Algorithm 1. permutationSort(allPermutations)
Require: A list of all permutations: allPermutations
Ensure: The sorted list if found
1: procedure permutationSort(allPermutations)
2: for perm in allPermutations do
3: if isSorted(perm) then
4: return perm ▷ Return the first sorted permutation found
5: end if
6: end for
7: return None ▷ No sorted permutation found
8: end procedure

Kandappan (SNU Chennai) CS2004 DAA January 6, 2025 7 / 14

Algorithm 2. find_permutations(A)
Require: An array: A
Ensure: A list of all permutations of the input array
1: procedure find_permutations(A)
2: if len(A) ≤ 1 then
3: return [A]
4: end if
5: perms ← []
6: for i in range(len(A)) do
7: remaining ← A[: i] + A[i + 1 :]
8: for perm in find_permutations(remaining ) do
9: perms.append([A[i]] + perm)
10: end for
11: end for
12: return perms
13: end procedure

The computational Complexity of the above algorithm is O (n!) and the computational
complexity of naive permutation sort is O (n × n!), where n is the size of the array.

Kandappan (SNU Chennai) CS2004 DAA January 6, 2025 8 / 14

Selection Sort Example
• Initial List: [64, 34, 25, 12, 22, 11, 90]
• Step 1:
• Find minimum: 11
• Swap with first element: [11, 34, 25, 12, 22, 64, 90]
• Step 2:
• Find minimum in remaining unsorted portion: 12
• Swap with second element: [11, 12, 25, 34, 22, 64, 90]
• (Remaining Steps):

??
Given a list of n element how many steps does it take to sort the elements
in the list.
• Sorted List: 11, 12, 22, 25, 34, 64, 90
Computational Complexity
The number of comparisons performed in this algorithm is O n2 .

Kandappan (SNU Chennai) CS2004 DAA January 6, 2025 9 / 14

Merge Sort - Better Solution to the problem

3, 1, 2

3 1, 2

1 2

1, 2

1, 2, 3
Kandappan (SNU Chennai) CS2004 DAA January 6, 2025 10 / 14
Decision Tree
Consider c1 , c2 , c3 be the elements of an array that are unique. Then the
decision tree model for any sorting algorithm that uses only comparisons
looks like below.

?c1 < c2

?c1 < c3 ?c2 < c3

?c2 < c3 (c1 , c2 , c3 ) ?c1 < c3 (c3 , c1 , c2 )

(c3 , c2 , c1 ) (c2 , c3 , c1 ) (c3 , c1 , c2 ) (c1 , c3 , c2 )

Kandappan (SNU Chennai) CS2004 DAA January 6, 2025 11 / 14

Lower bounds on the Sorting Problem

1 The decision tree for the Sorting problem is a binary tree.

2 The tree can have at the most n! leaves. The leaves are at different
levels.
3 Assuming that all leaves are at level L,

L = log2 (n!)

Stirling’s Approximation
√  n n
n! ≈ 2πn
e

Kandappan (SNU Chennai) CS2004 DAA January 6, 2025 12 / 14

1 Step 1 Find the number of digits of the maximum element in the array.
Maximum element is 802 and it has 3 digits.
2 Step 2 Sort by Units Digit:

Buckets Values
0 170,90
2 802,2
4 24
5 45,75
6 66

3 Step 3 Sort by Tens Digit:

Buckets Values
0 802,2
2 24
4 45
6 66
7 170,75
9 90

4 Step 4 Sort by Hundreds Digit:

Kandappan (SNU Chennai) CS2004 DAA January 6, 2025 13 / 14

Excercise

1 To find a maximum/minimum element in the array, the number of

comparisons required is Θ (n)
2 To find an element in a sorted array, the lower bound on the number
of comparisons is Ω (log2 (n))

Kandappan (SNU Chennai) CS2004 DAA January 6, 2025 14 / 14