Scribd
Upload
18 views24 pages

L6 - Linear Time Sort

The document discusses different sorting algorithms and their time complexities. It proves that any comparison-based sorting algorithm requires at least O(n log n) time due to the structure of decision trees used in sorting. It also discusses counting sort and radix sort, which can achieve linear time by exploiting additional information about input values.

Uploaded by

imran.khan03
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
18 views24 pages

L6 - Linear Time Sort

The document discusses different sorting algorithms and their time complexities. It proves that any comparison-based sorting algorithm requires at least O(n log n) time due to the structure of decision trees used in sorting. It also discusses counting sort and radix sort, which can achieve linear time by exploiting additional information about input values.

Uploaded by

imran.khan03
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 24

Lecture 06

Linear Time Sorting


CSE373: Design and Analysis of Algorithms
Sorting Algorithms: Time Complexity
• For all the sorting algorithms we have seen so far the
worst-case running time is at least O(n log n).
Sorting Algorithm Worst Case Time Complexity
Insertion Sort O(n2)
Merge Sort O(n lg n)
Quick Sort (randomized/not) O(n2)
Heap Sort O(n lg n)

• Can we design a sorting algorithm whose worst-case


time complexity is less than O(n lg n)?
Formal Question
• Sorting can be viewed as the process of determining the
permutation that restores the order of input numbers.
Input: <7,3,8> => Output: <3,7,8>
• We will consider only comparison sorts, algorithms that
sort numbers based only on comparisons between input
elements.
• The sorting methods we have seen so far are all
comparison sorts.

• We ask: what is the minimum number of comparisons


must a comparison sort execute to sort an array of n
elements?
To answer this question, we need to
learn about decision tree first.
Decision Tree for Comparison Sorts
• We think of two outcomes in a comparison of two
numbers a and b: a ≤ b and a > b.
• For which pair a comparison is done depends only on the
outcomes of the comparisons that have been made so far.
• So, for each n, the action of a comparison sort on an n-
element array can be viewed as a binary tree such that
• each node corresponds to a comparison and
• each leaf corresponds to the permutation that the algorithm
outputs.
• We call such a tree binary decision tree
Decision Tree for Comparison Sorts
A binary decision tree for applying a comparison sort
algorithm upon a 3-element array: A[1..3]

Can you draw another decision tree for sorting


a 3-element array?
Lower bound on Comparison Sorts
• Recall our question: what is the minimum number of
comparisons must a comparison sort execute to sort an
array of n elements?
• Theorem: No comparison sort has the worst-case
running time less than O(n lg n).
Proof:
Let’s consider an n-element numeric array: A[1..n] and draw
the binary decision tree that results from applying a
comparison sort algorithm upon it.
Lower bound on Comparison Sorts
The minimum no. of comparisons (in the worst-case) done
by this comparison-sort algorithm
= the length of the longest path from root to a leaf node in
the corresponding decision tree
= height of the decision tree = h (let)
Lower bound on Comparison Sorts
Max. no. of leaves in a binary tree with height h, is 2 h.
But out binary decision tree contains exactly n! leaves.
n! ≤ 2h
Þ lg(n!) ≤ h
Þ h ≥ lg(n!) = lg (n(n-1)(n-2) …. (n-(n/2-1)) …. 3.2.1)
Þ h ≥ lg (n(n-1)(n-2) …. (n-(n/2-1))
n/2 numbers, each of them ≥ n/2

Þ h ≥ lg ( (n/2) (n/2) (n/2)… (n/2))


= lg ( (n/2)n/2 )
= (n/2) lg (n/2)
Þ h ≥ O(n lg n)
Lower bound on Comparison Sorts
So The minimum no. of comparisons (in the worst-case)
done by this comparison-sort algorithm = h ≥ O(n lg n)
So the worst-case time-complexity of any comparison-sort
is at least O(n lg n) and as such it’s not possible to sort an n-
element array in less than O(n lg n) time.
Simulation of Counting Sort
A

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
B

A
d
B
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Simulation of Counting Sort

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Simulation of Counting Sort

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Time Complexity of Counting Sort

//O(k)
//O(n)

Total time complexity of counting


sort, T(n) = O(n+k), where
//O(k)
n = no. of elements in input array and
k = largest element in input array
//O(n)
So if k is O(n2) then T(n) = O(n2)
i.e. counting sort is linear only when
the largest element in input array is
linear in order
//O(n)
.17 .12
.26 .21 .23

.78 .72

Radix Sort Simulation (before sorting the buckets/linked-lists)


Radix Sort Simulation (after sorting the buckets/linked-lists)

You might also like