Lecture10 Sorting
Lecture10 Sorting
Algorithms
"
Sorting!
Outline"
• Merge Sort!
• Quick Sort!
• Sorting Lower Bound!
• Bucket-Sort!
• Radix Sort!
7 2 ⏐ 9 4 → 2 4 7 9
7 ⏐ 2 → 2 7 9 ⏐ 4 → 4 9
7 2 ⏐ 9 4 → 2 4 7 9
7 ⏐ 2 → 2 7 9 ⏐ 4 → 4 9
7 2 9 4 → 2 4 7 9 3 8 6 1 → 1 3 8 6
7 2 → 2 7 9 4 → 4 9 3 8 → 3 8 6 1 → 1 6
7 2 ⏐ 9 4 → 2 4 7 9 3 8 6 1 → 1 3 8 6
7 2 → 2 7 9 4 → 4 9 3 8 → 3 8 6 1 → 1 6
7 2 ⏐ 9 4 → 2 4 7 9 3 8 6 1 → 1 3 8 6
7 ⏐ 2 → 2 7 9 4 → 4 9 3 8 → 3 8 6 1 → 1 6
7 2 ⏐ 9 4 → 2 4 7 9 3 8 6 1 → 1 3 8 6
7 ⏐ 2 → 2 7 9 4 → 4 9 3 8 → 3 8 6 1 → 1 6
7 2 ⏐ 9 4 → 2 4 7 9 3 8 6 1 → 1 3 8 6
7 ⏐ 2 → 2 7 9 4 → 4 9 3 8 → 3 8 6 1 → 1 6
7 2 ⏐ 9 4 → 2 4 7 9 3 8 6 1 → 1 3 8 6
7 ⏐ 2 → 2 7 9 4 → 4 9 3 8 → 3 8 6 1 → 1 6
7 2 ⏐ 9 4 → 2 4 7 9 3 8 6 1 → 1 3 8 6
7 ⏐ 2 → 2 7 9 4 → 4 9 3 8 → 3 8 6 1 → 1 6
7 2 ⏐ 9 4 → 2 4 7 9 3 8 6 1 → 1 3 8 6
7 ⏐ 2 → 2 7 9 4 → 4 9 3 8 → 3 8 6 1 → 1 6
7 2 ⏐ 9 4 → 2 4 7 9 3 8 6 1 → 1 3 6 8
7 ⏐ 2 → 2 7 9 4 → 4 9 3 8 → 3 8 6 1 → 1 6
7 2 ⏐ 9 4 → 2 4 7 9 3 8 6 1 → 1 3 6 8
7 ⏐ 2 → 2 7 9 4 → 4 9 3 8 → 3 8 6 1 → 1 6
1 2 n/2
i 2i n/2i
… … …
Phạm Bảo Sơn DSA 18
Summary of Sorting Algorithms"
Algorithm Time Notes
! slow
selection-sort O(n2) ! in-place
! for small data sets (< 1K)
! slow
insertion-sort O(n2) ! in-place
! for small data sets (< 1K)
! fast
heap-sort O(n log n) ! in-place
! for large data sets (1K — 1M)
! fast
merge-sort O(n log n) ! sequential data access
! for huge data sets (> 1M)
Phạm Bảo Sơn DSA 19
Nonrecursive Merge-Sort"
public static void mergeSort(Object[] orig, Comparator c) { //
nonrecursive!
Object[] in = new Object[orig.length]; // make a new temporary array!
System.arraycopy(orig,0,in,0,in.length); // copy the input!
Object[] out = new Object[in.length]; // output array!
Object[] temp; // temp array reference used for swapping!
int n = in.length;!
for (int i=1; i < n; i*=2) { // each iteration sorts all length-2*i runs !
for (int j=0; j < n; j+=2*i) // each iteration merges two length-i pairs!
merge(in,out,c,j,i); // merge from in to out two length-i runs at j!
temp = in; in = out; out = temp; // swap arrays for next iteration!
}!
// the "in" array contains the sorted array, so re-copy it!
System.arraycopy(in,0,orig,0,in.length);!
}!
!
7 4 9 6 2 → 2 4 6 7 9
4 2 → 2 4 7 9 → 7 9
2→2 9→9
Quick-Sort"
• Quick-sort is a
randomized sorting x
algorithm based on the
divide-and-conquer
paradigm:!
– Divide: pick a random x
element x (called pivot) and
partition S into !
• L elements less than x
L E G
• E elements equal x!
• G elements greater than x!
– Recur: sort L and G! x
– Conquer: join L, E and G
Phạm Bảo Sơn DSA 23
Partition"
• We partition an input Algorithm partition(S, p)
sequence as follows:! Input sequence S, position p of pivot
– We remove, in turn, each Output subsequences L, E, G of the
element y from S and ! elements of S less than, equal to,
or greater than the pivot, resp.
– We insert y into L, E or G, L, E, G ← empty sequences
depending on the result of
x ← S.remove(p)
the comparison with the
pivot x while ¬S.isEmpty()
y ← S.remove(S.first())
• Each insertion and removal if y < x
is at the beginning or at the L.insertLast(y)
end of a sequence, and else if y = x
hence takes O(1) time! E.insertLast(y)
• Thus, the partition step of else { y > x }
quick-sort takes O(n) time! G.insertLast(y)
return L, E, G
Phạm Bảo Sơn DSA 24
Quick-Sort Tree"
• An execution of quick-sort is depicted by a binary tree!
– Each node represents a recursive call of quick-sort and stores!
• Unsorted sequence before the execution and its pivot!
• Sorted sequence at the end of the execution!
– The root is the initial call !
– The leaves are calls on subsequences of size 0 or 1!
7 4 9 6 2 → 2 4 6 7 9
4 2 → 2 4 7 9 → 7 9
2→2 9→9
Phạm Bảo Sơn DSA 25
Execution Example"
• Pivot selection!
7 2 9 43 7 6 1 → 1 2 3 4 6 7 8 9
7 2 9 4 → 2 4 7 9 3 8 6 1 → 1 3 8 6
9→9 4→4
Phạm Bảo Sơn DSA 26
Execution Example (cont.)"
• Partition, recursive call, pivot selection!
7 2 9 4 3 7 6 1→ 1 2 3 4 6 7 8 9
2 4 3 1→ 2 4 7 9 3 8 6 1 → 1 3 8 6
9→9 4→4
Phạm Bảo Sơn DSA 27
Execution Example (cont.)"
• Partition, recursive call, base case!
7 2 9 43 7 6 1→→ 1 2 3 4 6 7 8 9
2 4 3 1 →→ 2 4 7 3 8 6 1 → 1 3 8 6
9→9 4→4
Phạm Bảo Sơn DSA 28
Execution Example (cont.)"
• Recursive call, …, base case, join!
7 2 9 43 7 6 1→ 1 2 3 4 6 7 8 9
2 4 3 1 → 1 2 3 4 3 8 6 1 → 1 3 8 6
9→9 4→4
Phạm Bảo Sơn DSA 29
Execution Example (cont.)"
• Recursive call, pivot selection!
7 2 9 43 7 6 1→ 1 2 3 4 6 7 8 9
2 4 3 1 → 1 2 3 4 7 9 7 1 → 1 3 8 6
9→9 4→4
Phạm Bảo Sơn DSA 30
Execution Example (cont.)"
• Partition, …, recursive call, base case!
7 2 9 43 7 6 1→ 1 2 3 4 6 7 8 9
2 4 3 1 → 1 2 3 4 7 9 7 1 → 1 3 8 6
9→9 4→4
Phạm Bảo Sơn DSA 31
Execution Example (cont.)"
• Join, join!
7 2 9 4 3 7 6 1 →1 2 3 4 6 7 7 9
2 4 3 1 → 1 2 3 4 7 9 7 → 17 7 9
9→9 4→4
Phạm Bảo Sơn DSA 32
Worst-case Running Time"
• The worst case for quick-sort occurs when the pivot is the unique
minimum or maximum element!
• One of L and G has size n - 1 and the other has size 0
• The running time is proportional to the sum!
n + (n - 1) + … + 2 + 1!
• Thus, the worst-case running time of quick-sort is O(n2)
depth time
0 n
1 n-1
… …
n-1 1
Phạm Bảo Sơn DSA 33
Expected Running Time"
• Consider a recursive call of quick-sort on a sequence of size s
– Good call: the sizes of L and G are each less than 3s/4
– Bad call: one of L and G has size greater than 3s/4
7 2 9 43 7 6 19 7 2 9 43 7 6 1
2 4 3 1 7 9 7 1 → 1 1 7294376
no
Is xi < xj?
yes
Phạm Bảo Sơn DSA 41
Counting Comparisons"
• Let us just count comparisons then.!
• Each possible run of the algorithm
corresponds to a root-to-leaf path in a
decision tree! xi < xj ?
xa < xb ? xc < xd ?
xa < xb ? xc < xd ?
log (n!)
xe < xf ? xk < xl ? xm < xo ? xp < xq ?
1, c 3, a 3, b 7, d 7, g 7, e
B ∅ ∅ ∅ ∅ ∅ ∅ ∅
0 1 2 3 4 5 6 7 8 9
Bucket-Sort"
• Let be S be a sequence of n Algorithm bucketSort(S, N)
(key, element) entries with keys Input sequence S of (key, element)
in the range [0, N - 1] items with keys in the range
• Bucket-sort uses the keys as [0, N - 1]
indices into an auxiliary array B Output sequence S sorted by
of sequences (buckets)! increasing keys
Phase 1: Empty sequence S by B ← array of N empty sequences
moving each entry (k, o) into its while ¬S.isEmpty()
bucket B[k] f ← S.first()
Phase 2: For i = 0, …, N - 1, move (k, o) ← S.remove(f)
the entries of bucket B[i] to the B[k].insertLast((k, o))
end of sequence S!
for i ← 0 to N - 1
• Analysis:! while ¬B[i].isEmpty()
– Phase 1 takes O(n) time! f ← B[i].first()
– Phase 2 takes O(n + N) time! (k, o) ← B[i].remove(f)
!Bucket-sort takes O(n + N) time! S.insertLast((k, o))
Phạm Bảo Sơn DSA 46
Example"
• Key range [0, 9]
7, d 1, c 3, a 7, g 3, b 7, e
Phase 1
1, c 3, a 3, b 7, d 7, g 7, e
B ∅ ∅ ∅ ∅ ∅ ∅ ∅
0 1 2 3 4 5 6 7 8 9
Phase 2
1, c 3, a 3, b 7, d 7, g 7, e
Phạm Bảo Sơn DSA 47
Properties and Extensions"
• Key-type Property! Extensions!
– The keys are used as – Integer keys in the range [a, b]!
indices into an array • Put entry (k, o) into bucket
and cannot be arbitrary B[k - a] !
objects! – String keys from a set D of
possible strings, where D has
– No external comparator! constant size (e.g., names of
• Stable Sort Property! the 50 U.S. states)!
• Sort D and compute the rank
– The relative order of r(k) of each string k of D in the
any two items with the sorted sequence !
same key is preserved • Put entry (k, o) into bucket
after the execution of B[r(k)]!
the algorithm!