Elementary Sorting Methods

Bùi Tiến Lên

2024
Contents

1. Selection Sort
2. Insertion Sort
3. Bubble Sort
4. Shell Sort
5. Heap Sort
6. Merge Sort
7. Quick Sort
8. Key-indexed Counting
9. Radix Sort
10. Workshop
 Introduction
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort Concept 1

Merge Sort Sorting is a fundamental algorithm design problem. Many efficient algorithms use
Quick Sort
sorting as a subroutine, because it is often easier to process data if the elements
Key-indexed
Counting are in a sorted order.
Radix Sort

Workshop
• The basic problem in sorting is as follows: Given an array that contains n
elements (keys), our task is to sort the elements in increasing order. For
example, the array
1 3 8 2 9 2 5 6

will be as follows after sorting:

1 2 2 3 5 6 8 9

3
 Introduction (cont.)
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort Concept 2 nghch th

Merge Sort An inversion is a pair of keys that are out of order in the array
Quick Sort

Key-indexed
Counting
• For instance, an array {E, X, A, M, P, L, E} has 11 inversions:
Radix Sort E-A, X-A, X-M, X-P, X-L, X-E, M-L, M-E, P-L, P-E, and L-E.
Workshop
Concept 3
If the number of inversions in an array is less than a constant multiple of the array
size, we say that the array is partially sorted.

Mng có n ã sp

4
 Introduction (cont.)
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort
Concept 4
Merge Sort A sorting method is stable if it preserves the relative order of equal keys in the
Quick Sort array
Key-indexed
Counting

Radix Sort

Workshop

5
 Introduction (cont.)
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort Concept 5

Merge Sort An adaptive sorting algorithm is a type of sorting algorithm that takes
Quick Sort
advantage of existing order or structure in the input data to improve its
Key-indexed
Counting performance.
Radix Sort

Workshop Concept 6
A non-adaptive sorting algorithm is a type of sorting algorithm that does not
take advantage of any existing order or structure in the input data.

6
 Sorting Algorithms
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort
O(n2 ) algorithms
Heap Sort

Merge Sort
• Selection Sort
Quick Sort • Insertion Sort
Key-indexed • Bubble Sort
Counting

Radix Sort O(nk ) algorithms

Workshop • Shell Sort
O(n log n) algorithms
• Heap Sort
• Merge Sort
• Quick Sort
O(n) algorithms
• Radix Sort
• Counting Sort
7
 Visualizing Sorting Algorithms
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort

Merge Sort

Quick Sort

Key-indexed
Counting

Radix Sort

Workshop

8
Selection Sort
 Selection Sort
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort Idea

Merge Sort Assume that the array a is composed of two parts: the left part s is sorted and
Quick Sort the right part u is unsorted
Key-indexed
Counting 1. s = ∅ and u = a
Radix Sort
2. Find the smallest element x of the part u
Workshop
3. Remove x from u
4. Append x to s
Repeat the actions 2-4 until u is empty

10
 Implementation
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort template <class Item >

Merge Sort void selection (Item a[], int l, int r) {
Quick Sort for (int i = l; i < r; i++) {
Key-indexed
int min = i;
Counting for (int j = i + 1; j <= r; j++)
Radix Sort if (a[j] < a[min ]) min = j;
Workshop swap(a[i], a[min ]);
}
}

11
 Illustration
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort
• Trace (right before the call to swap())
Merge Sort a[]
Quick Sort
i min 0 1 2 3 4 5 6 7 8 9 10
Key-indexed
Counting S O R T E X A M P L E
Radix Sort 0 6 S O R T E X A M P L E
1 4 A O R T E X S M P L E
Workshop
2 10 A E R T O X S M P L E
3 9 A E E T O X S M P L R
4 7 A E E L O X S M P T R
5 7 A E E L M X S O P T R
6 8 A E E L M O S X P T R
7 10 A E E L M O P X S T R
8 8 A E E L M O P R S T X
9 9 A E E L M O P R S T X
10 10 A E E L M O P R S T X
A E E L M O P R S T X

12
 Analysis
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort Theorem 1

Merge Sort
Selection sort uses ∼ N 2 /2 compares and N exchanges to sort an array of length
Quick Sort
N.
Key-indexed

Analysis of selection sort for the input size of N (the number of keys)
Counting

Radix Sort

Workshop
• Time complexity:

best case ?
average case ?
worst case ?

• Space complexity: O(1)

• Stability: No
không duy trì th t trc và sau khi xp, không th dùng trong
excel
13
Insertion Sort
 Insertion Sort
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort Idea

Merge Sort Assume that the array a is composed of two parts: the left part s is sorted and
Quick Sort the right part u is unsorted
Key-indexed
Counting 1. s = ∅ and u = a
Radix Sort
2. Remove the first elements x of u
Workshop
3. Insert x into its proper place among s
Repeat the actions 2-3 until u is empty

15
 Implementation
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort
• Algorithm 1
Merge Sort
template <class Item >
Quick Sort
void insertion (Item a[], int l, int r) {
Key-indexed
Counting Item v;
Radix Sort
for (int i = l + 1; i <= r; i++) {
Workshop
v = a[i];
for (int j = i; j > l && v < a[j - 1]; j--)
a[j] = a[j - 1];
a[j] = v;
}
}

16
 Implementation (cont.)
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort
• Algorithm 2 (using sentinel technique)
Merge Sort
template <class Item >
Quick Sort
void insertion (Item a[], int l, int r) {
Key-indexed
Counting int i;
Radix Sort
for (i = r; i > l; i--) compare_swap (a[i - 1], a[i]);
Workshop
for (i = l + 2; i <= r; i++) {
v = a[i];
for (int j = i; v < a[j - 1]; j--)
a[j] = a[j - 1];
a[j] = v;
}
}

17
 Illustration
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort
• Trace (right after the inner loop is exhausted)
Merge Sort a[]
Quick Sort
i j 0 1 2 3 4 5 6 7 8 9 10
Key-indexed
Counting S O R T E X A M P L E
Radix Sort 1 0 O S R T E X A M P L E
2 1 O R S T E X A M P L E
Workshop
3 3 O R S T E X A M P L E
4 0 E O R S T X A M P L E
5 5 E O R S T X A M P L E
6 0 A E O R S T X M P L E
7 2 A E M O R S T X P L E
8 4 A E M O P R S T X L E
9 2 A E L M O P R S T X E
10 2 A E E L M O P R S T X
A E E L M O P R S T X

18
 Analysis
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort Theorem 2

Merge Sort The number of exchanges used by insertion sort is equal to the number of
Quick Sort inversions in the array, and the number of compares is at least equal to the
Key-indexed
Counting
number of inversions and at most equal to the number of inversions plus the array
Radix Sort size minus 1.
Workshop
Theorem 3
Insertion sort uses ∼ N 2 /4 compares and ∼ N 2 /4 exchanges to sort a randomly
ordered array of length N with distinct keys, on the average.

s ln dch chuyn có mi quan h bng vs

19
 Analysis (cont.)
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort
• Time complexity:
Merge Sort
best case ?
Quick Sort

Key-indexed
average case ?
Counting worst case ?
Radix Sort

Workshop • Space complexity: O(1)

• Stability: Yes

20
Bubble Sort
 Bubble Sort
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort
• Keep passing through the array, exchanging adjacent elements that are out of
Heap Sort

Merge Sort
order, continuing until the array is sorted.
Quick Sort For example, in the array
Key-indexed 1 3 8 2 9 2 5 6
Counting

Radix Sort the first round of bubble sort swaps elements as follows:
Workshop 1 3 2 8 9 2 5 6

1 3 2 8 2 9 5 6

1 3 2 8 2 5 9 6

1 3 2 8 2 5 6 9

• Bubble Sort is a kind of Selection Sort.

22
 Bubble Sort
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort template <class Item >

Merge Sort void bubble (Item a[], int l, int r) {
Quick Sort for (int i = l; i < r; i++)
Key-indexed
for (int j = r; j > i; j--)
Counting compare_swap (a[j - 1], a[j]);
Radix Sort }
Workshop

• Challenge: reimplement the function bubble using recursion technique

23
 Analysis
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort
• Time complexity:
Merge Sort
best case ?
Quick Sort

Key-indexed
average case ?
Counting worst case ?
Radix Sort

Workshop • Space complexity: O(1)

• Stability: Yes

24
Shell Sort
 Shell Sort
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort
• Insertion sort is slow because the only exchanges it does involve adjacent
Heap Sort

Merge Sort
items, so items can move through the array only one place at a time.
Quick Sort
• Shellsort is a simple extension of insertion sort that gains speed by allowing
Key-indexed exchanges of elements that are far apart.
Counting
• The running time is better than O(n2 )
Radix Sort

Workshop Concept 7
An h-sorted array is h independent sorted subsequences, interleaved together.

26
 Shell Sort
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort Idea

Merge Sort • Given the decrement sequence {h1 , h2 , ..., ht } where hi ∈ N and ht = 1
Quick Sort
• For each h ∈ {h1 , h2 , ..., ht }
Key-indexed
Counting 1. Split an array a into h subsequences
Radix Sort

Workshop a0 , a0+h , a0+2h , ...

a1 , a1+h , a1+2h , ...
a2 , a2+h , a2+2h , ...
...

2. Using Insertion Sort to sort each subsequence

27
 Increment/Decrement Sequence
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort
• Shell proposed
Heap Sort
N
h1 =
Merge Sort

Quick Sort 2
Key-indexed hi
Counting hi+1 = i >1 (1)
Radix Sort
2
Workshop
• Hibbard proposed
hi = 2 i − 1 (2)
• Knuth proposed
h1 = 1
hi+1 = 3hi + 1 i >1 (3)
• Pratt proposed
Successive numbers of the form 2p 3q , p, q ∈ N (4)
28
 Implementation
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort template <class Item >

Merge Sort void shellsort (Item a[], int l, int r) {
Quick Sort int h;
Key-indexed
for (h = 1; h <= (r - l) / 9; h = 3 * h + 1);
Counting for (; h > 0; h /= 3)
Radix Sort for (int i = l + h; i <= r; i++) {
Workshop int j = i;
Item v = a[i];
while (j >= l + h && v < a[j - h]) {
a[j] = a[j - h];
j -= h;
}
a[j] = v;
}
}

29
 Illustration
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort input S H E L L S O R T E X A M P L E

13-sort P H E L L S O R T E X A M S L E
Heap Sort P H E L L S O R T E X A M S L E
P H E L L S O R T E X A M S L E
Merge Sort 4-sort L H E L P S O R T E X A M S L E
L H E L P S O R T E X A M S L E
Quick Sort L H E L P S O R T E X A M S L E
L H E L P S O R T E X A M S L E
Key-indexed L H E L P S O R T E X A M S L E
Counting L E E L P H O R T S X A M S L E
L E E L P H O R T S X A M S L E
Radix Sort L E E A P H O L T S X R M S L E
L E E A M H O L P S X R T S L E
Workshop L E E A M H O L P S X R T S L E
L E E A M H L L P S O R T S X E
L E E A M H L E P S O L T S X R
1-sort E L E A M H L E P S O L T S X R
E E L A M H L E P S O L T S X R
A E E L M H L E P S O L T S X R
A E E L M H L E P S O L T S X R
A E E H L M L E P S O L T S X R
A E E H L L M E P S O L T S X R
A E E E H L L M P S O L T S X R
A E E E H L L M P S O L T S X R
A E E E H L L M P S O L T S X R
A E E E H L L M O P S L T S X R
A E E E H L L L M O P S T S X R
A E E E H L L L M O P S T S X R
A E E E H L L L M O P S S T X R
A E E E H L L L M O P S S T X R
A E E E H L L L M O P R S S T X

result A E E E H L L L M O P R S S T X 30
 Analysis
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort Theorem 4

Merge Sort The result of h-sorting an array that is k-ordered is an array that is both h- and
Quick Sort k-ordered
Key-indexed
Counting

Radix Sort
Theorem 5
Workshop Shellsort does less than N(h − 1)(k − 1)/g comparisons to g-sort an array that is
h- and k-ordered, provided that h and k are relatively prime

31
 Analysis (cont.)
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort
• Time complexity:
Merge Sort
best case ?
Quick Sort

Key-indexed
average case ?
Counting worst case ?
Radix Sort

Workshop • Space complexity: O(1)

• Stability: No

32
Heap Sort
 Heap Sort
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort Concept 8

Merge Sort

Quick Sort • Max heap: A tree is heap-ordered if the key in each node is larger than or
Key-indexed equal to the keys in all of that node’s children (if any)
Counting

Radix Sort
• Min heap: A tree is heap-ordered if the key in each node is smaller than or
Workshop equal to the keys in all of that node’s children (if any)

Theorem 6

• Max heap: No node in a heap-ordered tree has a key larger than the key at
the root
• Min heap: No node in a heap-ordered tree has a key smaller than the key at
the root

34
 Heap Representation
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort
• Array representation of a heap-ordered complete binary tree
Merge Sort

Quick Sort
a0
Key-indexed
Counting
a1 a2
Radix Sort

Workshop
a3 a4 a5 a6

a7 a8
Figure 1: Array {a0 , a1 , a2 , a3 , a4 , a5 , a6 , a7 , a8 } and complete binary tree

35
 Heap Representation (cont.)
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort
Note
Heap Sort
Array vs. Complete binary tree
Merge Sort

Quick Sort • Root node is a0

Key-indexed • Parent(ai ) is aj i−1 k or nothing
Counting
2
Radix Sort
• LeftChild(ai ) is a2i+1 or nothing
Workshop
• RightChild(ai ) is a2i+2 or nothing

36
 Top-down heapify
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort
At the given node ai
Heap Sort

Merge Sort
• Exchange the key in the given node ai with the largest key among that
Quick Sort node’s children a2i+1 and a2i+2
Key-indexed • Move down to that child, and continuing down the tree until we reach the
Counting

Radix Sort
bottom or a point where no child has a larger key.
Workshop

37
 Implementation
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort template <class Item >

Merge Sort void heapify (Item a[], int n, int i) {
Quick Sort Item v = a[i];
Key-indexed
while (i < n / 2) {
Counting int child = 2 * i + 1;
Radix Sort if (child < n - 1)
Workshop if (a[child] > a[child + 1])
child ++;
if (v >= a[child ]) break ;
a[i] = a[child ];
i = child;
}
a[i] = v;
}

38
 Heap Sort
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort
• Build max-heap array: use heapify operation to convert an array a to a
Merge Sort max-heap array
• All elements in the range a n , ..., an−1 are leaf nodes.
n o
Quick Sort
2
Key-indexed
• Apply heapify operation for these elements a n −1 , ..., a0
n o
Counting
2
Radix Sort

Workshop
• Sort a max-heap array a
1. Swap the first and the last element
2. Remove the last element
If |a| > 1 then apply heapify operation for a0 and repeat actions 1-2

39
 Heap Sort (cont.)
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort template <class Item >

Merge Sort void heapsort (Item a[], int l, int r) {
Quick Sort Item *pa = a + l;
Key-indexed int N = r - l + 1;
Counting
for (int k = N / 2 - 1; k >= 0; k--)
Radix Sort
heapify (pa , N, k);
Workshop
while (N > 1) {
swap(pa[0], pa[N - 1]);
N--;
heapify (pa , N, 0);
}
}

40
 Analysis
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort Theorem 7

Merge Sort Heapsort uses fewer than 2N log2 N comparisons to sort N elements
Quick Sort

Key-indexed
Counting
• Time complexity:

best case ?
Radix Sort

Workshop
average case ?
worst case ?

• Space complexity:
• Stability:

41
Merge Sort
 Top-Down Merge Sort
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort Idea

Merge Sort Merge sort sorts a subarray a[l . . . r ] as follows:
Quick Sort
1. If l ≥ r , do not do anything, because the subarray is already sorted or empty.
Key-indexed
Counting 2. Calculate the position of the middle element: m = b(l + r )/2c.
Radix Sort
3. Recursively sort the subarray a[l . . . m].
Workshop
4. Recursively sort the subarray a[m + 1 . . . r ].
5. Merge the sorted subarrays a[l . . . m] and a[m + 1 . . . r ] into a sorted subarray
a[l . . . r ].

43
 Illustration
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort
Sorting the following array:
Heap Sort

Merge Sort
1 3 6 2 8 2 5 9
Quick Sort

Key-indexed
Counting
• The array will be divided into two subarrays as follows:
Radix Sort

Workshop 1 3 6 2 8 2 5 9
• Then, the subarrays will be sorted recursively as follows:
1 2 3 6 2 5 8 9
• Finally, the algorithm merges the sorted subarrays and creates the final sorted
array:
1 2 2 3 5 6 8 9
44
 Implementation
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort template <class Item >

Merge Sort void mergesort (Item a[], Item aux [], int l, int r) {
Quick Sort if (r <= l) return ;
Key-indexed
int m = (l + r) / 2;
Counting mergesort (a, aux , l, m);
Radix Sort mergesort (a, aux , m + 1, r);
Workshop merge(a, aux , l, m, r);
}

45
 Implementation (cont.)
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort template <class Item >

Merge Sort void merge(Item a[], Item aux [], int l, int m, int r) {
Quick Sort int i, j, k;
Key-indexed for (k = l; k <= r; k++)
Counting
aux[k] = a[k];
Radix Sort
i = l; j = m + 1; k = l;
Workshop
while (i <= m && j <= r)
if (aux[i] <= aux[j]) a[k++] = aux[i++];
else a[k++] = aux[j++];
while (i <= m)
a[k++] = aux[i++];
while (j <= r)
a[k++] = aux[j++];
}

46
 Bottom-up Merge Sort
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort Idea

Merge Sort Bottom-up merge sort consists of
Quick Sort
• A sequence of passes over the whole array doing sz-by-sz merges
Key-indexed
Counting • Doubling sz on each pass.
Radix Sort
• The final subarray is of size sz only if the array size is an even multiple of sz,
Workshop
so the final merge is an sz-by-x merge, for some x less than or equal to sz.

Tit kim c na chi phí

47
 Implementation
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort template <class Item >

Merge Sort void mergesort (Item a[], Item aux [], int l, int r) {
Quick Sort for (int sz = 1; sz <= r - l; sz = sz + sz)
Key-indexed
for (int i = l; i <= r - sz; i += sz + sz)
Counting merge(a, aux , i, i + sz - 1, min(i + sz + sz - 1, r));
Radix Sort }
Workshop

48
 Analysis
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort Theorem 8

Merge Sort Mergesort requires about N log2 N comparisons to sort any array of N elements
Quick Sort

Key-indexed
Counting
• Time complexity:
Radix Sort
best case ?
Workshop
average case ?
worst case ?

• Space complexity:
• Stability:

49
Quick Sort
 Quick Sort
Selection Sort

Insertion Sort

Bubble Sort
1. chn 1 phn t pivot
Shell Sort
2. phân hoch a ra thành 2 phn a left và a right
Heap Sort Idea
Merge Sort Quicksort invented by C. A. R. Hoare in 1960 is a divide-and-conquer method for
Quick Sort sorting
Key-indexed
Counting 1. Partition an array a into two parts a left and a right such that ∀x ∈ a left and
Radix Sort ∀y ∈ a right then x ≤ y
Workshop
2. Sort the parts a left and a right independently
3. Join a left and a right to a = a left a right

51
 Implementation
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort template <class Item >

Merge Sort void quicksort (Item a[], int l, int r) {
Quick Sort if (r <= l) return ; if (r <= l + 10)
Key-indexed
int i = partition (a, l, r); {
Counting quicksort (a, l, i - 1); insertionsort(a, l, r);
Radix Sort quicksort (a, i + 1, r); return;
Workshop } }

Chia
Nhn xét: khi mng có kích thcvà
nh tr
thut toán insertion chy hiu qu hn
(vì k có quy)

52
 Implementation (cont.)
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort template <class Item >

Merge Sort int partition (Item a[], int l, int r) {
Quick Sort int i = l - 1, j = r; Item v = a[r];
Key-indexed for (;;) {
Counting
while (a[++i] < v);
Radix Sort
while (v < a[--j]) if (j == l) break ;
Workshop
if (i >= j) break ;
swap(a[i], a[j]);
}
swap(a[i], a[r]);
return i;
}

53
 Analysis
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort Theorem 9

Merge Sort Quicksort uses ∼ 2N log2 N compares (and one-sixth that many exchanges) on
Quick Sort
the average to sort an array of length N with distinct keys.
Key-indexed
Counting

Radix Sort Theorem 10

Workshop
Quicksort uses ∼ N 2 /2 compares in the worst case

54
 Analysis (cont.)
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort
• Time complexity:
Merge Sort
best case ?
Quick Sort

Key-indexed
average case ?
Counting worst case ?
Radix Sort

Workshop • Space complexity: O(1)?

• Stability: No

55
 A lower bound for the worst case
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort
Theorem 11
Merge Sort No compare-based sorting algorithm can guarantee to sort N items with fewer
Quick Sort than log2 (N!) ∼ N log2 N compares.
Key-indexed
Counting
Proof
Radix Sort
Comparison sorts can be viewed abstractly in terms of decision trees.
Workshop
0:1

1:2 0:2

012 0:2 102 1:2

021 201 120 210

56
 Comparing Sorting Algorithms
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort
• Performance characteristics of sorting algorithms
Merge Sort
algorithm stable? in-place? running time space
Quick Sort

Key-indexed
selection no yes N2 1
Counting insertion yes yes between N and N 2 1
Radix Sort shell no yes N log N?, N 6/5 ? 1
Workshop
merge yes no N log N N
quick no yes N log N log N
3-way quick no yes between N and N log N log N
heap no yes N log N 1

57
Key-indexed Counting
 Assumptions about keys
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort
Assumption. Keys are integers between 0 and R − 1.
Heap Sort

Merge Sort
Implication. Can use key as an array index.
Quick Sort

Key-indexed
Counting

Radix Sort

Workshop

59
 Key-indexed Counting
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort
Problem. Sort an array a of N items whose keys are integers between 0 and
Merge Sort R − 1.
Quick Sort
// Compute frequency counts .
Key-indexed
Counting for (int i = 0; i < N; i++)
Radix Sort count[a[i]. key () + 1]++;
Workshop // Transform counts to indices .
for (int r = 0; r < R; r++)
count[r + 1] += count[r];
// Distribute the items .
for (int i = 0; i < N; i++)
aux[count[a[i]. key () ]++] = a[i];
// Copy back.
for (int i = 0; i < N; i++)
a[i] = aux[i];

60
 Illustration
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort
• Input an array a of 12 letters
Merge Sort

Quick Sort

Key-indexed
Counting

Radix Sort

Workshop

61
 Illustration (cont.)
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort
• Compute frequency counts
Merge Sort

Quick Sort

Key-indexed
Counting

Radix Sort

Workshop

62
 Illustration (cont.)
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort
• Transform counts to indices.
Merge Sort

Quick Sort

Key-indexed
Counting

Radix Sort

Workshop

63
 Illustration (cont.)
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort
• Distribute the data
Merge Sort

Quick Sort

Key-indexed
Counting

Radix Sort

Workshop

64
 Illustration (cont.)
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort
• Copy back
Merge Sort

Quick Sort

Key-indexed
Counting

Radix Sort

Workshop

65
 Analysis
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort Theorem 12

Merge Sort Key-indexed counting uses 8N + 3R + 1 array accesses to stably sort N items
Quick Sort whose keys are integers between 0 and R − 1.
Key-indexed
Counting

Radix Sort
• Time complexity:
Workshop
best case ?
average case ?
worst case ?

• Space complexity:
• Stability:

66
Radix Sort
 Radix Sort
Selection Sort

Insertion Sort

Bubble Sort
s nh phân 2
Shell Sort s thp phân 10
Heap Sort Concept 9 s thp lc phân 16
Merge Sort

Quick Sort • A byte is a fixed-length sequence of bits.

Key-indexed • A word is a fixed-length sequence of bytes.
Counting

Radix Sort • A string is a variable-length sequence of bytes.

Workshop

68
 Radix Sort (cont.)
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort Idea

Merge Sort Radix-sorting algorithms treat the keys as numbers represented in a base-R
Quick Sort number system, for various values of R (the radix), and work with individual
Key-indexed
Counting
digits of the numbers.
Radix Sort

Workshop Concept 10
A key is a radix-R number, with digits numbered from the right (starting at 0)

• In programing, we use the abstract digit operation to access digits of keys

69
 Radix Sort (cont.)
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort
There are two basic approaches to radix sorting:
Heap Sort

Merge Sort
• The first class of methods: They examine the digits in the keys in a
Quick Sort left-to-right order, working with the most significant digits first. These
Key-indexed methods are generally referred to as most-significant-digit (MSD) radix
Counting
sorts or top-down methods.
Radix Sort

Workshop
• The second class of methods: They examine the digits in the keys in a
right-to-left order, working with the least significant digits first. These
methods are generally referred to as least-significant-digit (LSD) radix sorts
or bottom-up methods.

70
 LSD Implementation
Selection Sort
sp xp các ch s t hàng n v tr i
Insertion Sort

Bubble Sort

Shell Sort

Heap Sort template <class Item >

Merge Sort void radixsort (Item a[], int l, int r) {
Quick Sort vector <Item > bins[radix ];
Key-indexed
for (int d = 0; d < max_digit ; d++) {
Counting // clear bins
Radix Sort ...
Workshop // distribute
for (i = l; i <= r; i++)
bins[digit(a[i],d,radix)]. push_back (a[i]);
// join bins to a[l...r]
...
}
}

71
 Example 1
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort
Problem Sort an array of integer numbers {170, 45, 75, 90, 802, 2,
Merge Sort 24, 66}
Quick Sort
• Rewrite the numbers in 3-digit format {170, 045, 075, 090, 802, 002,
Key-indexed
Counting 024, 066}
Radix Sort

Workshop

72
 Example 1 (cont.)
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort
• Distribute the data into 10 bins on digit d = 0
Heap Sort

Merge Sort {170, 045, 075, 090, 802, 002, 024, 066}
Quick Sort bin 0: 170 090
Key-indexed
Counting
bin 1:
Radix Sort
bin 2: 802 002
Workshop bin 3:
bin 4: 024
bin 5: 045 075
bin 6: 066
bin 7:
bin 8:
bin 9:
• Join the bins
{170, 090, 802, 002, 024, 045, 075, 066}
73
 Example 1 (cont.)
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort
• Distribute the data into 10 bins on digit d = 1
Heap Sort

Merge Sort {170, 090, 802, 002, 024, 045, 075, 066}
Quick Sort bin 0: 802, 002
Key-indexed
Counting
bin 1:
Radix Sort
bin 2: 024
Workshop bin 3:
bin 4: 045
bin 5:
bin 6: 066
bin 7: 170, 075
bin 8:
bin 9: 090
• Join the bins
{802, 002, 024, 045, 066, 170, 075, 090}
74
 Example 1 (cont.)
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort
• Distribute the data into 10 bins on digit d = 2
Heap Sort

Merge Sort {802, 002, 024, 045, 066, 170, 075, 090}
Quick Sort bin 0: 002, 024, 045, 066, 075, 090
Key-indexed
Counting
bin 1: 170
Radix Sort
bin 2:
Workshop bin 3:
bin 4:
bin 5:
bin 6:
bin 7:
bin 8: 802
bin 9:
• Join the bins
{002, 024, 045, 066, 075, 090, 170, 802}
75
 Example 1 (cont.)
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort 170 170 802 002

Merge Sort 045 090 002 024
Quick Sort 075 802 024 045
Key-indexed
Counting
090 −→ 002 −→ 045 −→ 066
Radix Sort
802 024 066 075
Workshop 002 045 170 090
024 075 075 170
066 066 090 802

76
 Example 2
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort
Problem Sort 17 integer numbers
Merge Sort
{6, 7, 1, 3, 5, 2, 0, 4, 2, 1, 7, 2, 1, 3, 5, 2, 7}
Quick Sort

Key-indexed
Counting • Rewrite the numbers in 3-digit binary numbers
Radix Sort
110 111 001 011 101 010 000 100 010 001 111 010 001 011 101 010 111
Workshop

77
 Example 2 (cont.)
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort 110 111 001 011 101 010 000 100 010 001 111 010 001 011 101 010 111

Merge Sort

Quick Sort ↓
Key-indexed
Counting 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000

Radix Sort

Workshop
↓

000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000

000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000

78
 Analysis
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort
• Time complexity:
Merge Sort
best case ?
Quick Sort

Key-indexed
average case ?
Counting worst case ?
Radix Sort

Workshop • Space complexity:

• Stability:

79
Workshop
 � Quiz
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort
1. What is a sorting operation?
Merge Sort .........................................................................
Quick Sort .........................................................................
Key-indexed .........................................................................
Counting

Radix Sort
2. What is an inversion?
Workshop .........................................................................
.........................................................................
.........................................................................
3. How selection sort sorts the sample array E A S Y Q U E S T I O N?
.........................................................................
.........................................................................
.........................................................................

81
 Ï Exercises
Selection Sort

Insertion Sort

Bubble Sort

Shell Sort

Heap Sort
• Programming exercises in [Cormen, 2009, Sedgewick, 2002]
Merge Sort

Quick Sort

Key-indexed
Counting

Radix Sort

Workshop

82
References

Cormen, T. H. (2009).
Introduction to algorithms.
MIT press.
Sedgewick, R. (2002).
Algorithms in Java, Parts 1-4, volume 1.
Addison-Wesley Professional.
Walls and Mirrors (2014).
Data Abstraction And Problem Solving with C++.
Pearson.