Algorithms

Sorting (Ch. 7)
Morgan Ericsson

Algorithms/ Sorting 1
 Today
» Sorting
» Simple sorts: Selection, Insert, Bubble, Shell
» Merge
» Quick
» Specialized
» Radix

Algorithms/ Sorting 2
 Sorting

Algorithms/ Sorting 4
 Preliminaries
» We consider comparison-based sorting
» I.e., Comparable and compareTo in Java
» To keep it simple, we generally assume int
» But we can sort any type that is comparable
» and arrays (Python lists)
» But we obviously sort linked structures

Algorithms/ Sorting 5
 Total order
» A total order is a binary relation ≤ that satisfies
» Antisymmetry: if v ≤ w and w ≤ v , then v = w
» Transitivity: if v ≤ w and w ≤ x , then v ≤ x
» Totality: either v ≤ w or w ≤ v or both
» Standard order for, e.g., natural or real numbers

Algorithms/ Sorting 6
 A sorted list
» Total order holds
» So, if [a,b,c,d] is sorted, …
» … a ≤ b ≤ c ≤ d should hold

Algorithms/ Sorting 7
 Check if sorted
1 def is_sorted(l:list[int]) -> bool:
2 for i in range(1, len(l)):
3 if l[i - 1] > l[i]:
4 return False
5 return True

Algorithms/ Sorting 8
 Testing it
1 import random
2
3 lst = random.sample(range(1, 1_001), k=20)
4
5 assert is_sorted(lst) == False
6 assert is_sorted(sorted(lst)) == True

Algorithms/ Sorting 9
 Some sorting terminology
» In-place: the list is sorted in-place, i.e., it does not
require any additional storage to sort the list
» Stable: Elements with the same value maintains their
relative order

Algorithms/ Sorting 10
 Simple algorithms

Algorithms/ Sorting 12
 Selection sort
» Simple idea: in iteration i, find the index of the
smallest remaining entry
» Swap the element at index i and the smallest value

Algorithms/ Sorting 13
 Selection sort
Iteration 0 : find the smallest element in Iteration 1 : find the smallest element in Iteration 2 : find the smallest element in
[0, 8] and swap with index 0 [1, 8] and swap with index 1 [2, 8] and swap with index 2

min min min

1 4 8 2 5 9 3 7 6 1 4 8 2 5 9 3 7 6 1 2 8 4 5 9 3 7 6
0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8

Iteration 3 : find the smallest element in Iteration 7 : find the smallest element in
[3, 8] and swap with index 3 [7, 8] and swap with index 7

min Iterations 4 to 6 min

1 2 3 4 5 9 8 7 6 1 2 3 4 5 6 7 8 9
0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8

Algorithms/ Sorting 14
 Implementation
1 def selection_sort(l:list[int]) -> None:
2 n = len(l)
3 for i in range(n):
4 mn = i
5 for j in range(i + 1, n):
6 if l[mn] > l[j]:
7 mn = j
8 l[i], l[mn] = l[mn], l[i]

Algorithms/ Sorting 15
 Testing it
1 lst = random.sample(range(1, 1_001), k=20)
2
3 assert is_sorted(lst) == False
4 selection_sort(lst)
5 assert is_sorted(lst) == True

Algorithms/ Sorting 16
 Analysis
» In-place and unstable
» Consider [4, 3, 4 , 1]
′

» (n − 1) + (n − 2) + … + 1 + 0 ∼ n2 / 2 compares and n
swaps
» Insensitive to input, O(n2 ) whether sorted or
completely random
» Minimal data movement

Algorithms/ Sorting 17
 Insert sort
» In iteration i, swap the value at index i with each
larger entry to its left
» So, move the value at index i to the correct place

Algorithms/ Sorting 18
 Insert sort
Iteration 0 : do nothing Iteration 1 : move 4 left while the Iteration 2 : move 8 left while the
elements are larger. In this case, do elements are larger. In this case, do
nothing nothing

1 4 8 2 5 9 3 7 6 1 4 8 2 5 9 3 7 6 1 4 8 2 5 9 3 7 6
0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8

Iteration 3 : move 2 left while the Iteration 4 : move 5 left while the Iteration 5 : move 9 left while the
elements are larger. Swaps 2 and 8, elements are larger. Swaps 5 and 8. elements are larger. In this case, do
and 2 and 4 nothing

1 4 8 2 5 9 8 7 6 1 2 4 8 5 9 8 7 6 1 2 4 5 8 9 3 7 6
0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8

Algorithms/ Sorting 19
 Implementation
1 def insert_sort(l:list[int]) -> None:
2 n = len(l)
3 for i in range(n):
4 for j in range(i, 0, -1):
5 if l[j] < l[j-1]:
6 l[j], l[j - 1] = l[j - 1], l[j]
7 else:
8 break

Algorithms/ Sorting 20
 Testing it
1 lst = random.sample(range(1, 1_001), k=20)
2
3 assert is_sorted(lst) == False
4 insert_sort(lst)
5 assert is_sorted(lst) == True

Algorithms/ Sorting 21
 Analysis
» In-place and stable
» Depends on input
» If sorted, n − 1 compares and 0 exchanges
» If descending order, ∼ 0.5 ⋅ n2 compares and
exchanges
» Average case, same but 0.25
» Still O(n2 ), but runs in linear time if partially sorted

Algorithms/ Sorting 22
 Bubble sort
» Iterate over the list, compare pairs, and swap if left is
smaller than right
» Keep iterating until there are no swaps

Algorithms/ Sorting 23
 Bubble sort
Iteration 0 Iteration 1 Iteration 2

1 4 8 2 5 9 3 7 6 1 4 2 5 8 3 7 6 9 1 2 4 5 3 7 6 8 9
0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8

1 4 8 2 5 9 3 7 6 1 4 2 5 8 3 7 6 9 1 2 4 5 3 7 6 8 9
0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8

1 4 8 2 5 9 3 7 6 1 2 4 5 8 3 7 6 9 1 2 4 5 3 7 6 8 9
0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8

1 4 2 8 5 9 3 7 6 1 2 4 5 8 3 7 6 9 1 2 4 5 3 7 6 8 9
0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8

1 4 2 5 8 9 3 7 6 1 2 4 5 8 3 7 6 9 1 2 4 3 5 7 6 8 9
0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8

1 4 2 5 8 9 3 7 6 1 2 4 5 3 8 7 6 9 1 2 4 3 5 7 6 8 9
0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8

1 4 2 5 8 3 9 7 6 1 2 4 5 3 7 8 6 9
0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8

1 4 2 5 8 3 7 9 6
0 1 2 3 4 5 6 7 8

Algorithms/ Sorting 24
 Implementation
1 def bubble_sort(l:list[int]) -> None:
2 n = len(l)
3 for i in range(n):
4 swp = False
5 for j in range(n - i - 1):
6 if l[j] > l[j + 1]:
7 l[j], l[j + 1] = l[j + 1], l[j]
8 swp = True
9 if not swp:
10 break

Algorithms/ Sorting 25
 Testing it
1 lst = random.sample(range(1, 1_001), k=20)
2
3 assert is_sorted(lst) == False
4 bubble_sort(lst)
5 assert is_sorted(lst) == True

Algorithms/ Sorting 26
 Analysis
» In-place and stable
» Similar to insert sort
» Depends on input, if almost sorted, linear
» So, O(n2 )

Algorithms/ Sorting 27
 Shellsort
» Move elements more than one position at a time
» h-sorting
» if h is 4
» Check lst[h] < lst[h + 4]
» Shellsort
» h-sort the array with decreasing values of h
» 13 sort, 4 sort, 1 sort

Algorithms/ Sorting 28
 Shellsort
» We use insertion sort with stride h
» Big increments, small subarray
» Small increments, nearly in order

Algorithms/ Sorting 29
 Shellsort
4-sort 1-sort

27 54 30 38 89 29 95 57 80 42 99 27 29 30 38 80 42 95 57 89 54 99
0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10

27 54 30 38 89 29 95 57 80 42 99 27 29 30 38 80 42 95 57 89 54 99
0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10

27 29 30 38 89 54 95 57 80 42 99 27 29 30 38 80 42 95 57 89 54 99
0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10

27 29 30 38 89 54 95 57 80 42 99 27 29 30 38 80 42 95 57 89 54 99
0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10

27 29 30 38 89 54 95 57 80 42 99 27 29 30 38 80 42 95 57 89 54 99
0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10

27 29 30 38 80 54 95 57 89 42 99 27 29 30 38 42 80 95 57 89 54 99
0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10

27 29 30 38 80 42 95 57 89 54 99 27 29 30 38 42 80 95 57 89 54 99
0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10

27 29 30 38 42 57 80 95 89 54 99
0 1 2 3 4 5 6 7 8 9 10

27 29 30 38 42 57 80 89 95 54 99
0 1 2 3 4 5 6 7 8 9 10

27 29 30 38 42 54 57 80 89 95 99
0 1 2 3 4 5 6 7 8 9 10

Algorithms/ Sorting 30
 Shellsort
4-sort 1-sort

11 10 9 8 7 6 5 4 3 2 1 3 2 1 4 7 6 5 8 11 10 9
0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10

7 10 9 8 11 6 5 4 3 2 1 2 3 1 4 7 6 5 8 11 10 9
0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10

7 6 9 8 11 10 5 4 3 2 1 1 2 3 4 7 6 5 8 11 10 9
0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10

7 6 5 8 11 10 9 4 3 2 1 1 2 3 4 7 6 5 8 11 10 9
0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10

7
3 6 5 4 11
7 10 9 8 11
3 2 1 1 2 3 4 7 6 5 8 11 10 9
0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10

3 2 5 4 7 6 9 8 11 10 1 1 2 3 4 6 7 5 8 11 10 9
0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10

1 2 3 4 5 6 7 8 11 10 9
0 1 2 3 4 5 6 7 8 9 10

1 2 3 4 5 6 7 8 11 10 9
0 1 2 3 4 5 6 7 8 9 10

1 2 3 4 5 6 7 8 11 10 9
0 1 2 3 4 5 6 7 8 9 10

1 2 3 4 5 6 7 8 10 11 9
0 1 2 3 4 5 6 7 8 9 10

Algorithms/ Sorting 31
 Which sequence of h?
» Any should work, but there are better and worse
» Powers of two is bad (only even until 1)
» 3x − 1 is ok
» Performs reasonably well and is easy to compute
» There are better sequences

Algorithms/ Sorting 32
 Implementation
1 def shellsort(l:list[int]) -> None:
2 h, n = 1, len(l)
3 while h < n // 3:
4 h = 3 * h + 1
5
6 while h >= 1:
7 for i in range(h, n):
8 j = i
9 while j >= h and l[j] < l[j - h]:
10 l[j], l[j - h] = l[j - h], l[j]
11 j -= h
12 h = h // 3

Algorithms/ Sorting 33
 Testing it
1 lst = random.sample(range(1, 1_001), k=20)
2
3 assert is_sorted(lst) == False
4 shellsort(lst)
5 assert is_sorted(lst) == True

Algorithms/ Sorting 34
 Analysis
» Quite difficult, depends on the sequence
» And we do not know enough about it
» Bad sequence, O(n2 )
4
» Good sequence, O(n ) 3

3
» Ours, O(n ) 2

Algorithms/ Sorting 35
 What does this mean?

Algorithms/ Sorting 36
 In practice?

Algorithms/ Sorting 37
 In practice?
bubble : 2.540e − 08 ⋅ x2.089
insert : 2.605e − 08 ⋅ x2.046
selection : 6.772e − 09 ⋅ x2.139

Algorithms/ Sorting 38
 In practice

Algorithms/ Sorting 39
 Mergesort

Algorithms/ Sorting 41
 Mergesort
» Simple idea
» Split the list in half
» (Merge)Sort both halves (recursively)
» Merge the two sorted lists
» Divide and conquer

Algorithms/ Sorting 42
 Merge
» We can merge two sorted lists in O(m + n) , where m
and n are the sizes of the two lists
» Advance pointers in the two lists independently
» Pick the smallest and add to the merged list

Algorithms/ Sorting 43
 Merge
17 31 32 50 65 86 16 31 49 52 55 99 17 31 32 50 65 86 16 31 49 52 55 99 17 31 32 50 65 86 16 31 49 52 55 99

S 0 1 2 3 4 5 6 7 8 9 10 11
3 0 1 2 3 4 5 6 7 8 9 10 11
6 0 1 2 3 4 5 6 7 8 9 10 11

16 17 31 16 17 31 31 32 49
0 1 2 3 4 5 6 7 8 9 10 11 0 1 2 3 4 5 6 7 8 9 10 11 0 1 2 3 4 5 6 7 8 9 10 11

17 31 32 50 65 86 16 31 49 52 55 99 17 31 32 50 65 86 16 31 49 52 55 99 17 31 32 50 65 86 16 31 49 52 55 99

0 0 1 2 3 4 5 6 7 8 9 10 11
4 0 1 2 3 4 5 6 7 8 9 10 11
7 0 1 2 3 4 5 6 7 8 9 10 11

16 16 17 31 31 16 17 31 31 32 49 50
0 1 2 3 4 5 6 7 8 9 10 11 0 1 2 3 4 5 6 7 8 9 10 11 0 1 2 3 4 5 6 7 8 9 10 11

17 31 32 50 65 86 16 31 49 52 55 99 17 31 32 50 65 86 16 31 49 52 55 99 17 31 32 50 65 86 16 31 49 52 55 99

1 0 1 2 3 4 5 6 7 8 9 10 11
5 0 1 2 3 4 5 6 7 8 9 10 11
8 0 1 2 3 4 5 6 7 8 9 10 11

16 17 16 17 31 31 32 16 17 31 31 32 49 50 52
0 1 2 3 4 5 6 7 8 9 10 11 0 1 2 3 4 5 6 7 8 9 10 11 0 1 2 3 4 5 6 7 8 9 10 11

Algorithms/ Sorting 44
 Implementation
1 class MergeSort:
2 def _merge(self, a:list[int], tmp:list[int], \
3 lo:int, mid:int, hi:int) -> None:
4 for k in range(lo, hi+1):
5 tmp[k] = a[k]
6
7 i, j = lo, mid + 1
8 for k in range(lo, hi+1):
9 if i > mid:
10 a[k] = tmp[j]
11 j += 1
12 elif j > hi:
13 a[k] = tmp[i]
14 i += 1
15 elif tmp[j] < tmp[i]:
16 a[k] = tmp[j]
17 j += 1
18 else:
19 a[k] = tmp[i]
20 i += 1

Algorithms/ Sorting 45
 Testing it
1 lst = [17, 31, 32, 50, 65, 86, 16, 31, 49, 52, 55, 99]
2 tmp = [0] * len(lst)
3 ms = MergeSort()
4 ms._merge(lst, tmp, 0, len(lst) // 2 - 1, len(lst) - 1)
5 assert is_sorted(lst) == True

Algorithms/ Sorting 46
 Sorting
» When is a random list sorted?
» When it has 1 (or 0) elements
» Divide lists until they have one element
» Then merge them together in sorted order

Algorithms/ Sorting 47
 Mergesort
29 56 31 62 87 25 52 94

29 56 31 62 87 25 52 94

29 56 31 62 87 25 52 94

29 56 31 62 87 25
25 52
52 94

29 56 31 62 25 87 52 94

29 31 56 62 25 52 87 94

25 29 31 52 56 62 87 94

Algorithms/ Sorting 48
 Implementation
1 from fastcore.basics import patch
2
3 @patch
4 def _sort(self:MergeSort, a:list[int], tmp:list[int], \
5 lo:int, hi:int) -> None:
6 if hi <= lo:
7 return
8
9 mid = lo + (hi - lo) // 2
10 self._sort(a, tmp, lo, mid)
11 self._sort(a, tmp, mid+1, hi)
12 self._merge(a, tmp, lo, mid, hi)
13
14 @patch
15 def sort(self:MergeSort, a:list[int]) -> None:
16 tmp = [0] * len(a)
17 self._sort(a, tmp, 0, len(a) - 1)

Algorithms/ Sorting 49
 Testing it
1 lst = [29, 56, 31, 62, 87, 25, 52, 94]
2 ms = MergeSort()
3 ms.sort(lst)
4 assert is_sorted(lst) == True

Algorithms/ Sorting 50
 Analysis
D(N) N =N

D(N/2) D(N/2) 2(N/2) =N

D(N/4) D(N/4) D(N/4) D(N/4) 4(N/4) =N

D(N/2 k ) 2 k (N/2 k ) =N

D(2) D(2) D(2) D(2) D(2) D(2) D(2) D(2) (N/2)(2) =N

=N log 2 N
(Assuming N is a power of 2)

Algorithms/ Sorting 51
 Analysis
» Not in place, but can be
» Stable
» Almost perfect in terms or comparisons
» O(n log n )

Algorithms/ Sorting 52
 In practice

Algorithms/ Sorting 53
 Quicksort

Algorithms/ Sorting 55
 Quicksort
» Divide and conquer, just like Mergesort
» Split the input into two smaller parts
» But split around a pivot value and ensure that
» Values to the left are not greater than …
» .. and values to the right not less than the pivot
» Avoids the merge step

Algorithms/ Sorting 56
 Quicksort

Find the pivot 50 27 37 53 14 59 67 70 34 80

Not greater Not less

Move elements 27 37 14 34 50 59 67 53 70 80

Sort left (recursively) 14 27 34 37 50 59 67 53 70 80

Sort right (recursively) 14 27 34 37 50 53 59 67 70 80

Algorithms/ Sorting 57
 Implementation
1 class Quicksort:
2 def _partition(self, a:list[int], lo:int, hi:int) -> int:
3 i, j = lo, hi + 1
4
5 while True:
6 i += 1
7 while a[i] < a[lo]:
8 if i == hi: break
9 i += 1
10
11 j -= 1
12 while a[lo] < a[j]:
13 if j == lo: break
14 j -= 1
15
16 if i >= j: break
17 a[i], a[j] = a[j], a[i]
18
19 a[lo], a[j] = a[j], a[lo]
20 return j

Algorithms/ Sorting 58
 Partition
lo i j

1 50 27 37 53 14 59 67 70 34 80

lo i j

2 50 27 37 53 14 59 67 70 34 80

lo j i

3 50 27 37 34 14 59 67 70 53 80

lo j i

4 50 27 37 34 14 59 67 70 53 80

Algorithms/ Sorting 59
 Implementation
1 @patch
2 def _sort(self:Quicksort, a:list[int], \
3 lo:int, hi:int) -> None:
4 if hi <= lo:
5 return
6 j = self._partition(a, lo, hi)
7 self._sort(a, lo, j - 1)
8 self._sort(a, j + 1, hi)
9
10 @patch
11 def sort(self:Quicksort, a:list[int]) -> None:
12 self._sort(a, 0, len(a) - 1)

Algorithms/ Sorting 60
 Partition and sort

50 27 37 53 14 59 67 70 34 80

27 37 14 34 50 59 67 53 70 80

14 27 37 34 50 53 59 67 70 80

14 27 34 37 50 53 59 67 70 80

14 27 34 37 50 53 59 67 70 80

Algorithms/ Sorting 61
 Analysis
» In-place, not stable
» ∼ n log n average case
» ∼ n2 / 2 worst case

Algorithms/ Sorting 62
 Worst case?

1 2 3 4 5 6 7 8 9 10

1 2 3 4 5 6 7 8 9 10

1 2 3 4 5 6 7 8 9 10

1 2 3 4 5 6 7 8 9 10

1 2 3 4 5 6 7 8 9 10

Algorithms/ Sorting 63
 Improving the worst case?
» The worst case is extremely rare
» Ideally, we want the pivot to be the median
» Too expensive to compute (O(n))
» We can shuffle
» Or approximate the median from [lo, mid, hi]

Algorithms/ Sorting 64
 Implementation
1 @patch
2 def sort(self:Quicksort, a:list[int]) -> None:
3 random.shuffle(a)
4 self._sort(a, 0, len(a) - 1)

Algorithms/ Sorting 65
 Does it matter in reality?
Random arrays

Algorithms/ Sorting 66
 Does it matter in reality?
Worst case

Algorithms/ Sorting 67
 Does it matter in reality?
Worst and average case (shuffle)

Algorithms/ Sorting 68
 Heap vs merge vs quick

Algorithms/ Sorting 69
 Comparison-based sorts
» What is the lower bound of comparison-based
sorting?
» Compare each value with every other value
» Would suggest Ω(n2 )
» We know that some algorithms are O(n log n)
» Ω(n log n)?
» Would mean that merge and heap sort are
(asymptotically) optimal

Algorithms/ Sorting 70
 Comparison-based sorts
» How do we determine the lower bound?
» Sorting is a sequence of decisions
» a 0 < a 1 , a 1 > a 2 , … , a n−1 < a n
» How many decisions?

Algorithms/ Sorting 71
 Comparison-based sorts
a1< a2

a1< a2 a1< a3

<1,2,3> a1< a3 <2,1,3> a2< a3

<1,3,2> a1< a2
<3,1,2> a1< a2
<2,3,1> <3,2,1>

The maximal height of the tree is the number of comparisons performed in the worst case

Algorithms/ Sorting 72
 Comparison-based sorts
» Assume we sort the (distinct) numbers 1, 2, … n
» Since there are n! permutations, the decision tree
must contain n! leaves
» A binary tree of height h has at most 2h leaves
» So,
» 2h ≥ n! ⇒ h ≥ log(n!)
» log2 (n!) = n log2 n − n log2 e + O(log2 n) (Stirling’s
approximation)

Algorithms/ Sorting 73
 Really?

Algorithms/ Sorting 74
 Remember Lecture 2
First, we show that log n! is less than or equal to n log n.
This is true for all n > 0 .

log(n!) = log(1 ⋅ 2 ⋅ 3 ⋅ … ⋅ n)
= log 1 + log 2 + log 3 + … + log n
≤ log n + log n + log n + … + log n
= n log n

Algorithms/ Sorting 75
 Remember Lecture 2
Next, we show that log n! is greater than or equal to a
constant multiple of n log n.
n n n
log(n!) ≥ log + log( + 1) + log( + 2) + … + log n
2 2 2
n n n n
≥ log + log + log + … + log
2 2 2 2
n n n n n
= log = (log n − 1) = log n −
2 2 2 2 2
This is less than (n / 2) log n, so we pick a multiple less
than 1 / 2, for example 1 / 4. For n ≥ 4 ,

Algorithms/ Sorting 76
 Remember Lecture 2
log n ≥ 2
1 1
log n ≥
4 2
1 1
n log n ≥ n
4 2
1 1
n log n − n ≥ 0
4 2
1 1 1
n log n − n ≥ n log n
2 2 4

Algorithms/ Sorting 77
 Remember Lecture 2
n n
log(n!) ≥ log
2 2
n
= (log n − 1)
2
n n
= log n −
2 2
n
≥ log n
4
1
= n log n
4

Algorithms/ Sorting 78
 Remember Lecture 2
1
n log n ≤ log n! ≤ n log n
4
So, log n! = Θ(n log n)

Algorithms/ Sorting 79
 Radix sort

Algorithms/ Sorting 81
 “Counting” sorts
» We know that comparison-based sort is Ω(n log n)
» We can reduce this if we avoid comparing
» But how can we sort without comparing?
» We can count…

Algorithms/ Sorting 82
 Illustrating the idea
Step 1 Step 2 Step 3 Step 4
a count count aux a
0 5 0 0 0 0 0 1 1

1 6 1 0 1 0 0 keys < 1 1 1 1

2 5 2 2 2 2 2 keys < 2 2 5 5

3 1 3 0 3 2 So, 1 goes in 3 5 5
aux[0] and
4 5 4 0 4 2 4 5 5
aux[1]
5 5 5 0 5 2 5 5 5

6 8 6 4 6 6 6 6 6

7 1 7 1 7 7 7 8 8

8 0 8 7

9 1 9 8

0 8

Algorithms/ Sorting 83
 Implementation
1 def bucketsort(a:list[int], mx:int) -> None:
2 n = len(a)
3 cnt, aux = [0] * (mx + 1), [0] * n
4
5 for i in range(n):
6 cnt[a[i] + 1] += 1
7
8 for i in range(mx):
9 cnt[i+1] += cnt[i]
10
11 for i in range(n):
12 aux[cnt[a[i]]] = a[i]
13 cnt[a[i]] += 1
14
15 for i in range(n):
16 a[i] = aux[i]

Algorithms/ Sorting 84
 Testing
1 lst = random.choices(range(0, 10), k=10)
2 print(lst)
3 print(sorted(lst))
4 bucketsort(lst, 10)
5 print(lst)
[7, 9, 3, 4, 9, 7, 7, 0, 3, 4]
[0, 3, 3, 4, 4, 7, 7, 7, 9, 9]
[0, 3, 3, 4, 4, 7, 7, 7, 9, 9]

Algorithms/ Sorting 85
 Extending to characters/strings
» We can use the same idea to sort a list of strings
» We just to it character per character
» To keep it simple, we assume fixed length strings
» And 8-bit characters

Algorithms/ Sorting 86
 Illustrating the idea

d=2 d=1 d=0

1 e c e b a c c a d b f e

2 c a d c a d c b f c a d

3 f d b f f d e c e c b f

4 b a c e e e f d b e c e

5 f f d e c e e e e e e e

6 b f e b f e f f d f d b

7 c b f c b f b f e f f d

Algorithms/ Sorting 87
 Implementation
1 def radixsort(a:list[str]) -> None:
2 n, W = len(a), len(a[0])
3 aux = [0] * n
4
5 for d in range(W-1, -1, -1):
6 cnt = [0] * (256 + 1)
7
8 for i in range(n):
9 cnt[ord(a[i][d]) + 1] += 1
10
11 for i in range(256):
12 cnt[i+1] += cnt[i]
13
14 for i in range(n):
15 aux[cnt[ord(a[i][d])]] = a[i]
16 cnt[ord(a[i][d])] += 1
17
18 for i in range(n):
19 a[i] = aux[i]

Algorithms/ Sorting 88
 Testing it
1 lst = ['eee', 'ece', 'cad', 'fdb', 'bac', \
2 'ffd', 'bfe', 'cbf']
3 radixsort(lst)
4 assert is_sorted(lst) == True
5 print(lst)
['bac', 'bfe', 'cad', 'cbf', 'ece', 'eee', 'fdb', 'ffd']

Algorithms/ Sorting 89
 Analysis
» Not in-place, must be stable
» String length ⋅ number of strings
» O(w ⋅ n)
» Linear for short strings
» Can be effective for sorting, e.g., “personnummer”
(strings with 12 digits)

Algorithms/ Sorting 90
 Reading instructions

Algorithms/ Sorting 92
 Reading instructions
» Ch. 7.1 - 7.11

Algorithms/ Sorting 93