Data structures and algorithms

Spring 2024
Sorting Algorithms II
Lecturer: Do Thuy Duong
 Contents
• Quick sort
• Heap sort
• Linear time sorting algorithm
• Counting sort (Bucket sort)
• Radix sort

2
 Quicksort[1]
Quick sort is a kind of divide and conquer
algorithm

1. Select a value as a pivot

2. Divide array of data into 2 parts: left <

pivot value, right > pivot value

3. Repeat (1) and (2) for the left part and

4
right part
3
 Quicksort[2]
For example, we simply use the first element (ie. 25) aspivot
for partitioning: a[0..N-1]
Eg. 25 10 57 48 37 12 92 86 33
=> 12 10 25 48 37 57 92 86 33
pivot
After the partitioning step, we received two sub-lists, the
left contains all elements < pivot, the right contains all
elements
> pivot.
After this step pivot is in its correct position. 5
We repeat sorting for the left and the right sub-list.
4
 Quicksort[3]
Original

6
Sorted

5
 Quicksort[4]
Algorithm to partition the array:
1. Having 2 indices of sorting range: 1st and the
last called down index andup index
2. Move down towards up until a[down]>pivot
3. Move up towards down until a[up]<=pivot
4. Swapa[up] and a[down] and repeat from
step 2 until up<=down
5. Swappivot and a[up], then return up 7

6
 Quicksort[5]
Original: 25 10 57 48 37 12 92 86 33
Partitioning: Select pivot= 25. Use 2 indices:
down up
25 10 57 48 37 12 92 86 33
down up
25 10 57 48 37 12 92 86 33 *
down up
25 10 12 48 37 57 92 86 33 Swap a[down] and a[up]

up down
Continue repeat (*) until up
25 10 12 48 37 57 92 86 33 crosses down (ie. down >= up)

up is at right-most of smaller
12 10 25 48 37 57 92 86 33 partition, so swap pivot with a[up]

7
 Quicksort[6]
Code

Algorithm QuickSort(A, left, right):

Input: An array A storing integer elements.
Output: A is sorted in ascending order from left to right

if left < right then

pos Partition(A, left, right)
QuickSort(A, left, pos-1)
QuickSort(A, pos+1, right)

10

8
 Quicksort[7]
Code

Algorithm Partition(A, left, right):

Input: Sub-array A from left to right
Choose a[left] as the pivot
Output: Rearranged sub-array so that the left part is
the elements < pivot, the right part is the elements >
pivot. New position of pivot is returned.

down  left up  right pivot  a[left]

while down < up

while a[down]<=pivot
down++
while a[up]>pivot
up--
tmp  a[down] a[down]  a[up] a[up]  tmp
endwhile 9
a[left]  a[up] a[up]  pivot
return up
9
 Quicksort[8]
Analysis of Quicksort:
• See textbook section 7.7.5

• Suppose that the partitioning divides the original

problem A into two sub-problems A1 and A2 with
the size of A1 is i and the side of A2 is N-1-i.

T(N) = T(N-1-i) + T(i) + cN

• We consider the following cases

11

10
 Quicksort[9]
• The worst-case:
The partitioning produces one sub-problem with
n-1 elements and one with 0 element (i=0).
T(N) = T(N-1) + cN

• The time complexity is O(T(n)) = O(n2)

• The best-case:
Every time the partitioning produces two sub-
problems that have equal size, i=n/2.
T(N) = 2T(N/2) + cN
12
• The time complexity is O(T(n)) = O(nlogn)

11
 Quicksort[10]
• The average-case:
Assume the size i of A1 appears with the
probability 1/N. Then we have:

• The time complexity is O(T(n)) = O(nlogn)

13

12
 Heap sort[1]
The binary heap is: a data structure that is an array object and can be
represented asabinary tree.
- Shape: A complete binary tree (all levels of the tree, except possibly
the last one are fully filled; if the last level of the tree is not complete, the
nodes of that level are filled from left to right.)
- Node value: min/max heap

0 0
0 1 2 3 4 5 6 7 8 9 16
1
1 2 1 2
1 2 7 10 3 4 14 9 16 8 14 10
2 7
3 4 5 6 3 4 5 6
10 3 4 14 8 7 9 3
7 8 9 7 8 9
9 16 8 2 4 1

13 Heap Max Heap

14
 Heap sort

0 0
1 16
0 1 2 3 4 5 6 7 8 9 1 2 1 2
2 14 10
7
1 2 7 10 3 4 14 9 16 8 3 4 3 4 5 6
5 6
10 3 4 14 8 7 9 3
7 8 9 7 8 9
9 16 8 2 4 1
Parent(i) =  (i-1)/2 
Left(i) = 2i+1 Max Heap
Right(i) = 2i+2
child <= parent
Root is largest
Min Heap

child >= parent

Root is smallest

14
 Heap sort[2]
• Heap sort algorithm:
1. Build max/min heap: convert the array to a
max heap (or min heap) (heap has N nodes)
2. Sorting:
• While the heap has more than 1node
-Swapthe root and the last leaf (the first and the last
element of the array). Reduce the size of the heap by
one
- Re-arrange remaining elements to build a max
heap. (build a heap with N-1 nodes)

15
 Heap sort[3]
1. How to build max heap?
• Convert an array of unsorted elements into a max heap,

• In a max-heap, each sub-tree must be a max-heap too.

• Except leaf nodes, all other subtrees may not fulfill the
requirement for a max-heap.
trickle-down all non-leaf nodes one by one, starting from 21
the lowest ones to the top. ie. 3, 10, 7, 2, 1
16
 Heap sort[4]
How to trickledown
• The left and right sub-trees of the root are max- heaps.
But the root may be smaller than its children.
• Step 1: We determine the largest between: root, left child
and right child of root.
• If root is the largest, then its position is correct.
Otherwise, swap root and the largest. Repeat Step 1 again.
Example: Trickle down 1
0 0 0 0
1 14 14 14
1 2 1 2 1 2 1 2
14 10 1 10 8 10 8 10
3 4 5 6 3 4 5 6 3 4 5 6 3 4 5 6 19
8 7 9 3 8 7 9 3 1 7 9 3 4 7 9 3
7 8 7 8 7 8 7 8
17 2 4 2 4 2 4 2 1
 Heap sort[5]
• In any complete binary 1. Trickle down 3
tree, the last ⎡N/2⎤
1
elements are leaf nodes.
The first⎣N/2⎦are not. 2 7

10 3 4 14
.. ..
9 16 8
• Trickle down the first
⎣N/2⎦ elements.
Starting from the
1
lowest ones to the top.
1
2 7
5
2 7 10 8 4 14
4 3
10 3 4 14
2 1 9 16 3
9 16 8
18
 Heap sort[6]
• In any complete binary 2. Trickle down 10
tree, the last ⎡N/2⎤
1
elements are leaf nodes.
The first⎣N/2⎦are not. 2 7

10 8 4 14
.. ..
9 16 3
• Trickle down the first
⎣N/2⎦ elements.
Starting from the
lowest ones to the top. 1

1 2 7
5
2 7
4 3 16 8 4 14
10 3 4 14
2 1 9 10 3
9 16 8
19
 Heap sort[7]
• In any complete binary 3. Trickle down 7
tree, the last ⎡N/2⎤
elements are leaf nodes. 1
The first⎣N/2⎦are not.
2 7

16 8 4 14
.. ..
9 10 3
• Trickle down the first
⎣N/2⎦ elements.
Starting from the
lowest ones to the top. 1
1
5 2 14
2 7
4 3 16 8 4 7
10 3 4 14
2 1
9 10 3
9 16 8
20
 Heap sort[8]
4. Trickle down 2
• In any complete binary
tree, the last ⎡N/2⎤ 1
elements are leaf nodes.
2 14
The first⎣N/2⎦are not.
16 8 4 7 1

9 10 3 16 14
.. ..
2 8 4 7
• Trickle down the first
⎣N/2⎦ elements. 9 10 3
Starting from the
lowest ones to the top. 1
1
5 16 14
2 7
4 3 10 8 4 7
10 3 4 14
2 1
9 2 3
9 16 8
21
 Heap sort[9]
5. Trickle down 1
• In any complete binary
tree, the last ⎡N/2⎤
1 16
elements are leaf nodes.
The first⎣N/2⎦are not. 16 14 1 14

10 8 4 7 10 8 4 7
.. ..
9 2 3 9 2 3
• Trickle down the first
⎣N/2⎦ elements.
Starting from the
lowest ones to the top. 16 16

1 10 14 10 14
5
2 7
4 3
9 8 4 7 1 8 4 7
10 3 4 14
2 1 1 2 3 9 2 3
9 16 8
 Heap sort[10]
2. Sorting process:
2.1. Swap the root and the last leaf (the first and the last
element of the array). Reduce the size of the heap by
one
2.2. Trickle down the new root (1) to a correct
position (in order to build max heap again).
Example:
0
16 1 14
1 2
14 10 14 10 8 10
3 4 5 6 18
8 7 9 3 8 7 9 3 4 7 9 3
7 8 9
2 4 1 2 4 16 2 1 16
23
 Heap sort [11]
• Repeat step 2.1 & 2.2 until the heap has only
one node

16 14 10 9 8
1 2
14 10 8 10 8 9 8 3 7 3
3 4 5 6
8 7 9 3 4 7 9 3 4 7 1 3 4 7 1 2 4 2 1 9
7 8 9
2 4 1 2 1 16 2 14 16 10 14 16 10 14 16

7 4 3

4 3 2 3 2 1

1 2 8 9 1 7 8 9 4 7 8 9

10 14 16 10 14 16 10 14 16

24
 Heap sort[12]
Heap sort algorithm
Code

Algorithm Heapsort(A):
Input: Array unsorted A
Output: A is sorted

n  a.length

BuildHeap(a,n)

For i  n downto 2 do

tmp  a[1] a[1]  a[i] a[i]  tmp

n  n-1
TrickleDown(A,n,0) 24

Endfor
25
 Heap sort[13]
Build heap algorithm
Code

Algorithm BuildHeap(A, n)
Input: Array A of the size n, starting
from 1
Output: A becomes max heap.
For i  n/2 downto 1 do

TrickleDown(A,n,i)

Endfor

23

26
 Heap sort[14]
Trickle down algorithm
Code

Algorithm TrickleDown(A, n, i):

Input: The binary tree A of the size n starting from 1,
i is the root of the tree
Output: Tree A becomes max heap.

l  LeftChild(i) r  RightChild(i) maxpos  i

If (l<=n) and (a[l]>a[maxpos]) then

maxpos  l
If (r<=n) and (a[r]>a[maxpos]) then
maxpos  r

If (maxpos != i) then
tmp  a[i] a[i]  a[maxpos] a[maxpos]  tmp
TrickleDown(A, n, maxpos)
Endif 20

27
 Heap sort[15]
Time complexity analysis of heapsort
• Reference book chapter 6 Heapsort
• Running time = Time to build max heap + (n-1) *
Time to trickle down
• Time to build max heap is O(n)
• Time to trickle down is O(logn).
• The time complexity of heap sort is:
O(T(n)) = O(n)+(n-1)O(logn) = O(nlogn)
25

28
 Lineartimesortingalgorithm[1]
• All sorting algorithm introduced so far
shared the sameproperty:
• The sorted order is determined based only on
the comparisons between input elements.

• They are called comparison sort.

• The lower bound time complexity for any
comparison short is Ω(nlogn) for the worst case
(see chapter 8.1 reference book). It means the
fastest comparison sort algorithm has the time 26
complexity O(nlogn).
29
 Linear timesortingalgorithm[2]
• There are other sorting algorithms that used
operations other than comparisons to determine
the sorted order
• Counting sort

• Radix sort

• They can sort n elements in the timeO(n)

• They are called linear time sorting algorithm

27

30
 Counting sort[1]
• We want to sort n integers, each integer is in the
range [1..k].
• We determine, for each input element x, the number of
elements less than x.

• This information can be used to place directly into its

correct position. For example, if there 17 elements less
than x, than x belongs in output position 18.

• For example, we want to sort array A[1..5], where

A[ j]{1, 2, 3, 4} . We use array B[1..5] tostore the
sorted elements, array C[1..4] to store counting
28
information
31
 Counting sort[2]
• How many times does A[i] appear in A
1 2 3 4 5 1 2 3 4
A: 4 1 3 4 3 C: 0 0 0 1
1 2 3 4 5 1 2 3 4
A: 4 1 3 4 3 C: 1 0 0 1
1 2 3 4 5 1 2 3 4
A: 4 1 3 4 3 C: 1 0 1 1
1 2 3 4 5 1 2 3 4
A: 4 1 3 4 3 C: 1 0 1 2
1 2 3 4 5 1 2 3 4

A: 4 1 3 4 3 C: 1 0 2 2 30

32
 Counting sort[3]
• How many times does A[i] appear in A
Code

For i  1 to n do

C[A[i]]  C[A[i]]+1

Endfor

31

33
 Counting sort[4]
• C[x] is the number of elements equal to x

• Now we count the number of elements ≤ x(C[x] is the

number of elements ≤ x)
1 2 3 4 5
A: 4 1 3 4 3
1 2 3 4 1 2 3 4

C: 1 0 2 2 C: 1 1 3 2

1 2 3 4 1 2 3 4
C: 1 1 2 2 C: 1 1 3 5

34
 Counting sort[5]
• Count the elements ≤ A[i] in A
Code

For i  2 to k do

C[i]  C[i]+C[i-1]

Endfor

33

35
 Counting sort[6]
• How to arrange A[i] into B

36
 Counting sort[7]
• How to arrange A[i] into B

35

37
 Counting sort[8]
• How to arrange A[i] into B

• Now B is storing sortedelements

36

38
 Counting sort[9]
• How to arrange A[i] into B
Code

For i  n downto 1 do

B[C[A[i]]]  A[i]
C[A[i]]  C[A[i]]-1

Endfor

37

39
 Counting sort [10]

40
 Counting sort[11]
• Time complexity of counting sort algorithm is
O(n+k)
• Doesn’t compare the elements of array A

• Counting sort is a stable sort: it preserves the

input order among equal elements.

39

41
 Radixsort [1]
• Counting sort is inefficient when k is big:
• k=1000, 10000, or 100000 …

• We want to sort an array of n elements, each

element is an integer which has maximum k digits
• Sort the elements by its digits, one by onefrom right to
left.

• Use a stable sorting algorithm to sort elementsby

digits.

• The algorithm is called radix sort or column sort 40

42
 Radixsort [2]
• Radix sort example: n=7;k=3;

329 720 720 329

457 355 329 355
657 436 436 436
839 457 839 457
436 657 355 657
720 329 457 720
355 839 657 839
41

43
 Radixsort [3]
• Radix sort algorithm
Code

Algorithm RadixSort(A,n,k):
Input: Array unsorted A of the size n,
a[i] has maximum k digits. Digit 1 is the
right most and digit k is the left most.
Output: A is sorted

For i  1 to k do

Column_sort(A,n,i)
//use a stable sort to sort A on digit i
42
Endfor
44
 Radixsort [4]
• Time complexity of radix sort algorithm
• N elements, each element has k digits, each digit
is in the range [0..9]. Using counting sort as the
stable sorting algorithm:
O(n)=k.O(n+10)=O(k(n+10))

• N elements, each element has p digits in base b,

each digit is in the range [0..b-1]. Using counting
sort as the stable sorting algorithm:
O(n)=p.O(n+b)=O(p(n+b))
43

45
 Radixsort [5]
• Does radix sort algorithm work on negative
number or non-integer number?
• The answer is yes, but we need some modification and a
proper representation of the elements

44

46
 Bestsortingalgorithm?
• Linear time sorting algorithm
• Fastest,but need special input data
• The fastest comparison sorting algorithms has
O(N.logN)
• For large N (say >1000), Quicksort is faster, on most
machines, by a factor of 1.5 or 2. However, it requires a bit
of extra memory and is a moderately complicated program.
• Heap sort is a true “sort in place”, and is more compact to
program.
• For N < 50, an O(N2) sorting algorithm, is concise andfast
enough.
• For 50<N<1000, Shell sort is usually the method of choice 45
(see section 7.4 text book). It is O(N3/2) as in the worst case,
but is usually faster.
47
 Tutorial & next topic
• Preparing for the tutorial:
• Practice with examples and exercises in Tutorial 5
• Preparing for next topic:
• Read textbook chapter 3 List.

48