Sorting Algorithms

Sorting Algorithms
InsertionSort, BubbleSort, HeapSort
MergeSort, QuickSort, BucketSort
Complexity of Sorting Algorithms
Lower Bound of Problem
Optimal Algorithms

Motivation
Some problems are easy to solve and some
are not. How do we measure the difficulty of
a problem?
有些問題很容易解決，有些則不然。我們如何衡量一
個問題的困難度?
How do we know that an algorithm is optimal
for a problem?
對於某個問題來說，我們又如何知道一個演算法是最
佳的?
ISU/CSIE 2

1
 Insertion Sort (插入排序法)
Algorithm Insertion Sort 7 5 1 4 9
input: X=(x1,x2, ... ,xn)
output: sorted sequence of X
7 5 1 4 9
for j=2 to n do
i=j-1;
j=2 5 7 1 4 9
x=xj;
while (x<xi and i>0) do
j=3 1 5 7 4 9
xi+1=xi;
i=i-1;
j=4 1 4 5 7 9
end while
xi+1=x;
end for j=5 1 4 5 7 9

ISU/CSIE 3

Insertion Sort (cont.)

Time Complexity
Worst case: when inverse order 9 8 7 5 3 2

Tw(n)=1+2+ … +(n-1)=(n2)
Best case: when preorder 2 3 5 7 8 9

Tb(n)=1+1+ … +1=(n)
*Average case
Ta(n)=(n2)
Time Complexity=O(n2)
ISU/CSIE 4

2
 Bubble Sort (氣泡排序法)
STEP 1 STEP 2 STEP 3 STEP 4
8 3 9 6 1 3 8 6 1 9 3 6 1 8 9 3 1 6 8 9

3 8 9 6 1 3 8 6 1 9 3 6 1 8 9 1 3 6 8 9

3 8 9 6 1 3 6 8 1 9 3 1 6 8 9

3 8 6 9 1 3 6 1 8 9
Time Complexity
3 8 6 1 9 T(n)=(n-1)+(n-2)+…+1=O(n2)

STEP 1: Algorithm Bubble Sort

Check (1,2),(2,3), … ,(n-2,n-1),(n-1,n)
input: X=(x1,x2, ... ,xn)
STEP 2:
output: sorted sequence of X
Check (1,2),(2,3), … ,(n-2,n-1)
… for i=1 to n-1 do
STEP n-1: for j=1 to n-i do
Check (1,2) if xj>xj+1 then swap(xj,xj+1) 7

Heap Sort (堆積排序法)

A max-heap: a binary tree where parent  son
Algorithm HeapSort O(nlogn)
1. Build a heap of n elements O(n)
2. Perform n times of DelMax operations O(nlogn)

14

10 6 14 10 6 7 8 4 1 6 2 3 5

7 8 4 1
one–dimensional array
6 2 3 5

ISU/CSIE 8

3
 Phase 1: Heap Construction
Input data: for example, (4,37,26,15,48)
Restore the subtree rooted at A(2):
Restore the tree rooted at A(1):

T(n)=1h+2(h-1)+22(h-2)+ … + 2h-11
2*T(n)= 2h +22(h-1)+ … + 2h-12+2h1
-----------------------------------------------------
T(n)=2+22+…+2h-1+2h-h=2h+1-2-h=O(n)
ISU/CSIE 9

Phase 2: Output
 Output and Delete the maximum
 Replace the bottom by the root
 Reconstruct the heap

Time Complexity = n*O(logn) = O(nlogn)

ISU/CSIE 10

4
 Merging (兩數列的合併)
Merging: given two sorted lists, merge
them into one sorted list.
Example
A=(1,13,24,26)
B=(2,15,27,38)
C=Merge(A,B)=(1,2,13,15,24,26,27,38)
Running Time
Let #(A)=m and #(B)=n
Time complexity=O(m+n)
ISU/CSIE 11

Merge Sort (合併排序法)

3 4 7 2 6 1 5 8

3 4 2 7 1 6 5 8

2 3 4 7 1 5 6 8

1 2 3 4 5 6 7 8

Analysis
T(n)=2*T(n/2)+O(n), i.e. T(n)=O(nlogn)
Comment
Linear extra memory (需要一份額外的儲存空間)
Require copying to temporary array and back 12

5
 Quick Sort (快速排序法)
Algorithm QuickSort(S)
if #(S)=0 or 1, then return
pick vS
Partition S-{v} into S1={S-{v}|xv} and S2={S-{v}|xv}
return QuickSort(S1), v, QuickSort(S2)

11 5 24
24+ 2 31 7 8 26 10
10- 15

11 5 10 2 31+ 7 8- 26 24 15

11 5 10 2 8 7- 31+ 26 24 15

7 5 10 2 8 11 31 26 24 15
ISU/CSIE 13

Quick Sort (cont.)

Picking the pivot (如何挑選標竿)
The first element
Random choice
Median of three elements
Running Time
Worst case
Tw(n)=Tw(n-1)+(n), i.e. Tw(n)=(n2)
Best case
Tb(n)=2Tb(n/2)+(n), i.e. Tb(n)=(nlogn)
*Average case: Ta(n)=(nlogn)
Time Complexity=O(n2)
14

6
 Bucket Sort (桶子排序法)
Problem: Given a1,a2,…,an (0<aim, aiN), sort a1~an.
Algorithm
step 1: count[1..m]=0
step 2: for i=1 to n do count[ai]++;
step 3: for i=1 to m do
for j=1 to count[i] do output(i);

3 1 4 2 5 1 1 2 2 3
4 4 5 2 4 4 4 4 4 4
1 6 6 4 8 1 2 3 4 5 6 7 8
5 5 6 6 8

Time Complexity=O(m+n)
ISU/CSIE 15

Lower Bound of Problem

A lower bound (下限) of a problem is the least
time complexity required for any algorithm
which can be used to solve this problem.
(1), (n) and (nlogn) are all lower bounds
for sorting.
Theorem: Any sorting algorithm that uses
only comparisons requires (nlogn).

ISU/CSIE 16

7
 *Decision Tree for Sorting
a<b<c a<c<b b<a<c
b<c<a c<a<b c<b<a

a:b
< >
a<b<c b<a<c
a<c<b b<c<a
c<a<b b:c b:c c<b<a
< > a<c<b b<a<c < >
c<a<b b<c<a
a<b<c a:c a:c c<b<a

< > < >

a<c<b c<a<b b<a<c b<c<a

ISU/CSIE 20

*Decision Tree for Sorting

If #(leaves)=m, height=(logm)

Here, m=n!
If #(leaves)=n!, height=(log(n!))
=(nlog(n))
ISU/CSIE 21

8
 Optimal Algorithm (最佳演算法)
If the highest lower bound of a problem is (nlogn)
and the time complexity of the best algorithm is O(n2).
We may try to find a higher lower bound.
We may try to find a better algorithm.
Both of the lower bound and the algorithm may be improved.

If the lower bound is (nlogn) and there is an

algorithm with time complexity O(nlogn), then the
algorithm is optimal.

MergeSort and HeapSort is optimal!

ISU/CSIE 26

Exercise
A sorting algorithm is stable if the relative order
of items with equal keys remains unchanged by
the sorting process. (相同數值在排序中前後保持不變)
For example,
Original List = (3, 2, 7, 4, 1, 5, 9, 8, 2)
Sorted List = (1, 2, 2+, 3, 4, 5, 7, 8, 9)
Which of the following sorting methods are stable?

Insertion Sort Bubble Sort Quick Sort Merge Sort Heap Sort

ISU/CSIE 27