CHAPTER 03

Sorting Compiled by: Dr. Mohammad Omar Alhawarat

 Content
2

 Comparison Based Sorting:

 Bubble Sort.
 Insertion Sort.
 Selection Sort.
 Merge Sort.
 Quicksort
 Non-comparing sorts:
 Counting sort
 Radix Sort
 Bucket Sort
3

Comparison
Based Sorting
 Sorting
4

 Definition: Rearranging the values into a specific order:

(Ascending OR Descending).

 Sorting is important and is required in many

Applications, i.e., Searching.

 one of the fundamental problems in computer science

 can be solved in many ways:
 fast/slow
 use more/less memory
 depends on data
 utilize multiple computers / processors, ...
 Sorting
5

 Comparison-based sorting: determining order by comparing

pairs of elements.
‫ﺗﺣدﯾد اﻟﺗرﺗﯾب ﻣن ﺧﻼل ﻣﻘﺎرﻧﺔ أزواج اﻟﻌﻧﺎﺻر‬.

 An internal sort requires that the collection of data fit

entirely in the computer’s main memory.

 We can use an external sort when the collection of data

cannot fit in the computer’s main memory all at once but
must reside in secondary storage such as on a disk.

 We will analyze only Comparison-based and internal sorting

algorithms.
 Bubble Sort
6

 Idea:
 Repeatedlypass through the array
 Swaps adjacent elements that are out of order

 Easy to implement, but slow O(N2)

 Example
7
 Example
8
 Example
9
 Example
10
 Bubble Sort – Analysis
11

 Worst-case: O(n2)
 Array is in reverse order:

 Average-case: O(n2)
 We have to look at all possible initial data organizations.

 So, Bubble Sort is O(n2)

 Insertion Sort
12

 Insertion sort is a simple sorting algorithm that is

appropriate for small inputs.

 The list is divided into two parts: sorted and unsorted.

 In each pass, the first element of the unsorted part is

picked up, transferred to the sorted sublist, and
inserted at the appropriate place.

 A list of n elements will take at most n-1 passes to

sort the data.
 Example
13
 Insertion Sort – Analysis
14

 Worst-case: O(n2)
 Array is in reverse order:

 Average-case: O(n2)
 We have to look at all possible initial data organizations.

 So, Insertion Sort is O(n2)

 Selection Sort
15

 Idea:
 Find the smallest element in the array

 Exchange it with the element in the first position

 Find the second smallest element and exchange it with

the element in the second position

 Continue until the array is sorted

 Example
16

8 4 6 9 2 3 1 1 2 3 4 9 6 8

1 4 6 9 2 3 8 1 2 3 4 6 9 8

1 2 6 9 4 3 8 1 2 3 4 6 8 9

1 2 3 9 4 6 8 1 2 3 4 6 8 9
 Selection Sort – Analysis
17

 Worst-case: O(n2)
 Array is in reverse order:

 Average-case: O(n2)
 We have to look at all possible initial data organizations.

 So, Selection Sort is O(n2)

 Merge sort
18

 Idea:
 Isbased on “Merging” idea where two sorted lists are
combined in the right order.

 The start point is to consider each element in the list as

an ordered small list.
 The result is a list of two-element sorted lists.

 Repeatedly combine the ordered list until having one

list
 Merging Algorithm
19

Merging two ordered lists:

1. Access the first item from both lists
2. While neither sequence is finished
1. Compare the current items of both
2. Copy smaller current item to the output
3. Access next item from that input sequence
3. Copy any remaining from first sequence to output
4. Copy any remaining from second to output
 Example of Merging
20
 Example: Merge sort
21
 Merge sort – Analysis
22

 Worst-case: O(N LogN)

 Array is in reverse order:

 Average-case: O(N LogN)

 We have to look at all possible initial data organizations.

 So, Merge sort Sort is O(N LogN)

 But, merge sort requires an extra array whose

size equals to the size of the original array.
 Quicksort
23

 Idea:
 Repeatedly partition the data into two halves.
 Only the element in the middle is sorted.
 After (Log2N) repetitions then the data is sorted.

 Advantage: One of the practically best sorting

Algorithms [O(N Log2N)] in the average case.

 Drawbacks: O(N2) in the worst case.

 Quicksort
24

 A more efficient exchange sorting

scheme than bubble sort because
a typical exchange involves
elements that are far apart
fewer interchanges are required to
correctly position an element.
 Quicksort
25

 Quicksort uses a divide-and-conquer

strategy
 a recursive approach to problem-solving
in which the original problem partitioned
into simpler sub-problems,
 each sub-problem considered independently.
 Subdivision continues until sub-problems obtained
are simple enough to be solved directly
 Quicksort
26

 Choose some element called a pivot

 Perform a sequence of exchanges so that
 all elements that are less than this pivot are
to its left and
 all elements that are greater than the pivot
are to its right.
 divides the (sub-)list into two smaller sub-
lists, each of which may then be sorted
independently in the same way.
 Quicksort
27

1. If the list has 0 or 1 elements,

return. // the list is sorted
Else do:
2. Pick an element in the list to use as the pivot.

3. Split the remaining elements into two disjoint groups:

SmallerThanPivot = {all elements < pivot}
LargerThanPivot = {all elements > pivot}

4. Return the list rearranged as:

Quicksort(SmallerThanPivot), pivot,
Quicksort(LargerThanPivot).
 Quicksort Example
28

 Given 75, 70, 65, 84, 98, 78, 100, 93,

55, 61, 81, 68 to sort
 Select, arbitrarily, the first element, 75,
as pivot.
 Search from right for elements <= 75,
stop at first element <=75
 Search from left for elements > 75, stop
at first element >75
 Quicksort Example
29

 Swap these two elements, and then repeat two

elements same
75, 70, 65, 84, 98, 78, 100, 93, 55, 61, 81, 68
75, 70, 65, 68, 98, 78, 100, 93, 55, 61, 81, 84
75, 70, 65, 68, 98, 78, 100, 93, 55, 61, 81, 84
75, 70, 65, 68, 61, 78, 100, 93, 55, 98, 81, 84
75, 70, 65, 68, 61, 78, 100, 93, 55, 98, 81, 84
75, 70, 65, 68, 61, 55, 100, 93, 78, 98, 81, 84
75, 70, 65, 68, 61, 55, 100, 93, 78, 98, 81, 84
done, swap with pivot
 Quicksort Example
30

55, 70, 65, 68, 61, 75, 100, 93, 78, 98, 81,
84
 The previous SPLIT operation placed pivot 75
so that all elements to the left were <= 75
and all elements to the right were >75.
 75 is now placed appropriately
 Need to sort sub-lists on either side of 75
55, 70, 65, 68, 61, 75, 100, 93, 78, 98, 81,
84 pivot 75
 Quicksort Example
31

 Need to sort
(independently):
55, 70, 65, 68, 61
100, 93, 78, 98, 81, 84
 Quicksort performance
32

Quicksort performance:
O(nlogn) if the pivot results in sublists
of approximately the same size.

O(n2) worst-case (list already

ordered, elements in reverse) when
Split() repetitively results, for
example, in one empty sublist
 Quicksort performance
33

 It can be shown that even if the split

result in 90% in one partition and
10% in the other partition the
complexity of Quicksort will be O(n
log n).
90% 10%
 Strategies
34

We could try to ensure constant proportionality of splits,

but that seems much harder than simply making sure they
are never too bad, or almost never.
If we choose good and bad splits at random, the overall
effect is as if they were good.
So, use a random number generator to pick a pivot and
swap it with the item at A[0]. Partition itself need not
change:
Randomised-Partition(A, low, high)
1. i  Random(low, high)
2. swap A[0] with A[i]
3. return Partition(A, low, high)
Now we have one major advantage: no particular input
can be predicted to create the worst case.
 Strategies
35

We could try to ensure constant

proportionality of splits, but that seems
much harder than simply making sure they
are never too bad, or almost never.
If we choose good and bad splits at
random, the overall effect is as if they
were good.
So, use a random number generator to
pick a pivot and swap it with the item at
A[0]. Partition itself need not change:
 Strategies
36

Randomised-Partition(A, low, high)

1. i  Random(low, high)
2. swap A[0] with A[i]
3. return Partition(A, low, high)

Now we have one major advantage:

no particular input can be predicted to
create the worst case.
 Quicksort Improvement I
37

 Quicksort is a recursive function

 stack of activation records must be maintained by system to
manage recursion.
 The deeper the recursion is, the larger this stack will become.
 The depth of the recursion and the corresponding overhead
can be reduced
Sort the smaller sublist at each stage first
 Another improvement aimed at reducing the overhead
of recursion is to use an iterative version of Quicksort()
 To do so, use a stack to store the first and last positions
of the sublists sorted "recursively".
 Quicksort Improvement II
38

 An arbitrary pivot gives a poor partition for nearly

sorted lists (or lists in reverse)
 virtually all the elements go into either SmallerThanPivot or LargerThanPivot
 all through the recursive calls.
 Quicksort takes quadratic time to do essentially nothing at all.
 One common method for selecting the pivot is the
median-of-three rule,
 select the median of the first, middle, and last elements in each sublist as
the pivot.
 Often the list to be sorted is already partially ordered
 median-of-three rule will select a pivot closer to the middle of the sublist
than will the “first-element” rule.
 Quicksort Improvement III
39

 For small files (n <= 20), quicksort is worse than

insertion sort;
 small files occur often because of recursion.
 Use an efficient sort (e.g., insertion sort) for small
files.
 Better yet, use Quicksort() until sublists are of a
small size and then apply an efficient sort like
insertion sort.
 Split
40

template <typename ElementType>

void Split(ElementType x[],int first, int last, int& pos)
(
ElementType pivot = x[first]; // pivot element
int left = first, // index for left search
right = last; // index for right search
while (left < right)
{
while (x[right] > pivot) // Search from right for element <= pivot
right--;
while (left < right && x[left] <= pivot) // Search from left for element > pivot
left++;
// Interchange elements if searches haven’t met
if (left < right)
Swap(x[left], x[right]);
}
// End of searches; place pivot in correct position
pos = right;
x[first] = x[pos];
x[pos] = pivot;
}
 Quicksort
41

template <typename ElementType>

void Quicksort(ElementType x[], int first, int last)
{
int pos; // final position of pivot
if (first < last) // list has more than one element
{
// Split into two sublists
Split(x, first, last, pos);
// Sort left sublist
Quicksort(x, first, pos - 1);
// Sort right sublist
Quicksort(x, pos + 1, last);
}
// else list has 0 or 1 element and requires no sorting
return;
}
This function is called with a statement of the form
Quicksort(x, 1, n);
 Comparison Sorts
42

 All the sorts we have looked at up

until now are comparison sorts; i.e.
they are based only on
comparisons between the input
elements.
 Consider a decision tree for sorting

3 elements: a1, a2, and a3

 Comparison Sorts
43
 Comparison Sorts
44

There are n! possible permutations, all

of which must appear at leaves. A tree
of height h has at most 2h leaves, so n!
 2h
It can be shown that n log n = O(h)
Hence any comparison sorting algorithm
cannot be better than
O(n log n).
45

Non-comparing
sorts
 Non-comparing sorts
46

What about sorts that don’t compare elements (or at least not much)?
e.g. Counting Sort
1. for i  0 to k - 1 do
2. C[i]  0
3. for j  0 to n - 1 do
4. C[A[j]]++
5. for i  1 to k - 1 do
6. C[i] += C[i - 1]
7. for j  n - 1 down to 0 do
8. B[C[A[j]] - 1]  A[j]
9. C[A[j]]- -
If each for loop has to do around n things (i.e. k =n), then counting sort
time = O(n).
 Counting sort
47

 Which type of data can be sorted using counting

sort?
 Are there any restrictions on range of data?

 Try to sort
 12, 4, 2, 13, 6, 9, 5, 3, 8, 7
 Radix Sort
48

 Used by punch-card operators!

1. for i d downto 1 do
2. Use a stable sort to sort A on digit i
 If we use counting sort, each pass is O(n).
There are d passes, so if d is constant then
Radix sort is O(dn), which is O(n).
 Try on 329, 457, 657, 839, 436, 720, 355
 Note start from left to right
 More Radix Sort
49

 Consider sorting 1 million 64-bit numbers.

 Treat as 4 digit numbers of 16-bits each.

 Sort in 4 passes of counting sort = radix sort.

 Compare to n log n quicksort: 20 operations

per number.
 Disadvantages:
 requires some uniformity of number.
 Waste of storage (more than one copy, empty slots in the
array)
 Bucket Sort
50

 Counting sort assumes that the input is a bunch of

integers in a small range.
 Bucket sort assumes that the elements are distributed
uniformly over an interval.
 For n keys, create n buckets of even sub-intervals.
 Calculate which bucket to place each key into and
transfer.
 Sort buckets with insertion sort.
 Either copy back to the original list, or spit straight out.
 Bucket Sort
51

1. n  length of A
2. for i  0 to n - 1 do
3. insert A[i] into B[n * A[i] ]
4. for i  0 to n - 1 do
5. sort bucket B[i] with insertion sort
6. Concatenate the lists B[0] up to B[n - 1]
together in order
 Consider tossing n balls into n buckets:
chances are good that the buckets stay
small.
 Tricky aspects of Bucket Sort
52

 Textbook suggests linked lists for buckets.

 What are the advantages or
disadvantages of other data structures?
 What happens when the keys are not
over the interval [0,1)?
 What kind of data will make bucket sort
behave badly?